AITech/aitech-ium
112
8
Fork 2
forked from tzietkiewicz/aitech-ium
Code Issues Pull Requests Releases Wiki Activity

Compare commits

base: AITech:eb983c3ddd065a97e07434270f6a61383f33a1e2
Branches Tags
AITech:master
tzietkiewicz:master
tzietkiewicz:sacred-mongo
tzietkiewicz:zaoczne
..
compare: AITech:bfac49d4fedacc75e1f9a19fd028670e9db4e686
Branches Tags
AITech:master
tzietkiewicz:master
tzietkiewicz:sacred-mongo
tzietkiewicz:zaoczne

No commits in common. "eb983c3ddd065a97e07434270f6a61383f33a1e2" and "bfac49d4fedacc75e1f9a19fd028670e9db4e686" have entirely different histories.
eb983c3ddd ... bfac49d4fe

8 changed files with 454 additions and 11260 deletions
Show Stats Expand all files Collapse all files

2
IUM_00.Organizacyjne.ipynb
View File

@ -180,7 +180,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.9.16" "version": "3.9.12"
}, },
"slideshow": { "slideshow": {
"slide_type": "slide" "slide_type": "slide"

2
IUM_03.CI-Jenkins.ipynb
View File

@ -509,7 +509,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.9.16" "version": "3.9.12"
}, },
"slideshow": { "slideshow": {
"slide_type": "slide" "slide_type": "slide"

2
IUM_06.Jenkins-2.ipynb
View File

@ -248,7 +248,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.9.16" "version": "3.9.12"
}, },
"slideshow": { "slideshow": {
"slide_type": "slide" "slide_type": "slide"

18
IUM_07.Sacred.ipynb
View File

@ -981,25 +981,13 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": null,
"metadata": { "metadata": {
"slideshow": { "slideshow": {
"slide_type": "slide" "slide_type": "slide"
} }
}, },
"outputs": [ "outputs": [],
{
"name": "stdout",
"output_type": "stream",
"text": [
"total 16\r\n",
"-rw-r--r-- 1 tomek tomek 77 May 28 2022 config.json\r\n",
"-rw-r--r-- 1 tomek tomek 159 May 28 2022 cout.txt\r\n",
"-rw-r--r-- 1 tomek tomek 2 May 28 2022 metrics.json\r\n",
"-rw-r--r-- 1 tomek tomek 1659 May 28 2022 run.json\r\n"
]
}
],
"source": [ "source": [
"!ls -l my_runs/1" "!ls -l my_runs/1"
] ]
@ -1567,7 +1555,7 @@
} }
}, },
"source": [ "source": [
"## Zadanie [15 pkt] (do 2023-06-?)\n", "## Zadanie [15 pkt] (do 2023-05-12)\n",
"1. \"Owiń\" wywołanie swojego eksperymentu za pomocą Sacred, w ten sposób, żeby zapisane zostały [10pkt]:\n", "1. \"Owiń\" wywołanie swojego eksperymentu za pomocą Sacred, w ten sposób, żeby zapisane zostały [10pkt]:\n",
" - parametry, z którymi wywołany był trening\n", " - parametry, z którymi wywołany był trening\n",
" - powstały plik z modelem (jako artefakt)\n", " - powstały plik z modelem (jako artefakt)\n",

8264
IUM_08.MLFlow.ipynb
View File

File diff suppressed because it is too large Load Diff

3143
IUM_09.Python_srodowiska.ipynb
View File

File diff suppressed because it is too large Load Diff

281
IUM_11.CML.ipynb
View File

@ -103,7 +103,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 16,
"id": "f4916c1f", "id": "f4916c1f",
"metadata": { "metadata": {
"slideshow": { "slideshow": {
@ -128,12 +128,8 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 19, "execution_count": 19,
"id": "88ce689f", "id": "33abf69d",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
@ -197,12 +193,8 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 22, "execution_count": 22,
"id": "ff1e011e", "id": "780f41d6",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
@ -224,12 +216,8 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "3e237076", "id": "b56a1b02",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"source": [ "source": [
"### Zakładka actions na stronie repozytorium:\n", "### Zakładka actions na stronie repozytorium:\n",
"https://github.com/TomekZet/ium-ga-hello/actions" "https://github.com/TomekZet/ium-ga-hello/actions"
@ -238,12 +226,8 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 27, "execution_count": 27,
"id": "32701383", "id": "e3be3eb3",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
@ -263,12 +247,8 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "1c01acb5", "id": "b40a7f62",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"source": [ "source": [
"### Ręczne wywoływanie\n", "### Ręczne wywoływanie\n",
"Workflow można również wywołać ręcznie, podając parametry.\n", "Workflow można również wywołać ręcznie, podając parametry.\n",
@ -278,10 +258,10 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 42, "execution_count": 42,
"id": "a7250bf7", "id": "132d8d16",
"metadata": { "metadata": {
"slideshow": { "slideshow": {
"slide_type": "fragment" "slide_type": "slide"
} }
}, },
"outputs": [ "outputs": [
@ -321,12 +301,8 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 43, "execution_count": 43,
"id": "36ddaac0", "id": "c4e71172",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
@ -388,12 +364,8 @@
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"id": "a764cc0d", "id": "a0628b12",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"source": [ "source": [
"### Akcje wykonywane w kontenerze Docker\n", "### Akcje wykonywane w kontenerze Docker\n",
"Akcja może być wywołana w kontenerze Docker (pobranym z Docker hub albo zbudowanym z Dockerfile)\n", "Akcja może być wywołana w kontenerze Docker (pobranym z Docker hub albo zbudowanym z Dockerfile)\n",
@ -403,12 +375,8 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 59, "execution_count": 59,
"id": "ff4dab8c", "id": "b8222161",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
@ -439,13 +407,9 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 80, "execution_count": 47,
"id": "f1aaff7c", "id": "80883b9e",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
@ -465,29 +429,21 @@
"# Copies your code file from your action repository to the filesystem path `/` of the container\n", "# Copies your code file from your action repository to the filesystem path `/` of the container\n",
"COPY entrypoint.sh /entrypoint.sh\n", "COPY entrypoint.sh /entrypoint.sh\n",
"\n", "\n",
"VOLUME /github/workspace/\n",
"\n",
"WORKDIR /github/workspace/\n",
"\n",
"# Code file to execute when the docker container starts up (`entrypoint.sh`)\n", "# Code file to execute when the docker container starts up (`entrypoint.sh`)\n",
"ENTRYPOINT [\"/entrypoint.sh\"]" "ENTRYPOINT [\"/entrypoint.sh\"]"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 84, "execution_count": 50,
"id": "7f778025", "id": "f98fee9a",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"Overwriting entrypoint.sh\n" "Writing entrypoint.sh\n"
] ]
} }
], ],
@ -495,9 +451,7 @@
"%%writefile entrypoint.sh\n", "%%writefile entrypoint.sh\n",
"#!/bin/sh -l\n", "#!/bin/sh -l\n",
"\n", "\n",
"figlet \"Hello $1\" | tee figlet.txt\n", "figlet \"Hello $1\"\n",
"echo \"Entrypoint invoked in: $PWD\"\n",
"readlink -f figlet.txt\n",
"time=$(date)\n", "time=$(date)\n",
"echo \"time=$time\" >> $GITHUB_OUTPUT" "echo \"time=$time\" >> $GITHUB_OUTPUT"
] ]
@ -505,12 +459,8 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 60, "execution_count": 60,
"id": "911975de", "id": "352de060",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [], "outputs": [],
"source": [ "source": [
"!chmod +x entrypoint.sh" "!chmod +x entrypoint.sh"
@ -518,8 +468,8 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 62, "execution_count": 57,
"id": "483e0498", "id": "d73fb6c4",
"metadata": { "metadata": {
"slideshow": { "slideshow": {
"slide_type": "slide" "slide_type": "slide"
@ -551,7 +501,6 @@
" - name: Checkout repo\n", " - name: Checkout repo\n",
" uses: actions/checkout@v2\n", " uses: actions/checkout@v2\n",
" - name: Use docker action\n", " - name: Use docker action\n",
" id: hello\n",
" uses: ./\n", " uses: ./\n",
" with:\n", " with:\n",
" who-to-greet: \"${{ github.event.inputs.input_text }}\"\n", " who-to-greet: \"${{ github.event.inputs.input_text }}\"\n",
@ -564,29 +513,26 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 63, "execution_count": 61,
"id": "bc24dff3", "id": "94d8eec0",
"metadata": { "metadata": {},
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [ "outputs": [
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"[main 22a5094] Fix path\n", "[main 97c7272] Fix path\n",
" 1 file changed, 1 insertion(+)\n", " 1 file changed, 0 insertions(+), 0 deletions(-)\n",
"Enumerating objects: 9, done.\n", " mode change 100644 => 100755 entrypoint.sh\n",
"Counting objects: 100% (9/9), done.\n", "Enumerating objects: 3, done.\n",
"Counting objects: 100% (3/3), done.\n",
"Delta compression using up to 4 threads\n", "Delta compression using up to 4 threads\n",
"Compressing objects: 100% (5/5), done.\n", "Compressing objects: 100% (2/2), done.\n",
"Writing objects: 100% (5/5), 570 bytes | 570.00 KiB/s, done.\n", "Writing objects: 100% (2/2), 232 bytes | 232.00 KiB/s, done.\n",
"Total 5 (delta 1), reused 0 (delta 0), pack-reused 0\n", "Total 2 (delta 1), reused 0 (delta 0), pack-reused 0\n",
"remote: Resolving deltas: 100% (1/1), completed with 1 local object.\u001b[K\n", "remote: Resolving deltas: 100% (1/1), completed with 1 local object.\u001b[K\n",
"To github.com:TomekZet/ium-ga-hello.git\n", "To github.com:TomekZet/ium-ga-hello.git\n",
" 97c7272..22a5094 main -> main\n" " 0f9de38..97c7272 main -> main\n"
] ]
} }
], ],
@ -596,128 +542,13 @@
"!git push" "!git push"
] ]
}, },
{
"cell_type": "markdown",
"id": "12af9d1b",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Archiwizowanie artefaktów\n",
"https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts\n",
"\n",
"Do archiwizowania artefaktów służy akcja \"upload-artifact\":\n",
"\n",
"```yaml\n",
" - name: Archive artifacts\n",
" uses: actions/upload-artifact@v3\n",
" with:\n",
" name: figlet-output\n",
" path: figlet.txt\n",
"```"
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 11, "execution_count": null,
"id": "245f7c8a", "id": "9c396330",
"metadata": { "metadata": {},
"slideshow": { "outputs": [],
"slide_type": "slide" "source": []
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Overwriting .github/workflows/docker-artifact.yml\n"
]
}
],
"source": [
"%%writefile .github/workflows/docker-artifact.yml\n",
"name: github-actions-hello-docker-artifact\n",
"on: \n",
" workflow_dispatch:\n",
" inputs:\n",
" input_text:\n",
" description: 'Who to greet' \n",
" required: true\n",
" default: 'World'\n",
"jobs:\n",
" hello-job:\n",
" name: \"Do all the hard stuff\"\n",
" runs-on: ubuntu-latest\n",
" steps:\n",
" - name: Checkout repo\n",
" uses: actions/checkout@v2\n",
" - name: Use docker action\n",
" id: hello\n",
" uses: ./\n",
" with:\n",
" who-to-greet: \"${{ github.event.inputs.input_text }}\"\n",
" # Use the output from the `hello` step\n",
" - name: Get the output time\n",
" run: echo \"The time was ${{ steps.hello.outputs.time }}\" > time.txt\n",
" - name: Archive artifacts\n",
" uses: actions/upload-artifact@v3\n",
" with:\n",
" name: figlet-output\n",
" path: |\n",
" figlet.txt\n",
" time.txt\n",
" publish:\n",
" name: \"Publish as github comment\"\n",
" runs-on: ubuntu-latest\n",
" needs: hello-job\n",
" steps:\n",
" - uses: actions/checkout@v3\n",
" #We need to download the artifact first, jobs do not share workflow files\n",
" - name: get-artifact \n",
" uses: actions/download-artifact@v3\n",
" with:\n",
" name: figlet-output\n",
" - name: display_artifact_contents\n",
" run:\n",
" cat time.txt ; tr ' ' '#' < figlet.txt\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "47e301f9",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[main 5a40228] Archive in one job, use in other\n",
" 1 file changed, 1 insertion(+)\n",
"Enumerating objects: 9, done.\n",
"Counting objects: 100% (9/9), done.\n",
"Delta compression using up to 4 threads\n",
"Compressing objects: 100% (5/5), done.\n",
"Writing objects: 100% (5/5), 622 bytes | 622.00 KiB/s, done.\n",
"Total 5 (delta 2), reused 0 (delta 0), pack-reused 0\n",
"remote: Resolving deltas: 100% (2/2), completed with 2 local objects.\u001b[K\n",
"To github.com:TomekZet/ium-ga-hello.git\n",
" 4df6dc0..5a40228 main -> main\n"
]
}
],
"source": [
"!git add -u\n",
"!git commit -m \"Archive in one job, use in other\"\n",
"!git push"
]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
@ -1047,23 +878,19 @@
} }
}, },
"source": [ "source": [
"## Zadania [20 pkt] (termin: 24.05.2023)\n", "## Zadania [15 pkt]\n",
"1. Utwórz konto na Github (jeśli jeszcze nie masz)\n", "1. Utwórz konto na Github (jeśli jeszcze nie masz)\n",
"2. Stwórz publiczne repozytorium. Link do niego wklej do kolumny \"Link Github\" w arkuszu [\"Zapisy na zbiory\"](https://teams.microsoft.com/l/file/F62B5988-A797-418D-B085-52E0AF8BD55E?tenantId=73689ee1-b42f-4e25-a5f6-66d1f29bc092&fileType=xlsx&objectUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11%2FShared%20Documents%2FGeneral%2FZapisy%20na%20zbiory.xlsx&baseUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11&serviceName=teams&threadId=19:d67b0dc2ee0849eba517a2aa8507df9c@thread.tacv2&groupId=8cd6b30e-edd9-48db-85ab-259fc11d0c5b) [1 pkt]\n", "2. Stwórz publiczne repozytorium. Link do niego wklej do kolumny \"Link Github\" w arkuszu [\"Zapisy na zbiory\"](https://teams.microsoft.com/l/file/F62B5988-A797-418D-B085-52E0AF8BD55E?tenantId=73689ee1-b42f-4e25-a5f6-66d1f29bc092&fileType=xlsx&objectUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11%2FShared%20Documents%2FGeneral%2FZapisy%20na%20zbiory.xlsx&baseUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11&serviceName=teams&threadId=19:d67b0dc2ee0849eba517a2aa8507df9c@thread.tacv2&groupId=8cd6b30e-edd9-48db-85ab-259fc11d0c5b) [1 pkt]\n",
"2. Stwórz prosty Github workflow który:\n", "2. Stwórz prosty Github workflow wykorzystujący akcje CML, który:\n",
" - zrobi checkout Twojego repozytorium [1 pkt]\n", " - zrobi checkout Twojego repozytorium [2 pkt]\n",
" - ściągnie pliki trenujące. Najlepiej byłoby to zrobić za pomocą DVC, ale tym razem uprośćmy zadanie ze względu na komplikacje, które mogą się pojawić przy konfiguracji uwierzytelniania. Pliki można po prostu dodać do repozytorium albo ściągnąć przez wget jeśli są publicznie dostępne [2 pkt]\n", " - ściągnie pliki trenujące. Najlepiej byłoby to zrobić za pomocą DVC, ale tym razem uprośćmy zadanie ze względu na koplikacje, które mogą się pojawić przy konfiguracji uwierzytelniania. Pliki można po prostu dodać do repozytorium albo ściągnąć przez wget jeśli są publicznie dostępne [2 pkt]\n",
" - będzie wywoływalny przez \"Workflow dispatch\" z parametrami trenowania [2 pkt]\n", " - dokona trenowania i ewaluacji [8 pkt]\n",
" - składał się będzie z co najmniej 3 jobów:\n", " - wyniki opublikuje za pomocą `cml-send-github-check` i/lub `cml-send-comment` [2 pkt]"
" 1. dokona trenowania jako osobnej akcji wykonanej w Dockerze [8 pkt]\n",
" 2. dokona ewaluacji modelu [6 pkt]\n",
" 3. zarchiwizuje plik z modelem"
] ]
} }
], ],
"metadata": { "metadata": {
"author": "Tomasz Ziętkiewicz", "author": "Tomasz Ziętkiewicz",
"celltoolbar": "Slideshow",
"email": "tomasz.zietkiewicz@amu.edu.pl", "email": "tomasz.zietkiewicz@amu.edu.pl",
"kernelspec": { "kernelspec": {
"display_name": "Python 3", "display_name": "Python 3",

2
IUM_11/github-actions-hello

@ -1 +1 @@
Subproject commit 5a40228be54897d47e0480318adcdbdec0e74d60 Subproject commit 97c7272bcaccf4a578615526b71ca723bb2b6d00