forked from tzietkiewicz/aitech-ium
master #44
@ -180,7 +180,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.12"
|
||||
"version": "3.9.16"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
|
||||
@ -509,7 +509,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.12"
|
||||
"version": "3.9.16"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
|
||||
@ -248,7 +248,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.12"
|
||||
"version": "3.9.16"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
|
||||
@ -981,13 +981,25 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"total 16\r\n",
|
||||
"-rw-r--r-- 1 tomek tomek 77 May 28 2022 config.json\r\n",
|
||||
"-rw-r--r-- 1 tomek tomek 159 May 28 2022 cout.txt\r\n",
|
||||
"-rw-r--r-- 1 tomek tomek 2 May 28 2022 metrics.json\r\n",
|
||||
"-rw-r--r-- 1 tomek tomek 1659 May 28 2022 run.json\r\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!ls -l my_runs/1"
|
||||
]
|
||||
@ -1555,7 +1567,7 @@
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Zadanie [15 pkt] (do 2023-05-12)\n",
|
||||
"## Zadanie [15 pkt] (do 2023-06-?)\n",
|
||||
"1. \"Owiń\" wywołanie swojego eksperymentu za pomocą Sacred, w ten sposób, żeby zapisane zostały [10pkt]:\n",
|
||||
" - parametry, z którymi wywołany był trening\n",
|
||||
" - powstały plik z modelem (jako artefakt)\n",
|
||||
|
||||
8252
IUM_08.MLFlow.ipynb
8252
IUM_08.MLFlow.ipynb
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
281
IUM_11.CML.ipynb
281
IUM_11.CML.ipynb
@ -103,7 +103,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"execution_count": 4,
|
||||
"id": "f4916c1f",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
@ -128,8 +128,12 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 19,
|
||||
"id": "33abf69d",
|
||||
"metadata": {},
|
||||
"id": "88ce689f",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -193,8 +197,12 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 22,
|
||||
"id": "780f41d6",
|
||||
"metadata": {},
|
||||
"id": "ff1e011e",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -216,8 +224,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b56a1b02",
|
||||
"metadata": {},
|
||||
"id": "3e237076",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Zakładka actions na stronie repozytorium:\n",
|
||||
"https://github.com/TomekZet/ium-ga-hello/actions"
|
||||
@ -226,8 +238,12 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 27,
|
||||
"id": "e3be3eb3",
|
||||
"metadata": {},
|
||||
"id": "32701383",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "fragment"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -247,8 +263,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b40a7f62",
|
||||
"metadata": {},
|
||||
"id": "1c01acb5",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Ręczne wywoływanie\n",
|
||||
"Workflow można również wywołać ręcznie, podając parametry.\n",
|
||||
@ -258,10 +278,10 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 42,
|
||||
"id": "132d8d16",
|
||||
"id": "a7250bf7",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
"slide_type": "fragment"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
@ -301,8 +321,12 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 43,
|
||||
"id": "c4e71172",
|
||||
"metadata": {},
|
||||
"id": "36ddaac0",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -364,8 +388,12 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a0628b12",
|
||||
"metadata": {},
|
||||
"id": "a764cc0d",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Akcje wykonywane w kontenerze Docker\n",
|
||||
"Akcja może być wywołana w kontenerze Docker (pobranym z Docker hub albo zbudowanym z Dockerfile)\n",
|
||||
@ -375,8 +403,12 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 59,
|
||||
"id": "b8222161",
|
||||
"metadata": {},
|
||||
"id": "ff4dab8c",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -407,9 +439,13 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 47,
|
||||
"id": "80883b9e",
|
||||
"metadata": {},
|
||||
"execution_count": 80,
|
||||
"id": "f1aaff7c",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -429,21 +465,29 @@
|
||||
"# Copies your code file from your action repository to the filesystem path `/` of the container\n",
|
||||
"COPY entrypoint.sh /entrypoint.sh\n",
|
||||
"\n",
|
||||
"VOLUME /github/workspace/\n",
|
||||
"\n",
|
||||
"WORKDIR /github/workspace/\n",
|
||||
"\n",
|
||||
"# Code file to execute when the docker container starts up (`entrypoint.sh`)\n",
|
||||
"ENTRYPOINT [\"/entrypoint.sh\"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 50,
|
||||
"id": "f98fee9a",
|
||||
"metadata": {},
|
||||
"execution_count": 84,
|
||||
"id": "7f778025",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Writing entrypoint.sh\n"
|
||||
"Overwriting entrypoint.sh\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@ -451,7 +495,9 @@
|
||||
"%%writefile entrypoint.sh\n",
|
||||
"#!/bin/sh -l\n",
|
||||
"\n",
|
||||
"figlet \"Hello $1\"\n",
|
||||
"figlet \"Hello $1\" | tee figlet.txt\n",
|
||||
"echo \"Entrypoint invoked in: $PWD\"\n",
|
||||
"readlink -f figlet.txt\n",
|
||||
"time=$(date)\n",
|
||||
"echo \"time=$time\" >> $GITHUB_OUTPUT"
|
||||
]
|
||||
@ -459,8 +505,12 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 60,
|
||||
"id": "352de060",
|
||||
"metadata": {},
|
||||
"id": "911975de",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "fragment"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!chmod +x entrypoint.sh"
|
||||
@ -468,8 +518,8 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 57,
|
||||
"id": "d73fb6c4",
|
||||
"execution_count": 62,
|
||||
"id": "483e0498",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
@ -501,6 +551,7 @@
|
||||
" - name: Checkout repo\n",
|
||||
" uses: actions/checkout@v2\n",
|
||||
" - name: Use docker action\n",
|
||||
" id: hello\n",
|
||||
" uses: ./\n",
|
||||
" with:\n",
|
||||
" who-to-greet: \"${{ github.event.inputs.input_text }}\"\n",
|
||||
@ -513,26 +564,29 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 61,
|
||||
"id": "94d8eec0",
|
||||
"metadata": {},
|
||||
"execution_count": 63,
|
||||
"id": "bc24dff3",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[main 97c7272] Fix path\n",
|
||||
" 1 file changed, 0 insertions(+), 0 deletions(-)\n",
|
||||
" mode change 100644 => 100755 entrypoint.sh\n",
|
||||
"Enumerating objects: 3, done.\n",
|
||||
"Counting objects: 100% (3/3), done.\n",
|
||||
"[main 22a5094] Fix path\n",
|
||||
" 1 file changed, 1 insertion(+)\n",
|
||||
"Enumerating objects: 9, done.\n",
|
||||
"Counting objects: 100% (9/9), done.\n",
|
||||
"Delta compression using up to 4 threads\n",
|
||||
"Compressing objects: 100% (2/2), done.\n",
|
||||
"Writing objects: 100% (2/2), 232 bytes | 232.00 KiB/s, done.\n",
|
||||
"Total 2 (delta 1), reused 0 (delta 0), pack-reused 0\n",
|
||||
"Compressing objects: 100% (5/5), done.\n",
|
||||
"Writing objects: 100% (5/5), 570 bytes | 570.00 KiB/s, done.\n",
|
||||
"Total 5 (delta 1), reused 0 (delta 0), pack-reused 0\n",
|
||||
"remote: Resolving deltas: 100% (1/1), completed with 1 local object.\u001b[K\n",
|
||||
"To github.com:TomekZet/ium-ga-hello.git\n",
|
||||
" 0f9de38..97c7272 main -> main\n"
|
||||
" 97c7272..22a5094 main -> main\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@ -542,13 +596,128 @@
|
||||
"!git push"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "12af9d1b",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Archiwizowanie artefaktów\n",
|
||||
"https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts\n",
|
||||
"\n",
|
||||
"Do archiwizowania artefaktów służy akcja \"upload-artifact\":\n",
|
||||
"\n",
|
||||
"```yaml\n",
|
||||
" - name: Archive artifacts\n",
|
||||
" uses: actions/upload-artifact@v3\n",
|
||||
" with:\n",
|
||||
" name: figlet-output\n",
|
||||
" path: figlet.txt\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "9c396330",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
"execution_count": 11,
|
||||
"id": "245f7c8a",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Overwriting .github/workflows/docker-artifact.yml\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"%%writefile .github/workflows/docker-artifact.yml\n",
|
||||
"name: github-actions-hello-docker-artifact\n",
|
||||
"on: \n",
|
||||
" workflow_dispatch:\n",
|
||||
" inputs:\n",
|
||||
" input_text:\n",
|
||||
" description: 'Who to greet' \n",
|
||||
" required: true\n",
|
||||
" default: 'World'\n",
|
||||
"jobs:\n",
|
||||
" hello-job:\n",
|
||||
" name: \"Do all the hard stuff\"\n",
|
||||
" runs-on: ubuntu-latest\n",
|
||||
" steps:\n",
|
||||
" - name: Checkout repo\n",
|
||||
" uses: actions/checkout@v2\n",
|
||||
" - name: Use docker action\n",
|
||||
" id: hello\n",
|
||||
" uses: ./\n",
|
||||
" with:\n",
|
||||
" who-to-greet: \"${{ github.event.inputs.input_text }}\"\n",
|
||||
" # Use the output from the `hello` step\n",
|
||||
" - name: Get the output time\n",
|
||||
" run: echo \"The time was ${{ steps.hello.outputs.time }}\" > time.txt\n",
|
||||
" - name: Archive artifacts\n",
|
||||
" uses: actions/upload-artifact@v3\n",
|
||||
" with:\n",
|
||||
" name: figlet-output\n",
|
||||
" path: |\n",
|
||||
" figlet.txt\n",
|
||||
" time.txt\n",
|
||||
" publish:\n",
|
||||
" name: \"Publish as github comment\"\n",
|
||||
" runs-on: ubuntu-latest\n",
|
||||
" needs: hello-job\n",
|
||||
" steps:\n",
|
||||
" - uses: actions/checkout@v3\n",
|
||||
" #We need to download the artifact first, jobs do not share workflow files\n",
|
||||
" - name: get-artifact \n",
|
||||
" uses: actions/download-artifact@v3\n",
|
||||
" with:\n",
|
||||
" name: figlet-output\n",
|
||||
" - name: display_artifact_contents\n",
|
||||
" run:\n",
|
||||
" cat time.txt ; tr ' ' '#' < figlet.txt\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"id": "47e301f9",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"[main 5a40228] Archive in one job, use in other\n",
|
||||
" 1 file changed, 1 insertion(+)\n",
|
||||
"Enumerating objects: 9, done.\n",
|
||||
"Counting objects: 100% (9/9), done.\n",
|
||||
"Delta compression using up to 4 threads\n",
|
||||
"Compressing objects: 100% (5/5), done.\n",
|
||||
"Writing objects: 100% (5/5), 622 bytes | 622.00 KiB/s, done.\n",
|
||||
"Total 5 (delta 2), reused 0 (delta 0), pack-reused 0\n",
|
||||
"remote: Resolving deltas: 100% (2/2), completed with 2 local objects.\u001b[K\n",
|
||||
"To github.com:TomekZet/ium-ga-hello.git\n",
|
||||
" 4df6dc0..5a40228 main -> main\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"!git add -u\n",
|
||||
"!git commit -m \"Archive in one job, use in other\"\n",
|
||||
"!git push"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
@ -878,19 +1047,23 @@
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Zadania [15 pkt]\n",
|
||||
"## Zadania [20 pkt] (termin: 24.05.2023)\n",
|
||||
"1. Utwórz konto na Github (jeśli jeszcze nie masz)\n",
|
||||
"2. Stwórz publiczne repozytorium. Link do niego wklej do kolumny \"Link Github\" w arkuszu [\"Zapisy na zbiory\"](https://teams.microsoft.com/l/file/F62B5988-A797-418D-B085-52E0AF8BD55E?tenantId=73689ee1-b42f-4e25-a5f6-66d1f29bc092&fileType=xlsx&objectUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11%2FShared%20Documents%2FGeneral%2FZapisy%20na%20zbiory.xlsx&baseUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11&serviceName=teams&threadId=19:d67b0dc2ee0849eba517a2aa8507df9c@thread.tacv2&groupId=8cd6b30e-edd9-48db-85ab-259fc11d0c5b) [1 pkt]\n",
|
||||
"2. Stwórz prosty Github workflow wykorzystujący akcje CML, który:\n",
|
||||
" - zrobi checkout Twojego repozytorium [2 pkt]\n",
|
||||
" - ściągnie pliki trenujące. Najlepiej byłoby to zrobić za pomocą DVC, ale tym razem uprośćmy zadanie ze względu na koplikacje, które mogą się pojawić przy konfiguracji uwierzytelniania. Pliki można po prostu dodać do repozytorium albo ściągnąć przez wget jeśli są publicznie dostępne [2 pkt]\n",
|
||||
" - dokona trenowania i ewaluacji [8 pkt]\n",
|
||||
" - wyniki opublikuje za pomocą `cml-send-github-check` i/lub `cml-send-comment` [2 pkt]"
|
||||
"2. Stwórz prosty Github workflow który:\n",
|
||||
" - zrobi checkout Twojego repozytorium [1 pkt]\n",
|
||||
" - ściągnie pliki trenujące. Najlepiej byłoby to zrobić za pomocą DVC, ale tym razem uprośćmy zadanie ze względu na komplikacje, które mogą się pojawić przy konfiguracji uwierzytelniania. Pliki można po prostu dodać do repozytorium albo ściągnąć przez wget jeśli są publicznie dostępne [2 pkt]\n",
|
||||
" - będzie wywoływalny przez \"Workflow dispatch\" z parametrami trenowania [2 pkt]\n",
|
||||
" - składał się będzie z co najmniej 3 jobów:\n",
|
||||
" 1. dokona trenowania jako osobnej akcji wykonanej w Dockerze [8 pkt]\n",
|
||||
" 2. dokona ewaluacji modelu [6 pkt]\n",
|
||||
" 3. zarchiwizuje plik z modelem"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
|
||||
@ -1 +1 @@
|
||||
Subproject commit 97c7272bcaccf4a578615526b71ca723bb2b6d00
|
||||
Subproject commit 5a40228be54897d47e0480318adcdbdec0e74d60
|
||||
Loading…
Reference in New Issue
Block a user