forked from tzietkiewicz/aitech-ium
Merge pull request 'master' (#20) from tzietkiewicz/aitech-ium:master into master
Reviewed-on: #20
This commit is contained in:
commit
b8bc0d05ee
87
IUM.Dla prowadzacego.ipynb
Normal file
87
IUM.Dla prowadzacego.ipynb
Normal file
@ -0,0 +1,87 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ac87c554",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Środowisko wykorzystywane na zajęciach"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "167416f3",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Certificat SSL dla strony: https://letsencrypt.org/\n",
|
||||
"Instrukcja instalacji: https://certbot.eff.org/lets-encrypt/ubuntufocal-other\n",
|
||||
"\n",
|
||||
"Instalacja certyfikatu w Jenkins: https://www.vhinandrich.com/jenkins-standalone-ssl-lets-encrypt\n",
|
||||
"\n",
|
||||
"Ścieżka do certyfikatu, którego ma używać Jenkins może być podana w dwóch miejscach (na Ubuntu 20.04):\n",
|
||||
"`/etc/default/jenkins` lub `/etc/init.d/jenkins`\n",
|
||||
"Ważne, aby w końcu w zmiennej `JENKINS_ARGS` znalazły się następujące parametry:\n",
|
||||
"```\n",
|
||||
"--httpPort=-1\n",
|
||||
"--httpsPort=8080\n",
|
||||
"--httpsKeyStore=/var/lib/jenkins/jenkins.jks\n",
|
||||
"--httpsKeyStorePassword=TUTAJ_HASŁO_PODANE_PRZY_KONWERSJI\"\n",
|
||||
"```\n",
|
||||
"albo:\n",
|
||||
"```\n",
|
||||
"--httpPort=-1\n",
|
||||
"--httpsPort=8080\n",
|
||||
"--httpsCertificate=/var/lib/jenkins/fullchain.pem\n",
|
||||
"--httpsPrivateKey=/var/lib/jenkins/privkey-rsa.pem\n",
|
||||
"```\n",
|
||||
"W zależności od tego, do jakiego formatu przekonwertowaliśmy nasz certyfikat\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Po 3 miesiącach certyfikat wygasa. Jeśli narzędzie certbot zostało poprawnie skonfigurowane,to powinno automatycznie odświeżać certyfikat. Niestety nie jest to certyfikat bezpośrednio używany przez Jenkins (ten wymaga konwersji do formatu `*.pem` albo `*.jks`). Dlatego przed wygaśnięciem certyfikatu musimy ręcznie dokonać konwersji (patrz https://www.vhinandrich.com/jenkins-standalone-ssl-lets-encrypt) albo dodać konwersję do post-hook w /etc/letsencrypt/renewal-hooks/post/ \n",
|
||||
"\n",
|
||||
"Przykładowy skrypt post (przy okazji restartuje działający na porcie 80 serwer nginx strzegący dostępu do MlFlow:\n",
|
||||
"```\n",
|
||||
"cd /home/tzietkiewicz/docker_mlflow_db && docker-compose start\n",
|
||||
"\n",
|
||||
"cd /etc/letsencrypt/live/tzietkiewicz.vm.wmi.amu.edu.pl/ && \\ \n",
|
||||
"openssl pkcs12 -inkey privkey.pem -in fullchain.pem -export -out keys.pkcs12 -password HASLO_CERTYFIKATU && \\\n",
|
||||
"keytool -importkeystore -srckeystore keys.pkcs12 -srcstoretype pkcs12 -destkeystore /var/lib/jenkins/jenkins.jks.new -srcstorepass HASLO_CERTYFIKATU -deststorepass HASLO_CERTYFIKATU && \\\n",
|
||||
"mv /var/lib/jenkins/jenkins.jks.new /var/lib/jenkins/jenkins.jks && \\\n",
|
||||
"service jenkins restart\n",
|
||||
"```\n",
|
||||
"`HASLO_CERTYFIKATU` może być dowolne - tutaj je generujemy, ale to samo musimy podać Jenkinsowi w `/etc/default/jenkins` lub `/etc/init.d/jenkins`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "60970c71",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.1"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@ -8,10 +8,14 @@
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"# AITech Inżynieria Uczenia Maszynowego\n",
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 0. <i>Informacje organizacyjne</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## Informacje organizacyjne"
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -116,7 +120,7 @@
|
||||
"\n",
|
||||
"W trakcie kolejnych zajęć będą Państwo poznawać różne techniki i narzędzia wspomagające proces rozwoju modeli uczenia maszynowgo.\n",
|
||||
"\n",
|
||||
"W wyniku realizacji zadań z poszczególnych zajęć powstaną części składowe potoku uczenia maszynowego (Machine Learning Pipeline), które na końcu zostaną zintegrowane w jedną całość.\n",
|
||||
"W wyniku realizacji zadań z poszczególnych zajęć powstaną części składowe potoku uczenia maszynowego (Machine Learning Pipeline) z wykorzystaniem różnych, czasami komplementarnych a czasami alternatywnych technologii.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" - Za wykonywanie na bieżąco zadań z poszczególnych zajęć będą przyznawane punkty.\n",
|
||||
@ -126,8 +130,6 @@
|
||||
" \n",
|
||||
" - W przypadku braku wykonania zadań w terminie nie zostaną przyznane za nie punkty.\n",
|
||||
" - W przypadku braku wystawienia przez prowadzącego punktów w terminie, uznaje się przyznanie za zadanie maksymalnej ilości punktów\n",
|
||||
" - Za wykonanie zadań cząstkowych w terminie można zdobyć 50% całkowitej liczby punktów\n",
|
||||
" - Za wykonanie finalnego projektu można zdobyć 50% całkowitej liczby punktów\n",
|
||||
" - Punkty przysługujące za każde zadanie cząstkowe będą podane przy opisie zadania\n",
|
||||
" \n",
|
||||
" Przelicznik punktów na oceny:\n",
|
||||
@ -155,12 +157,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -171,8 +176,13 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.5"
|
||||
"version": "3.9.1"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "0.Organizacyjne[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"toc": {
|
||||
"base_numbering": 1,
|
||||
"nav_menu": {},
|
||||
@ -185,7 +195,8 @@
|
||||
"toc_position": {},
|
||||
"toc_section_display": false,
|
||||
"toc_window_display": false
|
||||
}
|
||||
},
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
|
||||
@ -8,8 +8,14 @@
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### AITech Inżynieria Uczenia Maszynowego\n",
|
||||
"# 1. Wprowadzenie"
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 1. <i>Wprowadzenie</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -197,12 +203,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -213,8 +222,13 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.5"
|
||||
"version": "3.9.1"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "1.Wprowadzenie[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"toc": {
|
||||
"base_numbering": 1,
|
||||
"nav_menu": {},
|
||||
@ -227,7 +241,8 @@
|
||||
"toc_position": {},
|
||||
"toc_section_display": false,
|
||||
"toc_window_display": false
|
||||
}
|
||||
},
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
|
||||
@ -1,5 +1,23 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 2. <i>Dane</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
@ -1462,12 +1480,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -1478,8 +1499,13 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.5"
|
||||
"version": "3.9.1"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "2.Dane[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"toc": {
|
||||
"base_numbering": 1,
|
||||
"nav_menu": {},
|
||||
@ -1492,7 +1518,8 @@
|
||||
"toc_position": {},
|
||||
"toc_section_display": false,
|
||||
"toc_window_display": false
|
||||
}
|
||||
},
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
|
||||
@ -2,14 +2,29 @@
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"# System ciągłej integracji na przykładzie Jenkins"
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 3. <i>System ciągłej integracji na przykładzie Jenkins</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Ciągła integracja \n",
|
||||
" - Jest to praktyka rozwoju projektów informatycznych polegająca na bardzo częstym włączaniu nowych zmian do głównej gałęzi (branch)\n",
|
||||
@ -21,7 +36,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Systemy ciągłej integracji \n",
|
||||
"- Umożliwiają utomatyczne :\n",
|
||||
@ -39,7 +58,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Systemy ciągłej integracji\n",
|
||||
" - Jenkins <br/><img style=\"height: 50px;\" src=\"img/ci/jenkins.png\"/>\n",
|
||||
@ -51,7 +74,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Jenkins \n",
|
||||
"<br/><img style=\"height: 50px;\" src=\"img/ci/jenkins.png\"/>\n",
|
||||
@ -67,7 +94,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Terminologia\n",
|
||||
" - **Job, aka. Pipleine (Projekt)** - podstawowa jednostka organizacji pracy wykonywanej przez Jenkinsa. \n",
|
||||
@ -79,7 +110,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
" - **Build (Zadanie)** - instancja uruchomionego projektu. Może być w trakcie wykonywania, albo zakończona z jednym z rezultatów:\n",
|
||||
" - Successful <img style=\"height: 30px;\" src=\"IUM_03/blue.png\"/>\n",
|
||||
@ -96,7 +131,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Terminologia c.d.\n",
|
||||
" - **Step (Krok?)** - każdy pipeline to sekwencja kroków do wykonania. \n",
|
||||
@ -113,7 +152,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Dokumentacja\n",
|
||||
"- https://www.jenkins.io/doc/book/pipeline/\n",
|
||||
@ -123,7 +166,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Zadania [15 pkt]\n",
|
||||
"Na dzisiejszych zajęciach przygotujemy dwa połączone ze sobą zadania:\n",
|
||||
@ -138,14 +185,22 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Polecenia"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"#### 1. Zaloguj się\n",
|
||||
" - zaloguj się na https://tzietkiewicz.vm.wmi.amu.edu.pl:8080 za pomocą konta wydziałowego (jak w laboratoriach WMI)"
|
||||
@ -153,7 +208,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"#### 2. Utwórz nowy projekt (Job) [1pkt]\n",
|
||||
"\n",
|
||||
@ -167,7 +226,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"#### 3. Definicja Pipeline [2pkt]\n",
|
||||
"\n",
|
||||
@ -189,7 +252,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"Przykładowy declarative Pipeline (https://tzietkiewicz.vm.wmi.amu.edu.pl:8080/job/hello-world/):\n",
|
||||
"\n",
|
||||
@ -228,7 +295,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"Przykładowy scripted Pipeline (https://tzietkiewicz.vm.wmi.amu.edu.pl:8080/job/hello-world-scripted/):\n",
|
||||
"\n",
|
||||
@ -263,7 +334,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"#### 4. Repozytorium [2 pkt]\n",
|
||||
"\n",
|
||||
@ -285,7 +360,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"#### 5. Zapisanie zbioru danych [4 pkt]\n",
|
||||
"Edytuj zawartość Jenkinsfile tak, żeby realizował on następujące zadania:\n",
|
||||
@ -301,7 +380,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"#### 6. Hasło do Kaggle\n",
|
||||
" - Jeśli pobierasz swój zbiór z Kaggle, to możesz mieć problem z uwierzytelnieniem.\n",
|
||||
@ -352,7 +435,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"#### 7. Parametry zadania [1 pkt]\n",
|
||||
"1. Dodaj do projektu `s123456-create-dataset` parametr teksotwy `CUTOFF`, który umożliwi zdefiniowanie wielkości odcięcia zbioru danych (czyli obetnie liczbę przykładów do `CUTOFF` pierwszych/losowych przykładów).\n",
|
||||
@ -366,7 +453,11 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"#### 6. Statystki [5 pkt]\n",
|
||||
"Stwórz projekt (job) s123456-dataset-stats, który zrealizuje następujące kroki:\n",
|
||||
@ -398,19 +489,32 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Groovy",
|
||||
"language": "groovy",
|
||||
"name": "groovy"
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": "groovy",
|
||||
"file_extension": ".groovy",
|
||||
"mimetype": "",
|
||||
"name": "Groovy",
|
||||
"nbconverter_exporter": "",
|
||||
"version": "2.5.6"
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.1"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "3.CI-Jenkins[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"toc": {
|
||||
"base_numbering": 1,
|
||||
"nav_menu": {},
|
||||
@ -423,7 +527,8 @@
|
||||
"toc_position": {},
|
||||
"toc_section_display": false,
|
||||
"toc_window_display": false
|
||||
}
|
||||
},
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
|
||||
@ -8,7 +8,24 @@
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"# Konteneryzacja\n",
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 4. <i>Konteneryzacja</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"Plan na dziś\n",
|
||||
"\n",
|
||||
"1. Konteneryzacja i Docker\n",
|
||||
@ -562,12 +579,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -578,8 +598,13 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.5"
|
||||
"version": "3.9.1"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "4.Konteneryzacja[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"toc": {
|
||||
"base_numbering": 1,
|
||||
"nav_menu": {},
|
||||
@ -592,7 +617,8 @@
|
||||
"toc_position": {},
|
||||
"toc_section_display": false,
|
||||
"toc_window_display": false
|
||||
}
|
||||
},
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
|
||||
@ -8,7 +8,14 @@
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"# Biblioteki Deep Learning"
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 5. <i>Biblioteki Deep Learning</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -305,12 +312,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -323,6 +333,11 @@
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.1"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "5.Biblioteki_DL[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"toc": {
|
||||
"base_numbering": 1,
|
||||
"nav_menu": {},
|
||||
@ -335,7 +350,8 @@
|
||||
"toc_position": {},
|
||||
"toc_section_display": false,
|
||||
"toc_window_display": false
|
||||
}
|
||||
},
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
|
||||
@ -8,7 +8,14 @@
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"# Jenkins II"
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 6. <i>Jenkins, część 2.</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -216,12 +223,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -232,8 +242,13 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.5"
|
||||
"version": "3.9.1"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "6.Jenkins-2[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"toc": {
|
||||
"base_numbering": 1,
|
||||
"nav_menu": {},
|
||||
@ -246,7 +261,8 @@
|
||||
"toc_position": {},
|
||||
"toc_section_display": false,
|
||||
"toc_window_display": false
|
||||
}
|
||||
},
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
|
||||
@ -1,5 +1,23 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 7. <i>Sacred</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {
|
||||
@ -1493,12 +1511,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -1509,8 +1530,13 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.8.5"
|
||||
"version": "3.9.1"
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "7.Sacred[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"toc": {
|
||||
"base_numbering": 1,
|
||||
"nav_menu": {},
|
||||
@ -1523,7 +1549,8 @@
|
||||
"toc_position": {},
|
||||
"toc_section_display": false,
|
||||
"toc_window_display": false
|
||||
}
|
||||
},
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
|
||||
@ -8,8 +8,14 @@
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"# MLflow\n",
|
||||
"</br><img style=\"width: 50%;\" src=\"img/expcontrol/mlflow-logo-d.png\"/>\n"
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 8. <i>MLFlow</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -21,6 +27,8 @@
|
||||
},
|
||||
"source": [
|
||||
" ## MLflow\n",
|
||||
" </br><img style=\"width: 50%;\" src=\"img/expcontrol/mlflow-logo-d.png\"/>\n",
|
||||
"\n",
|
||||
" - https://mlflow.org/\n",
|
||||
" - Narzędzie podobne do omawianego na poprzednich zajęciach Sacred\n",
|
||||
" - Nieco inne podejście: mniej ingerencji w istniejący kod\n",
|
||||
@ -1413,12 +1421,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -1430,7 +1441,13 @@
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.1"
|
||||
}
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "8.MLFlow[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
|
||||
@ -2,14 +2,21 @@
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "be5ab2df",
|
||||
"id": "4b9b298d",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"# Środowiska wirtualne"
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 9. <i>Środowiska wirtualne</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -150,7 +157,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "90605b49",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## pipx\n",
|
||||
" - pipx: polecenie, które instaluje moduł Pythonowy w odrębnym środowisku wirtualnym venv\n",
|
||||
@ -1715,12 +1726,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -1732,7 +1746,13 @@
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.1"
|
||||
}
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "9.Python_srodowiska[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
|
||||
@ -1,5 +1,24 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7fe475ae",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 10. <i>DVC</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0c6f27a5",
|
||||
@ -9,7 +28,6 @@
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"# DVC\n",
|
||||
"<img src=\"img/expcontrol/dvc-logo.png\">"
|
||||
]
|
||||
},
|
||||
@ -866,7 +884,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "c7f24f75",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!dvc push"
|
||||
@ -1343,7 +1365,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2f5a8590",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## SSH remote\n",
|
||||
"Jednym z remote obsługiwanych przez DVC jest SFTP/SSH.\n",
|
||||
@ -1354,7 +1380,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "82a61107",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Lokalnie\n",
|
||||
"Będziemy potrzebować zależności :\n",
|
||||
@ -1370,7 +1400,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"id": "c48c5b8e",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -1446,7 +1480,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 27,
|
||||
"id": "e9a04876",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -1465,7 +1503,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 28,
|
||||
"id": "e3f27bbb",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -1485,7 +1527,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 32,
|
||||
"id": "5b2fa175",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -1528,7 +1574,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1468c44c",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Jenkins\n",
|
||||
"\n",
|
||||
@ -1546,20 +1596,18 @@
|
||||
"```Jenkinsfile\n",
|
||||
"withCredentials(\n",
|
||||
" [sshUserPrivateKey(credentialsId: '48ac7004-216e-4260-abba-1fe5db753e18', keyFileVariable: 'IUM_SFTP_KEY', passphraseVariable: '', usernameVariable: '')]) {\n",
|
||||
" sh \"dvc init -f\"\n",
|
||||
" sh \"dvc remote add -d ium_ssh_remote ssh://ium-sftp@tzietkiewicz.vm.wmi.amu.edu.pl/ium-sftp\"\n",
|
||||
" sh \"dvc remote modify --local ium_ssh_remote keyfile $IUM_SFTP_KEY\"\n",
|
||||
" sh \"dvc pull\"}\n",
|
||||
" sh 'dvc remote add -d ium_ssh_remote ssh://ium-sftp@tzietkiewicz.vm.wmi.amu.edu.pl/ium-sftp'\n",
|
||||
" sh 'dvc remote modify --local ium_ssh_remote keyfile $IUM_SFTP_KEY'\n",
|
||||
" sh 'dvc pull'}\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Secret text tak:\n",
|
||||
"\n",
|
||||
"```Jenkinsfile\n",
|
||||
" withCredentials([string(credentialsId: 'ium-sftp-password', variable: 'IUM_SFTP_PASS')]) {\n",
|
||||
" sh \"dvc init -f\"\n",
|
||||
" sh \"dvc remote add -d ium_ssh_remote ssh://ium-sftp@tzietkiewicz.vm.wmi.amu.edu.pl/ium-sftp\"\n",
|
||||
" sh \"dvc remote modify --local ium_ssh_remote password $IUM_SFTP_KEY\"\n",
|
||||
" sh \"dvc pull\"\n",
|
||||
" sh 'dvc remote add -d ium_ssh_remote ssh://ium-sftp@tzietkiewicz.vm.wmi.amu.edu.pl/ium-sftp'\n",
|
||||
" sh 'dvc remote modify --local ium_ssh_remote password $IUM_SFTP_KEY'\n",
|
||||
" sh 'dvc pull'\n",
|
||||
" }\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
@ -1570,12 +1618,15 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"celltoolbar": "Slideshow",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -1587,7 +1638,13 @@
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.1"
|
||||
}
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "10.DVC[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
|
||||
182
IUM_11.CML.ipynb
182
IUM_11.CML.ipynb
@ -2,16 +2,31 @@
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "08cbeca2",
|
||||
"metadata": {},
|
||||
"id": "9d06fc91",
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"# Github actions i CML"
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 11. <i>Github actions i CML</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "beeb17b2",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Github actions\n",
|
||||
"<img src=\"img/expcontrol/github-actions.jpeg\">"
|
||||
@ -20,7 +35,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "752995e1",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
" - https://docs.github.com/en/actions\n",
|
||||
" - System ciągłej integracji \"wbudowany\" w GitHub\n",
|
||||
@ -31,7 +50,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2b06cc01",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"https://youtu.be/cP0I9w2coGU"
|
||||
]
|
||||
@ -39,7 +62,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b66dd41f",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Terminologia Github Actions\n",
|
||||
" - *Workflow* - workflow odpowiada \"Pipeline\" z Jenkinsa.\n",
|
||||
@ -55,7 +82,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9f1f6d0a",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Definicja workflow\n",
|
||||
" - Workflow definiuje się w plikach YAM(o rozszerzeniu `*.yml` albo `*.yaml`) umieszczonych w specjalnym folderze `.github/workflows/` wewnątrz repozytorium\n",
|
||||
@ -74,7 +105,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "955585a9",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -96,7 +131,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "f4916c1f",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -115,7 +154,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"id": "dde8d432",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -145,7 +188,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ed780dea",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Zależności\n",
|
||||
"\n",
|
||||
@ -159,7 +206,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "28b582c4",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Akcje\n",
|
||||
"Za pomocą polecenia `uses` możemy używać przygotowanych wcześniej akcji. Mogą one pochodzić:\n",
|
||||
@ -171,7 +222,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "805622e8",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## CML - Continous Machine Learning\n",
|
||||
"<img src=\"img/expcontrol/cml.png\">"
|
||||
@ -180,7 +235,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e0b3acbf",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
" - Tworzone przez [iterative.ai](iterative.ai) (tak jak DVC)\n",
|
||||
" - https://cml.dev/\n",
|
||||
@ -197,7 +256,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cdb54b38",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"### Przykładowy Workflow CML:"
|
||||
]
|
||||
@ -206,7 +269,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "07b1035a",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -229,7 +296,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "bf27a2b3",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -248,7 +319,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"id": "64f6e21d",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -285,7 +360,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "83e49d3b",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# %load train.py\n",
|
||||
@ -323,7 +402,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8dc5748f",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"Wprowadźmy zmianę do pliku (linijka 17: `depth= = 6`)"
|
||||
]
|
||||
@ -332,7 +415,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"id": "afeaf939",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -377,7 +464,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3e4a711a",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"Stwórzmy nowy branch \"deep_depth\":"
|
||||
]
|
||||
@ -386,7 +477,11 @@
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"id": "ab019b0b",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
@ -422,7 +517,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b50f46a8",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"<img src=\"IUM_11/img/github-pr.png\">"
|
||||
]
|
||||
@ -430,7 +529,11 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c56c8785",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"<img src=\"IUM_11/img/github-checks.png\">"
|
||||
]
|
||||
@ -438,25 +541,32 @@
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fb25c587",
|
||||
"metadata": {},
|
||||
"metadata": {
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"## Zadania\n",
|
||||
"1. Utwórz konto na Github (jeśli jeszcze nie masz)\n",
|
||||
"2. Stwórz publiczne repozytorium. Link do niego wklej do kolumny \"Link Github\" w arkuszu [\"Zapisy na zbiory\"](https://teams.microsoft.com/l/file/F62B5988-A797-418D-B085-52E0AF8BD55E?tenantId=73689ee1-b42f-4e25-a5f6-66d1f29bc092&fileType=xlsx&objectUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11%2FShared%20Documents%2FGeneral%2FZapisy%20na%20zbiory.xlsx&baseUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11&serviceName=teams&threadId=19:d67b0dc2ee0849eba517a2aa8507df9c@thread.tacv2&groupId=8cd6b30e-edd9-48db-85ab-259fc11d0c5b)\n",
|
||||
"2. Stwórz publiczne repozytorium. Link do niego wklej do kolumny \"Link Github\" w arkuszu [\"Zapisy na zbiory\"](https://teams.microsoft.com/l/file/F62B5988-A797-418D-B085-52E0AF8BD55E?tenantId=73689ee1-b42f-4e25-a5f6-66d1f29bc092&fileType=xlsx&objectUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11%2FShared%20Documents%2FGeneral%2FZapisy%20na%20zbiory.xlsx&baseUrl=https%3A%2F%2Fuam.sharepoint.com%2Fsites%2F2021SL06-DIUMUI0LABInynieriauczeniamaszynowego-Grupa11&serviceName=teams&threadId=19:d67b0dc2ee0849eba517a2aa8507df9c@thread.tacv2&groupId=8cd6b30e-edd9-48db-85ab-259fc11d0c5b) [1 pkt]\n",
|
||||
"2. Stwórz prosty Github workflow wykorzystujący akcje CML, który:\n",
|
||||
" - zrobi checkout Twojego repozytorium\n",
|
||||
" - ściągnie pliki trenujące. Najlepiej byłoby to zrobić za pomocą DVC, ale tym razem uprośćmy zadanie ze względu na koplikacje, które mogą się pojawić przy konfiguracji uwierzytelniania. Pliki można po prostu dodać do repozytorium albo ściągnąć przez wget jeśli są publicznie dostępne\n",
|
||||
" - dokona trenowania i ewaluacji\n",
|
||||
" - wyniki opublikuje za pomocą `cml-send-github-check` i/lub `cml-send-comment`"
|
||||
" - zrobi checkout Twojego repozytorium [2 pkt]\n",
|
||||
" - ściągnie pliki trenujące. Najlepiej byłoby to zrobić za pomocą DVC, ale tym razem uprośćmy zadanie ze względu na koplikacje, które mogą się pojawić przy konfiguracji uwierzytelniania. Pliki można po prostu dodać do repozytorium albo ściągnąć przez wget jeśli są publicznie dostępne [2 pkt]\n",
|
||||
" - dokona trenowania i ewaluacji [8 pkt]\n",
|
||||
" - wyniki opublikuje za pomocą `cml-send-github-check` i/lub `cml-send-comment` [2 pkt]"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -468,7 +578,13 @@
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.1"
|
||||
}
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "11.CML[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
|
||||
@ -2,10 +2,17 @@
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "85b20432",
|
||||
"id": "805668ca",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Przegląd rynku pracy"
|
||||
"\n",
|
||||
"<div class=\"alert alert-block alert-info\">\n",
|
||||
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
||||
"<h2> 12. <i>Przegląd rynku pracy</i> [laboratoria]</h2> \n",
|
||||
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
|
||||
"</div>\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -56,11 +63,14 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"author": "Tomasz Ziętkiewicz",
|
||||
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"lang": "pl",
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
@ -72,7 +82,13 @@
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.1"
|
||||
}
|
||||
},
|
||||
"slideshow": {
|
||||
"slide_type": "slide"
|
||||
},
|
||||
"subtitle": "12.Praca[laboratoria]",
|
||||
"title": "Inżynieria uczenia maszynowego",
|
||||
"year": "2021"
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
|
||||
Loading…
Reference in New Issue
Block a user