pms/ium
4
5
Fork 1
forked from AITech/aitech-ium
Code Pull Requests Activity

Merge pull request 'Zaktualizowano meteriały dotyczące Sacred' (#41) from tzietkiewicz/aitech-ium:master into master

Browse Source 
Reviewed-on: AITech/aitech-ium#41
This commit is contained in:
Tomasz Zietkiewicz 2023-04-12 15:18:26 +02:00
commit 1d3e3ff71c
2 changed files with 169 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
IUM_06.Jenkins-2.ipynb
View File

@ -8,7 +8,7 @@
}
},
"source": [
"![Logo 1](https://git.wmi.amu.edu.pl/AITech/Szablon/raw/branch/master/Logotyp_AITech1.jpg)\n",
"## ![Logo 1](https://git.wmi.amu.edu.pl/AITech/Szablon/raw/branch/master/Logotyp_AITech1.jpg)\n",
"<div class=\"alert alert-block alert-info\">\n",
"<h1> Inżynieria uczenia maszynowego </h1>\n",
"<h2> 6. <i>Jenkins, część 2.</i> [laboratoria]</h2> \n",
@ -184,7 +184,7 @@
}
},
"source": [
"## Zadanie 1 [5 pkt] (termin: 2 V 2022)\n",
"## Zadanie 1 [5 pkt] (termin: 2023-05-12)\n",
"1. Stwórz na Jenkins projekt typu Multibranch pipeline o nazwie s123456-training\n",
" Projekt ten powinien przeprowadzać trenowanie modelu korzystając z kodu przygotowanego na poprzednich zajęciach. Trenowanie powinno odbywać się wewnątrz kontenera docker. [2 pkt]\n",
"2. Projekt powinien odpalać się automatycznie po zakończonym budowaniu projektu s123456-create-dataset i kopiować z niego zbiór danych [1 pkt]\n",
@ -201,7 +201,7 @@
}
},
"source": [
"## Zadanie 2 [15 pkt] (termin: 2023-05-09)\n",
"## Zadanie 2 [15 pkt] (termin: 2023-05-12)\n",
"1. Stwórz na Jenkins projekt typu Multibranch pipeline o nazwie s123456-evaluation.eg\n",
" Projekt ten będzie przeprowadzał ewaluację modelu stworzonego w s123456-training na danych ze zbioru testującego [1 pkt]\n",
"2. Ewaluacja polega na:\n",

236
IUM_07.Sacred.ipynb
View File

@ -12,7 +12,7 @@
"<div class=\"alert alert-block alert-info\">\n",
"<h1> Inżynieria uczenia maszynowego </h1>\n",
"<h2> 7. <i>Sacred</i> [laboratoria]</h2> \n",
"<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
"<h3> Tomasz Ziętkiewicz (2023)</h3>\n",
"</div>\n",
"\n",
"![Logo 2](https://git.wmi.amu.edu.pl/AITech/Szablon/raw/branch/master/Logotyp_AITech2.jpg)"
@ -35,6 +35,21 @@
" <cite>&mdash;https://github.com/IDSIA/sacred / [Sens życia według Monty Pythona](https://en.wikipedia.org/wiki/Every_Sperm_Is_Sacred) </cite>"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"- https://sacred.readthedocs.io/ - dokumentacja\n",
"- https://github.com/IDSIA/sacred - Github\n",
"- Open source\n",
"- Prosty w użyciu\n",
"- Wiele webowych frontendów"
]
},
{
"cell_type": "markdown",
"metadata": {
@ -100,7 +115,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 2,
"metadata": {
"slideshow": {
"slide_type": "slide"
@ -112,69 +127,39 @@
"output_type": "stream",
"text": [
"Collecting sacred\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Downloading https://files.pythonhosted.org/packages/f4/8c/b99f668e8ca9747dcd374bb46cac808e58f3cb8e446df1b3e667f6be9778/sacred-0.8.2-py2.py3-none-any.whl (106kB)\n",
"\u001b[K 100% |████████████████████████████████| 112kB 1.2MB/s ta 0:00:01\n",
"\u001b[?25hCollecting py-cpuinfo>=4.0 (from sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Downloading https://files.pythonhosted.org/packages/e6/ba/77120e44cbe9719152415b97d5bfb29f4053ee987d6cb63f55ce7d50fadc/py-cpuinfo-8.0.0.tar.gz (99kB)\n",
"\u001b[K 100% |████████████████████████████████| 102kB 1.4MB/s a 0:00:01\n",
"\u001b[?25hCollecting wrapt<2.0,>=1.0 (from sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Downloading https://files.pythonhosted.org/packages/ba/8c/3d3dff02ae905157ba417b801f4a7aa4e6fedbc43882e9c765b7aae438ac/wrapt-1.14.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (74kB)\n",
"\u001b[K 100% |████████████████████████████████| 81kB 2.0MB/s ta 0:00:01\n",
"\u001b[?25hCollecting jsonpickle<2.0,>=1.2 (from sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Downloading https://files.pythonhosted.org/packages/e9/ec/35910cf6ab87f8a013036f01f732f871a23b6058123a7bd0c7b08fbbc937/jsonpickle-1.5.2-py2.py3-none-any.whl\n",
"Collecting colorama>=0.4 (from sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Downloading https://files.pythonhosted.org/packages/44/98/5b86278fbbf250d239ae0ecb724f8572af1c91f4a11edf4d36a206189440/colorama-0.4.4-py2.py3-none-any.whl\n",
"Collecting munch<3.0,>=2.0.2 (from sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Downloading https://files.pythonhosted.org/packages/cc/ab/85d8da5c9a45e072301beb37ad7f833cd344e04c817d97e0cc75681d248f/munch-2.5.0-py2.py3-none-any.whl\n",
"Collecting packaging>=18.0 (from sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Using cached https://files.pythonhosted.org/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572d127/packaging-21.3-py3-none-any.whl\n",
"Collecting GitPython (from sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Downloading https://files.pythonhosted.org/packages/55/60/f884f01eef2a7255875862ec1b12d57d74113ec6e8d9e16c4d254cd6aa3c/GitPython-3.1.20-py3-none-any.whl (178kB)\n",
"\u001b[K 100% |████████████████████████████████| 184kB 1.9MB/s ta 0:00:01\n",
"\u001b[?25hCollecting docopt<1.0,>=0.3 (from sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\n",
"Collecting importlib-metadata; python_version < \"3.8\" (from jsonpickle<2.0,>=1.2->sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Using cached https://files.pythonhosted.org/packages/a0/a1/b153a0a4caf7a7e3f15c2cd56c7702e2cf3d89b1b359d1f1c5e59d68f4ce/importlib_metadata-4.8.3-py3-none-any.whl\n",
"Collecting six (from munch<3.0,>=2.0.2->sacred)\n",
" Using cached https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl\n",
"Collecting pyparsing!=3.0.5,>=2.0.2 (from packaging>=18.0->sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Using cached https://files.pythonhosted.org/packages/d9/41/d9cfb4410589805cd787f8a82cddd13142d9bf7449d12adf2d05a4a7d633/pyparsing-3.0.8-py3-none-any.whl\n",
"Collecting gitdb<5,>=4.0.1 (from GitPython->sacred)\n",
" Downloading https://files.pythonhosted.org/packages/a3/7c/5d747655049bfbf75b5fcec57c8115896cb78d6fafa84f6d3ef4c0f13a98/gitdb-4.0.9-py3-none-any.whl (63kB)\n",
"\u001b[K 100% |████████████████████████████████| 71kB 1.9MB/s ta 0:00:01\n",
"\u001b[?25hCollecting typing-extensions>=3.7.4.3; python_version < \"3.10\" (from GitPython->sacred)\n",
" Using cached https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl\n",
"Collecting zipp>=0.5 (from importlib-metadata; python_version < \"3.8\"->jsonpickle<2.0,>=1.2->sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Using cached https://files.pythonhosted.org/packages/bd/df/d4a4974a3e3957fd1c1fa3082366d7fff6e428ddb55f074bf64876f8e8ad/zipp-3.6.0-py3-none-any.whl\n",
"Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython->sacred)\n",
"\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n",
" Downloading https://files.pythonhosted.org/packages/6d/01/7caa71608bc29952ae09b0be63a539e50d2484bc37747797a66a60679856/smmap-5.0.0-py3-none-any.whl\n",
"Building wheels for collected packages: py-cpuinfo, docopt\n",
" Running setup.py bdist_wheel for py-cpuinfo ... \u001b[?25ldone\n",
"\u001b[?25h Stored in directory: /home/tomek/.cache/pip/wheels/2e/15/f5/aa2a056d223903b52cf4870134e3a01df0c723816835dd08db\n",
" Running setup.py bdist_wheel for docopt ... \u001b[?25ldone\n",
"\u001b[?25h Stored in directory: /home/tomek/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\n",
"Successfully built py-cpuinfo docopt\n",
"Installing collected packages: py-cpuinfo, wrapt, typing-extensions, zipp, importlib-metadata, jsonpickle, colorama, six, munch, pyparsing, packaging, smmap, gitdb, GitPython, docopt, sacred\n",
"Successfully installed GitPython-3.1.20 colorama-0.4.4 docopt-0.6.2 gitdb-4.0.9 importlib-metadata-4.8.3 jsonpickle-1.5.2 munch-2.5.0 packaging-21.3 py-cpuinfo-8.0.0 pyparsing-3.0.8 sacred-0.8.2 six-1.16.0 smmap-5.0.0 typing-extensions-4.1.1 wrapt-1.14.0 zipp-3.6.0\n"
" Downloading sacred-0.8.4-py2.py3-none-any.whl (107 kB)\n",
"\u001b[K |████████████████████████████████| 107 kB 637 kB/s eta 0:00:01\n",
"\u001b[?25hCollecting py-cpuinfo>=4.0\n",
" Downloading py_cpuinfo-9.0.0-py3-none-any.whl (22 kB)\n",
"Collecting GitPython\n",
" Downloading GitPython-3.1.31-py3-none-any.whl (184 kB)\n",
"\u001b[K |████████████████████████████████| 184 kB 617 kB/s eta 0:00:01\n",
"\u001b[?25hCollecting munch<3.0,>=2.5\n",
" Downloading munch-2.5.0-py2.py3-none-any.whl (10 kB)\n",
"Collecting wrapt<2.0,>=1.0\n",
" Downloading wrapt-1.15.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)\n",
"\u001b[K |████████████████████████████████| 78 kB 508 kB/s eta 0:00:01\n",
"\u001b[?25hCollecting colorama>=0.4\n",
" Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\n",
"Collecting docopt<1.0,>=0.3\n",
" Downloading docopt-0.6.2.tar.gz (25 kB)\n",
"Collecting jsonpickle>=1.2\n",
" Downloading jsonpickle-3.0.1-py2.py3-none-any.whl (40 kB)\n",
"\u001b[K |████████████████████████████████| 40 kB 518 kB/s eta 0:00:01\n",
"\u001b[?25hRequirement already satisfied: packaging>=18.0 in /home/tomek/miniconda3/lib/python3.9/site-packages (from sacred) (23.0)\n",
"Requirement already satisfied: six in /home/tomek/miniconda3/lib/python3.9/site-packages (from munch<3.0,>=2.5->sacred) (1.16.0)\n",
"Collecting gitdb<5,>=4.0.1\n",
" Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)\n",
"\u001b[K |████████████████████████████████| 62 kB 430 kB/s eta 0:00:01\n",
"\u001b[?25hCollecting smmap<6,>=3.0.1\n",
" Using cached smmap-5.0.0-py3-none-any.whl (24 kB)\n",
"Building wheels for collected packages: docopt\n",
" Building wheel for docopt (setup.py) ... \u001b[?25ldone\n",
"\u001b[?25h Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13723 sha256=9ae6354916cc7817db186faf89036b67a6562e0391c04e901e72698d9ce5bd8b\n",
" Stored in directory: /home/tomek/.cache/pip/wheels/70/4a/46/1309fc853b8d395e60bafaf1b6df7845bdd82c95fd59dd8d2b\n",
"Successfully built docopt\n",
"Installing collected packages: smmap, gitdb, wrapt, py-cpuinfo, munch, jsonpickle, GitPython, docopt, colorama, sacred\n",
"Successfully installed GitPython-3.1.31 colorama-0.4.6 docopt-0.6.2 gitdb-4.0.10 jsonpickle-3.0.1 munch-2.5.0 py-cpuinfo-9.0.0 sacred-0.8.4 smmap-5.0.0 wrapt-1.15.0\n"
]
}
],
@ -873,8 +858,17 @@
" - podstawowe informacje o środowisku (maszynie), na której przeprowadzono eksperyment\n",
" - użyte pliki źródłowe\n",
" - użyte zależności i ich wersje\n",
" - pliki otwarte za pomocą ex.open_resource\n",
" - pliki dodane za pomocą ex.add_artifact"
" - pliki otwarte za pomocą [ex.open_resource()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.Experiment.open_resource) albo [ex.add_resource()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.Experiment.add_resource)\n",
" - pliki dodane za pomocą [ex.add_artifact()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.run.Run.add_artifact)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!ls -l runs"
]
},
{
@ -1319,6 +1313,107 @@
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Otwieranie zasobów\n",
"- Zmiana danych wejściowych wpłwa w oczywisty sposób na wyniki\n",
"- Dlatego warto śledzić te zmiany za pomocą:\n",
" - [ex.open_resource()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.Experiment.open_resource)\n",
" - [ex.add_resource()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.Experiment.add_resource)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO - resources - Running command 'my_main'\n",
"INFO - resources - Started run with ID \"3\"\n",
"INFO - resources - Completed after 0:00:00\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Id,SepalLengthCm,SepalWidthCm,PetalLengthCm,PetalWidthCm,Species\n",
"\n"
]
},
{
"data": {
"text/plain": [
"<sacred.run.Run at 0x7f9b8821cd90>"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sacred import Experiment\n",
"from sacred.observers import FileStorageObserver\n",
"\n",
"\n",
"ex = Experiment(\"resources\", interactive=True)\n",
"ex.observers.append(FileStorageObserver('my_runs'))\n",
"\n",
"@ex.main\n",
"def my_main():\n",
" f = ex.open_resource(\"Iris.csv\", \"r\")\n",
" print(f.readline())\n",
" \n",
"ex.run()"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"total 8\r\n",
"-rw-r--r-- 1 tomek tomek 5107 Apr 12 15:11 Iris_717820ef0af287ff346c5cabfb4c612c.csv\r\n"
]
}
],
"source": [
"!ls -l my_runs/_resources"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"my_runs/3/run.json: \"name\": \"resources\",\r\n",
"my_runs/3/run.json: \"resources\": [\r\n",
"my_runs/3/run.json: \"my_runs/_resources/Iris_717820ef0af287ff346c5cabfb4c612c.csv\"\r\n"
]
}
],
"source": [
"!grep -e \"resources\" -R my_runs/3"
]
},
{
"cell_type": "markdown",
"metadata": {
@ -1601,11 +1696,12 @@
}
},
"source": [
"## Zadanie [15 pkt] (do 9 V 2021)\n",
"## Zadanie [15 pkt] (do 2023-05-12)\n",
"1. \"Owiń\" wywołanie swojego eksperymentu za pomocą Sacred, w ten sposób, żeby zapisane zostały [10pkt]:\n",
" - parametry, z którymi wywołany był trening\n",
" - powstały plik z modelem (jako artefakt)\n",
" - kod źródłowy użyty do przeprowadzenia treningu\n",
" - pliki wejściowe otwarte za pomocą open_resource\n",
" - metryki\n",
" \n",
"Jako nazwę eksperymentu użyj swojego numeru indeksu tak, żebyś mogła/mógł je odnaleźć w Omniboard\n",
@ -1621,7 +1717,7 @@
"celltoolbar": "Slideshow",
"email": "tomasz.zietkiewicz@amu.edu.pl",
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
@ -1636,7 +1732,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"version": "3.9.12"
},
"slideshow": {
"slide_type": "slide"