From 28189bfeef09bec4576c4f26a9525288db753563 Mon Sep 17 00:00:00 2001 From: Tomasz Date: Wed, 12 Apr 2023 15:14:37 +0200 Subject: [PATCH] =?UTF-8?q?Zaktualizowano=20meteria=C5=82y=20dotycz=C4=85c?= =?UTF-8?q?e=20Sacred?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- IUM_06.Jenkins-2.ipynb | 6 +- IUM_07.Sacred.ipynb | 236 +++++++++++++++++++++++++++++------------ 2 files changed, 169 insertions(+), 73 deletions(-) diff --git a/IUM_06.Jenkins-2.ipynb b/IUM_06.Jenkins-2.ipynb index 632a6cf..2b6dab8 100644 --- a/IUM_06.Jenkins-2.ipynb +++ b/IUM_06.Jenkins-2.ipynb @@ -8,7 +8,7 @@ } }, "source": [ - "![Logo 1](https://git.wmi.amu.edu.pl/AITech/Szablon/raw/branch/master/Logotyp_AITech1.jpg)\n", + "## ![Logo 1](https://git.wmi.amu.edu.pl/AITech/Szablon/raw/branch/master/Logotyp_AITech1.jpg)\n", "
\n", "

Inżynieria uczenia maszynowego

\n", "

6. Jenkins, część 2. [laboratoria]

\n", @@ -184,7 +184,7 @@ } }, "source": [ - "## Zadanie 1 [5 pkt] (termin: 2 V 2022)\n", + "## Zadanie 1 [5 pkt] (termin: 2023-05-12)\n", "1. Stwórz na Jenkins projekt typu Multibranch pipeline o nazwie s123456-training\n", " Projekt ten powinien przeprowadzać trenowanie modelu korzystając z kodu przygotowanego na poprzednich zajęciach. Trenowanie powinno odbywać się wewnątrz kontenera docker. [2 pkt]\n", "2. Projekt powinien odpalać się automatycznie po zakończonym budowaniu projektu s123456-create-dataset i kopiować z niego zbiór danych [1 pkt]\n", @@ -201,7 +201,7 @@ } }, "source": [ - "## Zadanie 2 [15 pkt] (termin: 2023-05-09)\n", + "## Zadanie 2 [15 pkt] (termin: 2023-05-12)\n", "1. Stwórz na Jenkins projekt typu Multibranch pipeline o nazwie s123456-evaluation.eg\n", " Projekt ten będzie przeprowadzał ewaluację modelu stworzonego w s123456-training na danych ze zbioru testującego [1 pkt]\n", "2. Ewaluacja polega na:\n", diff --git a/IUM_07.Sacred.ipynb b/IUM_07.Sacred.ipynb index 07a508b..0f41bc6 100644 --- a/IUM_07.Sacred.ipynb +++ b/IUM_07.Sacred.ipynb @@ -12,7 +12,7 @@ "
\n", "

Inżynieria uczenia maszynowego

\n", "

7. Sacred [laboratoria]

\n", - "

Tomasz Ziętkiewicz (2021)

\n", + "

Tomasz Ziętkiewicz (2023)

\n", "
\n", "\n", "![Logo 2](https://git.wmi.amu.edu.pl/AITech/Szablon/raw/branch/master/Logotyp_AITech2.jpg)" @@ -35,6 +35,21 @@ " —https://github.com/IDSIA/sacred / [Sens życia według Monty Pythona](https://en.wikipedia.org/wiki/Every_Sperm_Is_Sacred) " ] }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "- https://sacred.readthedocs.io/ - dokumentacja\n", + "- https://github.com/IDSIA/sacred - Github\n", + "- Open source\n", + "- Prosty w użyciu\n", + "- Wiele webowych frontendów" + ] + }, { "cell_type": "markdown", "metadata": { @@ -100,7 +115,7 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 2, "metadata": { "slideshow": { "slide_type": "slide" @@ -112,69 +127,39 @@ "output_type": "stream", "text": [ "Collecting sacred\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Downloading https://files.pythonhosted.org/packages/f4/8c/b99f668e8ca9747dcd374bb46cac808e58f3cb8e446df1b3e667f6be9778/sacred-0.8.2-py2.py3-none-any.whl (106kB)\n", - "\u001b[K 100% |████████████████████████████████| 112kB 1.2MB/s ta 0:00:01\n", - "\u001b[?25hCollecting py-cpuinfo>=4.0 (from sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Downloading https://files.pythonhosted.org/packages/e6/ba/77120e44cbe9719152415b97d5bfb29f4053ee987d6cb63f55ce7d50fadc/py-cpuinfo-8.0.0.tar.gz (99kB)\n", - "\u001b[K 100% |████████████████████████████████| 102kB 1.4MB/s a 0:00:01\n", - "\u001b[?25hCollecting wrapt<2.0,>=1.0 (from sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Downloading https://files.pythonhosted.org/packages/ba/8c/3d3dff02ae905157ba417b801f4a7aa4e6fedbc43882e9c765b7aae438ac/wrapt-1.14.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (74kB)\n", - "\u001b[K 100% |████████████████████████████████| 81kB 2.0MB/s ta 0:00:01\n", - "\u001b[?25hCollecting jsonpickle<2.0,>=1.2 (from sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Downloading https://files.pythonhosted.org/packages/e9/ec/35910cf6ab87f8a013036f01f732f871a23b6058123a7bd0c7b08fbbc937/jsonpickle-1.5.2-py2.py3-none-any.whl\n", - "Collecting colorama>=0.4 (from sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Downloading https://files.pythonhosted.org/packages/44/98/5b86278fbbf250d239ae0ecb724f8572af1c91f4a11edf4d36a206189440/colorama-0.4.4-py2.py3-none-any.whl\n", - "Collecting munch<3.0,>=2.0.2 (from sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Downloading https://files.pythonhosted.org/packages/cc/ab/85d8da5c9a45e072301beb37ad7f833cd344e04c817d97e0cc75681d248f/munch-2.5.0-py2.py3-none-any.whl\n", - "Collecting packaging>=18.0 (from sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Using cached https://files.pythonhosted.org/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572d127/packaging-21.3-py3-none-any.whl\n", - "Collecting GitPython (from sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Downloading https://files.pythonhosted.org/packages/55/60/f884f01eef2a7255875862ec1b12d57d74113ec6e8d9e16c4d254cd6aa3c/GitPython-3.1.20-py3-none-any.whl (178kB)\n", - "\u001b[K 100% |████████████████████████████████| 184kB 1.9MB/s ta 0:00:01\n", - "\u001b[?25hCollecting docopt<1.0,>=0.3 (from sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\n", - "Collecting importlib-metadata; python_version < \"3.8\" (from jsonpickle<2.0,>=1.2->sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Using cached https://files.pythonhosted.org/packages/a0/a1/b153a0a4caf7a7e3f15c2cd56c7702e2cf3d89b1b359d1f1c5e59d68f4ce/importlib_metadata-4.8.3-py3-none-any.whl\n", - "Collecting six (from munch<3.0,>=2.0.2->sacred)\n", - " Using cached https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl\n", - "Collecting pyparsing!=3.0.5,>=2.0.2 (from packaging>=18.0->sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Using cached https://files.pythonhosted.org/packages/d9/41/d9cfb4410589805cd787f8a82cddd13142d9bf7449d12adf2d05a4a7d633/pyparsing-3.0.8-py3-none-any.whl\n", - "Collecting gitdb<5,>=4.0.1 (from GitPython->sacred)\n", - " Downloading https://files.pythonhosted.org/packages/a3/7c/5d747655049bfbf75b5fcec57c8115896cb78d6fafa84f6d3ef4c0f13a98/gitdb-4.0.9-py3-none-any.whl (63kB)\n", - "\u001b[K 100% |████████████████████████████████| 71kB 1.9MB/s ta 0:00:01\n", - "\u001b[?25hCollecting typing-extensions>=3.7.4.3; python_version < \"3.10\" (from GitPython->sacred)\n", - " Using cached https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl\n", - "Collecting zipp>=0.5 (from importlib-metadata; python_version < \"3.8\"->jsonpickle<2.0,>=1.2->sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Using cached https://files.pythonhosted.org/packages/bd/df/d4a4974a3e3957fd1c1fa3082366d7fff6e428ddb55f074bf64876f8e8ad/zipp-3.6.0-py3-none-any.whl\n", - "Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython->sacred)\n", - "\u001b[33m Cache entry deserialization failed, entry ignored\u001b[0m\n", - " Downloading https://files.pythonhosted.org/packages/6d/01/7caa71608bc29952ae09b0be63a539e50d2484bc37747797a66a60679856/smmap-5.0.0-py3-none-any.whl\n", - "Building wheels for collected packages: py-cpuinfo, docopt\n", - " Running setup.py bdist_wheel for py-cpuinfo ... \u001b[?25ldone\n", - "\u001b[?25h Stored in directory: /home/tomek/.cache/pip/wheels/2e/15/f5/aa2a056d223903b52cf4870134e3a01df0c723816835dd08db\n", - " Running setup.py bdist_wheel for docopt ... \u001b[?25ldone\n", - "\u001b[?25h Stored in directory: /home/tomek/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\n", - "Successfully built py-cpuinfo docopt\n", - "Installing collected packages: py-cpuinfo, wrapt, typing-extensions, zipp, importlib-metadata, jsonpickle, colorama, six, munch, pyparsing, packaging, smmap, gitdb, GitPython, docopt, sacred\n", - "Successfully installed GitPython-3.1.20 colorama-0.4.4 docopt-0.6.2 gitdb-4.0.9 importlib-metadata-4.8.3 jsonpickle-1.5.2 munch-2.5.0 packaging-21.3 py-cpuinfo-8.0.0 pyparsing-3.0.8 sacred-0.8.2 six-1.16.0 smmap-5.0.0 typing-extensions-4.1.1 wrapt-1.14.0 zipp-3.6.0\n" + " Downloading sacred-0.8.4-py2.py3-none-any.whl (107 kB)\n", + "\u001b[K |████████████████████████████████| 107 kB 637 kB/s eta 0:00:01\n", + "\u001b[?25hCollecting py-cpuinfo>=4.0\n", + " Downloading py_cpuinfo-9.0.0-py3-none-any.whl (22 kB)\n", + "Collecting GitPython\n", + " Downloading GitPython-3.1.31-py3-none-any.whl (184 kB)\n", + "\u001b[K |████████████████████████████████| 184 kB 617 kB/s eta 0:00:01\n", + "\u001b[?25hCollecting munch<3.0,>=2.5\n", + " Downloading munch-2.5.0-py2.py3-none-any.whl (10 kB)\n", + "Collecting wrapt<2.0,>=1.0\n", + " Downloading wrapt-1.15.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78 kB)\n", + "\u001b[K |████████████████████████████████| 78 kB 508 kB/s eta 0:00:01\n", + "\u001b[?25hCollecting colorama>=0.4\n", + " Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\n", + "Collecting docopt<1.0,>=0.3\n", + " Downloading docopt-0.6.2.tar.gz (25 kB)\n", + "Collecting jsonpickle>=1.2\n", + " Downloading jsonpickle-3.0.1-py2.py3-none-any.whl (40 kB)\n", + "\u001b[K |████████████████████████████████| 40 kB 518 kB/s eta 0:00:01\n", + "\u001b[?25hRequirement already satisfied: packaging>=18.0 in /home/tomek/miniconda3/lib/python3.9/site-packages (from sacred) (23.0)\n", + "Requirement already satisfied: six in /home/tomek/miniconda3/lib/python3.9/site-packages (from munch<3.0,>=2.5->sacred) (1.16.0)\n", + "Collecting gitdb<5,>=4.0.1\n", + " Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)\n", + "\u001b[K |████████████████████████████████| 62 kB 430 kB/s eta 0:00:01\n", + "\u001b[?25hCollecting smmap<6,>=3.0.1\n", + " Using cached smmap-5.0.0-py3-none-any.whl (24 kB)\n", + "Building wheels for collected packages: docopt\n", + " Building wheel for docopt (setup.py) ... \u001b[?25ldone\n", + "\u001b[?25h Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13723 sha256=9ae6354916cc7817db186faf89036b67a6562e0391c04e901e72698d9ce5bd8b\n", + " Stored in directory: /home/tomek/.cache/pip/wheels/70/4a/46/1309fc853b8d395e60bafaf1b6df7845bdd82c95fd59dd8d2b\n", + "Successfully built docopt\n", + "Installing collected packages: smmap, gitdb, wrapt, py-cpuinfo, munch, jsonpickle, GitPython, docopt, colorama, sacred\n", + "Successfully installed GitPython-3.1.31 colorama-0.4.6 docopt-0.6.2 gitdb-4.0.10 jsonpickle-3.0.1 munch-2.5.0 py-cpuinfo-9.0.0 sacred-0.8.4 smmap-5.0.0 wrapt-1.15.0\n" ] } ], @@ -873,8 +858,17 @@ " - podstawowe informacje o środowisku (maszynie), na której przeprowadzono eksperyment\n", " - użyte pliki źródłowe\n", " - użyte zależności i ich wersje\n", - " - pliki otwarte za pomocą ex.open_resource\n", - " - pliki dodane za pomocą ex.add_artifact" + " - pliki otwarte za pomocą [ex.open_resource()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.Experiment.open_resource) albo [ex.add_resource()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.Experiment.add_resource)\n", + " - pliki dodane za pomocą [ex.add_artifact()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.run.Run.add_artifact)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!ls -l runs" ] }, { @@ -1319,6 +1313,107 @@ "```" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Otwieranie zasobów\n", + "- Zmiana danych wejściowych wpłwa w oczywisty sposób na wyniki\n", + "- Dlatego warto śledzić te zmiany za pomocą:\n", + " - [ex.open_resource()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.Experiment.open_resource)\n", + " - [ex.add_resource()](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.Experiment.add_resource)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "INFO - resources - Running command 'my_main'\n", + "INFO - resources - Started run with ID \"3\"\n", + "INFO - resources - Completed after 0:00:00\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Id,SepalLengthCm,SepalWidthCm,PetalLengthCm,PetalWidthCm,Species\n", + "\n" + ] + }, + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from sacred import Experiment\n", + "from sacred.observers import FileStorageObserver\n", + "\n", + "\n", + "ex = Experiment(\"resources\", interactive=True)\n", + "ex.observers.append(FileStorageObserver('my_runs'))\n", + "\n", + "@ex.main\n", + "def my_main():\n", + " f = ex.open_resource(\"Iris.csv\", \"r\")\n", + " print(f.readline())\n", + " \n", + "ex.run()" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "total 8\r\n", + "-rw-r--r-- 1 tomek tomek 5107 Apr 12 15:11 Iris_717820ef0af287ff346c5cabfb4c612c.csv\r\n" + ] + } + ], + "source": [ + "!ls -l my_runs/_resources" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "my_runs/3/run.json: \"name\": \"resources\",\r\n", + "my_runs/3/run.json: \"resources\": [\r\n", + "my_runs/3/run.json: \"my_runs/_resources/Iris_717820ef0af287ff346c5cabfb4c612c.csv\"\r\n" + ] + } + ], + "source": [ + "!grep -e \"resources\" -R my_runs/3" + ] + }, { "cell_type": "markdown", "metadata": { @@ -1601,11 +1696,12 @@ } }, "source": [ - "## Zadanie [15 pkt] (do 9 V 2021)\n", + "## Zadanie [15 pkt] (do 2023-05-12)\n", "1. \"Owiń\" wywołanie swojego eksperymentu za pomocą Sacred, w ten sposób, żeby zapisane zostały [10pkt]:\n", " - parametry, z którymi wywołany był trening\n", " - powstały plik z modelem (jako artefakt)\n", " - kod źródłowy użyty do przeprowadzenia treningu\n", + " - pliki wejściowe otwarte za pomocą open_resource\n", " - metryki\n", " \n", "Jako nazwę eksperymentu użyj swojego numeru indeksu tak, żebyś mogła/mógł je odnaleźć w Omniboard\n", @@ -1621,7 +1717,7 @@ "celltoolbar": "Slideshow", "email": "tomasz.zietkiewicz@amu.edu.pl", "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3", "language": "python", "name": "python3" }, @@ -1636,7 +1732,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.4" + "version": "3.9.12" }, "slideshow": { "slide_type": "slide"