{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![Logo 1](https://git.wmi.amu.edu.pl/AITech/Szablon/raw/branch/master/Logotyp_AITech1.jpg)\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<h1> Inżynieria uczenia maszynowego </h1>\n",
    "<h2> 7. <i>Sacred</i>  [laboratoria]</h2> \n",
    "<h3> Tomasz Ziętkiewicz (2021)</h3>\n",
    "</div>\n",
    "\n",
    "![Logo 2](https://git.wmi.amu.edu.pl/AITech/Szablon/raw/branch/master/Logotyp_AITech2.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Sacred\n",
    "> Every experiment is sacred <br>\n",
    "> Every experiment is great <br>\n",
    "> If an experiment is wasted <br>\n",
    "> God gets quite irate\n",
    ">\n",
    " <cite>&mdash;https://github.com/IDSIA/sacred / [Sens życia według Monty Pythona](https://en.wikipedia.org/wiki/Every_Sperm_Is_Sacred) </cite>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Przeprowadzanie eksperymentów (zmiana parametrów, trenowanie, ewaluacja) uczenia maszynowego jest kosztowne i czasochłonne\n",
    "- Dlatego warto przeprowadzać je w zorganizowany sposób\n",
    "- I tak, żebyśmy mogli powtórzyć / odtworzyć raz uzyskane wyniki"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "> Sacred is a tool to help you:\n",
    "> - configure\n",
    "> - organize\n",
    "> - log \n",
    "> - reproduce \n",
    "> experiments. \n",
    "> \n",
    ">It is designed to do all the tedious overhead work that you need to do around your actual experiment in order to:\n",
    "> - keep track of all the parameters of your experiment\n",
    "> - easily run your experiment for different settings\n",
    "> - save configurations for individual runs in a database\n",
    "> - reproduce your results\n",
    " \n",
    " <cite>&mdash;https://github.com/IDSIA/sacred</cite>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- **ConfigScopes** A very convenient way of the local variables in a function to define the parameters your experiment uses.\n",
    "- **Config Injection** You can access all parameters of your configuration from every function. They are automatically injected by name.\n",
    "- **Command-line interface** You get a powerful command-line interface for each experiment that you can use to change parameters and run different variants.\n",
    "- **Observers** Sacred provides Observers that log all kinds of information about your experiment, its dependencies, the configuration you used, the machine it is run on, and of course the result. These can be saved to a MongoDB, for easy access later.\n",
    "- **Automatic seeding** helps controlling the randomness in your experiments, such that the results remain reproducible."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Instalacja"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting sacred\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Downloading https://files.pythonhosted.org/packages/f4/8c/b99f668e8ca9747dcd374bb46cac808e58f3cb8e446df1b3e667f6be9778/sacred-0.8.2-py2.py3-none-any.whl (106kB)\n",
      "\u001b[K    100% |████████████████████████████████| 112kB 1.2MB/s ta 0:00:01\n",
      "\u001b[?25hCollecting py-cpuinfo>=4.0 (from sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Downloading https://files.pythonhosted.org/packages/e6/ba/77120e44cbe9719152415b97d5bfb29f4053ee987d6cb63f55ce7d50fadc/py-cpuinfo-8.0.0.tar.gz (99kB)\n",
      "\u001b[K    100% |████████████████████████████████| 102kB 1.4MB/s a 0:00:01\n",
      "\u001b[?25hCollecting wrapt<2.0,>=1.0 (from sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Downloading https://files.pythonhosted.org/packages/ba/8c/3d3dff02ae905157ba417b801f4a7aa4e6fedbc43882e9c765b7aae438ac/wrapt-1.14.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (74kB)\n",
      "\u001b[K    100% |████████████████████████████████| 81kB 2.0MB/s ta 0:00:01\n",
      "\u001b[?25hCollecting jsonpickle<2.0,>=1.2 (from sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Downloading https://files.pythonhosted.org/packages/e9/ec/35910cf6ab87f8a013036f01f732f871a23b6058123a7bd0c7b08fbbc937/jsonpickle-1.5.2-py2.py3-none-any.whl\n",
      "Collecting colorama>=0.4 (from sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Downloading https://files.pythonhosted.org/packages/44/98/5b86278fbbf250d239ae0ecb724f8572af1c91f4a11edf4d36a206189440/colorama-0.4.4-py2.py3-none-any.whl\n",
      "Collecting munch<3.0,>=2.0.2 (from sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Downloading https://files.pythonhosted.org/packages/cc/ab/85d8da5c9a45e072301beb37ad7f833cd344e04c817d97e0cc75681d248f/munch-2.5.0-py2.py3-none-any.whl\n",
      "Collecting packaging>=18.0 (from sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Using cached https://files.pythonhosted.org/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572d127/packaging-21.3-py3-none-any.whl\n",
      "Collecting GitPython (from sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Downloading https://files.pythonhosted.org/packages/55/60/f884f01eef2a7255875862ec1b12d57d74113ec6e8d9e16c4d254cd6aa3c/GitPython-3.1.20-py3-none-any.whl (178kB)\n",
      "\u001b[K    100% |████████████████████████████████| 184kB 1.9MB/s ta 0:00:01\n",
      "\u001b[?25hCollecting docopt<1.0,>=0.3 (from sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\n",
      "Collecting importlib-metadata; python_version < \"3.8\" (from jsonpickle<2.0,>=1.2->sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Using cached https://files.pythonhosted.org/packages/a0/a1/b153a0a4caf7a7e3f15c2cd56c7702e2cf3d89b1b359d1f1c5e59d68f4ce/importlib_metadata-4.8.3-py3-none-any.whl\n",
      "Collecting six (from munch<3.0,>=2.0.2->sacred)\n",
      "  Using cached https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl\n",
      "Collecting pyparsing!=3.0.5,>=2.0.2 (from packaging>=18.0->sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Using cached https://files.pythonhosted.org/packages/d9/41/d9cfb4410589805cd787f8a82cddd13142d9bf7449d12adf2d05a4a7d633/pyparsing-3.0.8-py3-none-any.whl\n",
      "Collecting gitdb<5,>=4.0.1 (from GitPython->sacred)\n",
      "  Downloading https://files.pythonhosted.org/packages/a3/7c/5d747655049bfbf75b5fcec57c8115896cb78d6fafa84f6d3ef4c0f13a98/gitdb-4.0.9-py3-none-any.whl (63kB)\n",
      "\u001b[K    100% |████████████████████████████████| 71kB 1.9MB/s ta 0:00:01\n",
      "\u001b[?25hCollecting typing-extensions>=3.7.4.3; python_version < \"3.10\" (from GitPython->sacred)\n",
      "  Using cached https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl\n",
      "Collecting zipp>=0.5 (from importlib-metadata; python_version < \"3.8\"->jsonpickle<2.0,>=1.2->sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Using cached https://files.pythonhosted.org/packages/bd/df/d4a4974a3e3957fd1c1fa3082366d7fff6e428ddb55f074bf64876f8e8ad/zipp-3.6.0-py3-none-any.whl\n",
      "Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython->sacred)\n",
      "\u001b[33m  Cache entry deserialization failed, entry ignored\u001b[0m\n",
      "  Downloading https://files.pythonhosted.org/packages/6d/01/7caa71608bc29952ae09b0be63a539e50d2484bc37747797a66a60679856/smmap-5.0.0-py3-none-any.whl\n",
      "Building wheels for collected packages: py-cpuinfo, docopt\n",
      "  Running setup.py bdist_wheel for py-cpuinfo ... \u001b[?25ldone\n",
      "\u001b[?25h  Stored in directory: /home/tomek/.cache/pip/wheels/2e/15/f5/aa2a056d223903b52cf4870134e3a01df0c723816835dd08db\n",
      "  Running setup.py bdist_wheel for docopt ... \u001b[?25ldone\n",
      "\u001b[?25h  Stored in directory: /home/tomek/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\n",
      "Successfully built py-cpuinfo docopt\n",
      "Installing collected packages: py-cpuinfo, wrapt, typing-extensions, zipp, importlib-metadata, jsonpickle, colorama, six, munch, pyparsing, packaging, smmap, gitdb, GitPython, docopt, sacred\n",
      "Successfully installed GitPython-3.1.20 colorama-0.4.4 docopt-0.6.2 gitdb-4.0.9 importlib-metadata-4.8.3 jsonpickle-1.5.2 munch-2.5.0 packaging-21.3 py-cpuinfo-8.0.0 pyparsing-3.0.8 sacred-0.8.2 six-1.16.0 smmap-5.0.0 typing-extensions-4.1.1 wrapt-1.14.0 zipp-3.6.0\n"
     ]
    }
   ],
   "source": [
    "!pip3 install sacred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Funkcja main"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting sacred_hello.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile sacred_hello.py\n",
    "from sacred import Experiment\n",
    "\n",
    "ex = Experiment()\n",
    "\n",
    "@ex.automain\n",
    "def my_main():\n",
    "    print('Witaj świecie!')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING - sacred_hello - No observers have been added to this run\r\n",
      "INFO - sacred_hello - Running command 'my_main'\r\n",
      "INFO - sacred_hello - Started\r\n",
      "Witaj świecie!\r\n",
      "INFO - sacred_hello - Completed after 0:00:00\r\n"
     ]
    }
   ],
   "source": [
    "!python3 IUM_07/sacred_hello.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### Co się dzieje w kodzie powyżej?\n",
    "1. Tworzymy obiekt klasy Experiment\n",
    "2. Dekorujemy funkcję \"my_main\" dekoratorem [automain](https://sacred.readthedocs.io/en/stable/apidoc.html#sacred.Experiment.automain)\n",
    "   Dzięki temu:\n",
    "    - otrzymujemy interfejs CLI, m.in. do kontrolowania poziomu logowania, przekazywania parametrów itp.\n",
    "    - oznaczamy funkcję \"my_main\" jako główną funkcję, która będzie wywoływana podczas wykonywania eksperymentu\n",
    "    - funkcja oznaczona jako główna musi być ostatnią funkcją zdefiniowaną w pliku!\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### Co nam daje interejs CLI:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Usage:\r\n",
      "  sacred_hello.py [(with UPDATE...)] [options]\r\n",
      "  sacred_hello.py help [COMMAND]\r\n",
      "  sacred_hello.py (-h | --help)\r\n",
      "  sacred_hello.py COMMAND [(with UPDATE...)] [options]\r\n",
      "\r\n",
      "\r\n",
      "\r\n",
      "Options:\r\n",
      "  -b VALUE --beat-interval=VALUE  Set the heart-beat interval for this run.  Time\r\n",
      "                                between two heartbeat events is measured in\r\n",
      "                                seconds.\r\n",
      "  -C VALUE --capture=VALUE      Control the way stdout and stderr are captured.\r\n",
      "                                The argument value must be one of [no, sys, fd]\r\n",
      "  -c VALUE --comment=VALUE      Add a comment to this run.\r\n",
      "  -d --debug                    Set this run to debug mode.  Suppress warnings\r\n",
      "                                about missing observers and don't filter the\r\n",
      "                                stacktrace. Also enables usage with ipython\r\n",
      "                                `--pdb`.\r\n",
      "  -e --enforce_clean            Fail if any version control repository is\r\n",
      "                                dirty.\r\n",
      "  -F VALUE --file_storage=VALUE  Add a file-storage observer to the experiment.\r\n",
      "                                The value of the arguement should be the base-\r\n",
      "                                directory to write the runs to\r\n",
      "  -f --force                    Disable warnings about suspicious changes for\r\n",
      "                                this run.\r\n",
      "  -h --help                     Print this help message and exit.\r\n",
      "  -l VALUE --loglevel=VALUE     Set the LogLevel.  Loglevel either as 0 - 50 or\r\n",
      "                                as string: DEBUG(10), INFO(20), WARNING(30),\r\n",
      "                                ERROR(40), CRITICAL(50)\r\n",
      "  -m VALUE --mongo_db=VALUE     Add a MongoDB Observer to the experiment.  The\r\n",
      "                                argument value is the database specification.\r\n",
      "                                Should be in the form:  `[host:port:]db_name[.c\r\n",
      "                                ollection[:id]][!priority]`\r\n",
      "  -n VALUE --name=VALUE         Set the name for this run.\r\n",
      "  -D --pdb                      Automatically enter post-mortem debugging with\r\n",
      "                                pdb on failure.\r\n",
      "  -p --print-config             Always print the configuration first.\r\n",
      "  -P VALUE --priority=VALUE     Sets the priority for a queued up experiment.\r\n",
      "                                `--priority=NUMBER` The number represent the\r\n",
      "                                priority for this run.\r\n",
      "  -q --queue                    Only queue this run, do not start it.\r\n",
      "  -S VALUE --s3=VALUE           Add a S3 File observer to the experiment.  The\r\n",
      "                                argument value should be\r\n",
      "                                `s3://<bucket>/path/to/exp`.\r\n",
      "  -s VALUE --sql=VALUE          Add a SQL Observer to the experiment.  The\r\n",
      "                                typical form is:\r\n",
      "                                dialect://username:password@host:port/database\r\n",
      "  -t VALUE --tiny_db=VALUE      Add a TinyDB Observer to the experiment.  The\r\n",
      "                                argument is the path to be given to the\r\n",
      "                                TinyDbObserver.\r\n",
      "  -u --unobserved               Ignore all observers for this run.\r\n",
      "\r\n",
      "\r\n",
      "Arguments:\r\n",
      "  COMMAND   Name of command to run (see below for list of commands)\r\n",
      "  UPDATE    Configuration assignments of the form foo.bar=17\r\n",
      "\r\n",
      "\r\n",
      "Commands:\r\n",
      "  print_config         Print the updated configuration and exit.\r\n",
      "  print_dependencies   Print the detected source-files and dependencies.\r\n",
      "  save_config          Store the updated configuration in a file.\r\n",
      "  print_named_configs  Print the available named configs and exit.\r\n",
      "  my_main              \r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "!python3 IUM_07/sacred_hello.py -h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Konfiguracje\n",
    " - Konfiguracje pozwalają nam sparametryzować wywołania eksperymentu.\n",
    " - Ułatwiają przekazywanie parametrów - zmienne z konfiguracji są wstrzykiwane do funkcji wywoływanych \n",
    " - Mogą być automatycznie zapisywane (dzięki czemu możemy śledzić jak zmieniały się parametry i jaki miały wpływ na wyniki)\n",
    " - Konfigurację można stworzyć w jeden z 3 sposobów:\n",
    "   - używając config scopes (z dekoratorem `@config`)\n",
    "   - jako słownik\n",
    "   - wczytując ją z pliku"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Konfiguracje - config scopes\n",
    "Jeśli oznaczymy jakąś funkcję dekoratorem `@config`, to zostanie ona uruchoniona przed wywołaniem eksperymentu i wszystkie jej lokalne zmienne, które da się zserializować jako json, zostaną dodane do konfiguracji. Potem ich wartości zostaną wstrzyknięte do innych funkcji wywoływanych w eksperymencie. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "from sacred import Experiment\n",
    "\n",
    "exint = Experiment(\"sacred_scopes\", interactive=True) #Jeśli wykonujemy interaktywnie (w konsoli Pythona albo w Jupyter):\n",
    "# - musimy podać nazwę eksperymentu (domyślnie jako nazwa używana jest nazwa pliku źródłowego)\n",
    "# - musimy dodać parametr \"interactive=True\"\n",
    "# - zamiast dekoratora \"@ex.automain\" używamy \"@ex.main\"\n",
    "\n",
    "@exint.config\n",
    "def my_config():\n",
    "    recipient = \"Świecie\"\n",
    "    greeting = \"Witaj\"\n",
    "    message = \"{0} {1}!\".format(greeting, recipient)\n",
    "\n",
    "\n",
    "@exint.main\n",
    "def my_main(message):\n",
    "    print(message)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING - sacred_scopes - No observers have been added to this run\n",
      "INFO - sacred_scopes - Running command 'my_main'\n",
      "INFO - sacred_scopes - Started\n",
      "INFO - sacred_scopes - Completed after 0:00:00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Witaj Świecie!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<sacred.run.Run at 0x7f641e9ddba8>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exint.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "##### Możemy podejrzeć wartości zmiennych w konfiguracji:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'recipient': 'Świecie', 'greeting': 'Witaj', 'message': 'Witaj Świecie!'}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_config()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Parametry możemy podejrzeć i modyfikować z poziomu CLI\n",
    " - wartości podane w CLI nadpiszą te podane w kodzie"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# %load IUM_07/sacred_scopes.py\n",
    "from sacred import Experiment\n",
    "\n",
    "ex = Experiment()\n",
    "\n",
    "@ex.config\n",
    "def my_config():\n",
    "    recipient = \"Świecie\"\n",
    "    greeting = \"Witaj\"\n",
    "    message = \"{0} {1}!\".format(greeting, recipient)\n",
    "\n",
    "@ex.automain\n",
    "def my_main(message):\n",
    "    print(message)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING - sacred_scopes - No observers have been added to this run\r\n",
      "INFO - sacred_scopes - Running command 'my_main'\r\n",
      "INFO - sacred_scopes - Started\r\n",
      "Witaj Przygodo!\r\n",
      "INFO - sacred_scopes - Completed after 0:00:00\r\n"
     ]
    }
   ],
   "source": [
    "!python3 IUM_07/sacred_scopes.py with 'recipient=Przygodo'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO - sacred_scopes - Running command 'print_config'\r\n",
      "INFO - sacred_scopes - Started\r\n",
      "Configuration (\u001b[34mmodified\u001b[0m, \u001b[32madded\u001b[0m, \u001b[31mtypechanged\u001b[0m, \u001b[2mdoc\u001b[0m):\r\n",
      "  greeting = 'Witaj'\r\n",
      "  message = 'Witaj Świecie!'\r\n",
      "  recipient = 'Świecie'\r\n",
      "  seed = 539001265                   \u001b[2m# the random seed for this experiment\u001b[0m\r\n",
      "INFO - sacred_scopes - Completed after 0:00:00\r\n"
     ]
    }
   ],
   "source": [
    "!python3 IUM_07/sacred_scopes.py print_config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO - sacred_scopes - Running command 'print_config'\r\n",
      "INFO - sacred_scopes - Started\r\n",
      "Configuration (\u001b[34mmodified\u001b[0m, \u001b[32madded\u001b[0m, \u001b[31mtypechanged\u001b[0m, \u001b[2mdoc\u001b[0m):\r\n",
      "  greeting = 'Witaj'\r\n",
      "  message = 'Witaj Przygodo!'\r\n",
      "\u001b[34m  recipient = 'Przygodo'\u001b[0m\r\n",
      "  seed = 215765170                   \u001b[2m# the random seed for this experiment\u001b[0m\r\n",
      "INFO - sacred_scopes - Completed after 0:00:00\r\n"
     ]
    }
   ],
   "source": [
    "!python IUM_07/sacred_scopes.py print_config with 'recipient=Przygodo'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Wczytywanie konfiguracji z pliku"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'recipient': 'samotności', 'greeting': 'Żegnaj'}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# %load IUM_07/config.json\n",
    "{\n",
    "    \"recipient\": \"samotności\",\n",
    "    \"greeting\": \"Żegnaj\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "from sacred import Experiment\n",
    "\n",
    "ex = Experiment(\"sacred_scopes\", interactive=True) #Jeśli wykonujemy interaktywnie (w konsoli Pythona albo w Jupyter):\n",
    "# - musimy podać nazwę eksperymentu (domyślnie jako nazwa używana jest nazwa pliku źródłowego)\n",
    "# - musimy dodać parametr \"interactive=True\"\n",
    "# - zamiast \"automain\" używamy parametru \"main\"\n",
    "\n",
    "@ex.config\n",
    "def my_config():\n",
    "    recipient = \"Świecie\"\n",
    "    greeting = \"Witaj\"\n",
    "\n",
    "ex.add_config(\"IUM_07/config.json\")\n",
    "\n",
    "\n",
    "@ex.main\n",
    "def my_main(recipient, greeting):\n",
    "    print(\"{0} {1}!\".format(greeting, recipient))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING - sacred_scopes - No observers have been added to this run\n",
      "INFO - sacred_scopes - Running command 'my_main'\n",
      "INFO - sacred_scopes - Started\n",
      "INFO - sacred_scopes - Completed after 0:00:00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Żegnaj samotności!\n"
     ]
    }
   ],
   "source": [
    "r = ex.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'recipient': 'samotności', 'greeting': 'Żegnaj', 'seed': 529757761}"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.config"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Możemy modyfikować części konfiguracji bezpośrednio przed wywołaniem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING - sacred_scopes - No observers have been added to this run\n",
      "INFO - sacred_scopes - Running command 'my_main'\n",
      "INFO - sacred_scopes - Started\n",
      "INFO - sacred_scopes - Completed after 0:00:00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Żegnaj nudo!\n"
     ]
    }
   ],
   "source": [
    "r = ex.run(config_updates={\"recipient\":\"nudo\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Wstrzykiwanie zależności\n",
    " - Oprócz funkcji głównej, wartości z konfiguracji są też wstrzykiwane do funkcji udekorowanych dekoratorem `@ex.capture`\n",
    " - Możemy korzystać w nich ze specjalnych parametrów, np.:\n",
    "   - `_log` - daje nam dostęp do obiektu logera (więcej: [logowanie](https://sacred.readthedocs.io/en/stable/logging.html))\n",
    "   - `_run` - daje dostęp do obiektu reprezentującego aktualne wywołanie eksperymentu (przykład później)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING - sacred_scopes - No observers have been added to this run\n",
      "INFO - sacred_scopes - Running command 'my_main'\n",
      "INFO - sacred_scopes - Started\n",
      "INFO - prepare_message - Enterred prepare_message\n",
      "INFO - sacred_scopes - Completed after 0:00:00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Witaj Świecie!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<sacred.run.Run at 0x7f423c40d820>"
      ]
     },
     "execution_count": 193,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sacred import Experiment\n",
    "\n",
    "ex = Experiment(\"sacred_scopes\", interactive=True)\n",
    "\n",
    "@ex.config\n",
    "def my_config():\n",
    "    recipient = \"Świecie\"\n",
    "    greeting = \"Witaj\"\n",
    "\n",
    "@ex.capture\n",
    "def prepare_message(recipient, greeting, _log):\n",
    "    _log.info(\"Enterred prepare_message\")\n",
    "    return \"{0} {1}!\".format(greeting, recipient)\n",
    "\n",
    "@ex.main\n",
    "def my_main():\n",
    "    print(prepare_message()) ## Nie musimy przekazywać wartości\n",
    "    \n",
    "ex.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Obserwowanie eksperymentów\n",
    "Sacred zapisuje szereg informacji na temat każdego eksperymentu:\n",
    " - czas wykonania\n",
    " - konfigurację\n",
    " - tekst zwrócony na stdout/stderr\n",
    " - błędy, jeśli wystąpiły\n",
    " - podstawowe informacje o środowisku (maszynie), na której przeprowadzono eksperyment\n",
    " - użyte pliki źródłowe\n",
    " - użyte zależności i ich wersje\n",
    " - pliki otwarte za pomocą ex.open_resource\n",
    " - pliki dodane za pomocą ex.add_artifact"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Obserwowane infromacje mogą zostać zapisane za pomocą jednego z [obserwatorów](https://sacred.readthedocs.io/en/stable/observers.html):\n",
    "  - Mongo Observer - zapisuje dane w MongoDB\n",
    "  - File Storage Observer - zapisuje dane lokalnie w pliku\n",
    "  - TinyDB Observer - korzysta z lokalnej bazy zapisanej w pliku JSON\n",
    "  - SQL Observer - przechowuje informacje w bazie SQL\n",
    "  - S3 Observer - korzysta z AWS S3\n",
    "  - gcs_observer - korzysta z Google Cloud Storage\n",
    "  - Queue Observer - rodzaj lokalnego bufora nakładanego na jeden z powyższych\n",
    "  - Slack Observer - używany do powiadomień wysyłanych na komunikator Slack\n",
    "  - Telegram Observer - używany do powiadomień wysyłanych na komunikator Telegram"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### File storage observer\n",
    "- zapisuje informacje o eksperymencie w lokalnych plikach \n",
    "- można go dodać tak: `ex.observers.append(FileStorageObserver('my_runs_directory'))`, gdzie `my_runs_directory` to ścieżka, gdzie będą zapisywane informacje o eksperymentach"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting IUM_07/file_observer.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile IUM_07/file_observer.py\n",
    "from sacred.observers import FileStorageObserver\n",
    "from sacred import Experiment\n",
    "\n",
    "ex = Experiment(\"file_observer\")\n",
    "\n",
    "ex.observers.append(FileStorageObserver('my_runs'))\n",
    "\n",
    "@ex.config\n",
    "def my_config():\n",
    "    recipient = \"Świecie\"\n",
    "    greeting = \"Witaj\"\n",
    "\n",
    "@ex.capture\n",
    "def prepare_message(recipient, greeting):\n",
    "    return \"{0} {1}!\".format(greeting, recipient)\n",
    "\n",
    "@ex.automain\n",
    "def my_main(recipient, greeting):\n",
    "    print(prepare_message()) ## Nie musimy przekazywać wartości"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO - file_observer - Running command 'my_main'\r\n",
      "INFO - file_observer - Started run with ID \"1\"\r\n",
      "Witaj Świecie!\r\n",
      "INFO - file_observer - Completed after 0:00:00\r\n"
     ]
    }
   ],
   "source": [
    "!python3 IUM_07/file_observer.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Zobaczmy jakie informacje zostały zapisane"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 0\r\n",
      "drwxr-xr-x 1 tomek tomek 512 Apr 25 09:51 1\r\n",
      "drwxr-xr-x 1 tomek tomek 512 Apr 25 09:51 _sources\r\n"
     ]
    }
   ],
   "source": [
    "!ls -l my_runs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 4\r\n",
      "-rw-r--r-- 1 tomek tomek   77 Apr 25 09:51 config.json\r\n",
      "-rw-r--r-- 1 tomek tomek  159 Apr 25 09:51 cout.txt\r\n",
      "-rw-r--r-- 1 tomek tomek    2 Apr 25 09:51 metrics.json\r\n",
      "-rw-r--r-- 1 tomek tomek 1659 Apr 25 09:51 run.json\r\n"
     ]
    }
   ],
   "source": [
    "!ls -l my_runs/1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "# %load my_runs/1/config.json\n",
    "{\n",
    "  \"greeting\": \"Witaj\",\n",
    "  \"recipient\": \"\\u015awiecie\",\n",
    "  \"seed\": 805857632\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO - file_observer - Running command 'my_main'\r\n",
      "INFO - file_observer - Started run with ID \"1\"\r\n",
      "Witaj Świecie!\r\n",
      "INFO - file_observer - Completed after 0:00:00\r\n"
     ]
    }
   ],
   "source": [
    "!cat my_runs/1/cout.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "# %load my_runs/1/run.json\n",
    "{\n",
    "  \"artifacts\": [],\n",
    "  \"command\": \"my_main\",\n",
    "  \"experiment\": {\n",
    "    \"base_dir\": \"/home/tomek/repos/aitech/aitech-ium/IUM_07\",\n",
    "    \"dependencies\": [\n",
    "      \"sacred==0.8.2\"\n",
    "    ],\n",
    "    \"mainfile\": \"file_observer.py\",\n",
    "    \"name\": \"file_observer\",\n",
    "    \"repositories\": [\n",
    "      {\n",
    "        \"commit\": \"3055a4f1c2ef06ea1c29e3d41d862827cede7e2a\",\n",
    "        \"dirty\": true,\n",
    "        \"url\": \"git@git.wmi.amu.edu.pl:tzietkiewicz/aitech-ium.git\"\n",
    "      }\n",
    "    ],\n",
    "    \"sources\": [\n",
    "      [\n",
    "        \"file_observer.py\",\n",
    "        \"_sources/file_observer_cd34a0ef4a32fb0a966eaa01ea6371ad.py\"\n",
    "      ]\n",
    "    ]\n",
    "  },\n",
    "  \"heartbeat\": \"2022-04-25T07:51:37.853633\",\n",
    "  \"host\": {\n",
    "    \"ENV\": {},\n",
    "    \"cpu\": \"Intel(R) Core(TM) i5-4200H CPU @ 2.80GHz\",\n",
    "    \"hostname\": \"ASUSEK\",\n",
    "    \"os\": [\n",
    "      \"Linux\",\n",
    "      \"Linux-4.4.0-19041-Microsoft-x86_64-with-Ubuntu-18.04-bionic\"\n",
    "    ],\n",
    "    \"python_version\": \"3.6.9\"\n",
    "  },\n",
    "  \"meta\": {\n",
    "    \"command\": \"my_main\",\n",
    "    \"options\": {\n",
    "      \"--beat-interval\": null,\n",
    "      \"--capture\": null,\n",
    "      \"--comment\": null,\n",
    "      \"--debug\": false,\n",
    "      \"--enforce_clean\": false,\n",
    "      \"--file_storage\": null,\n",
    "      \"--force\": false,\n",
    "      \"--help\": false,\n",
    "      \"--loglevel\": null,\n",
    "      \"--mongo_db\": null,\n",
    "      \"--name\": null,\n",
    "      \"--pdb\": false,\n",
    "      \"--print-config\": false,\n",
    "      \"--priority\": null,\n",
    "      \"--queue\": false,\n",
    "      \"--s3\": null,\n",
    "      \"--sql\": null,\n",
    "      \"--tiny_db\": null,\n",
    "      \"--unobserved\": false,\n",
    "      \"COMMAND\": null,\n",
    "      \"UPDATE\": [],\n",
    "      \"help\": false,\n",
    "      \"with\": false\n",
    "    }\n",
    "  },\n",
    "  \"resources\": [],\n",
    "  \"result\": null,\n",
    "  \"start_time\": \"2022-04-25T07:51:37.831461\",\n",
    "  \"status\": \"COMPLETED\",\n",
    "  \"stop_time\": \"2022-04-25T07:51:37.849334\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 4\r\n",
      "-rw-rw-r-- 1 tomek tomek 463 kwi 26 10:21 file_observer_bb0a5c4720d1072b641d23da080696b6.py\r\n"
     ]
    }
   ],
   "source": [
    "! ls -l my_runs/_sources\n",
    "## W run.json możemy znaleźć ścieżkę do pliku z źródłami:         \"_sources/file_observer_bb0a5c4720d1072b641d23da080696b6.py\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "## Źródła zostały zapisane\n",
    "# %load my_runs/_sources/file_observer_bb0a5c4720d1072b641d23da080696b6.py\n",
    "from sacred.observers import FileStorageObserver\n",
    "from sacred import Experiment\n",
    "\n",
    "ex = Experiment(\"file_observer\")\n",
    "\n",
    "ex.observers.append(FileStorageObserver('my_runs'))\n",
    "\n",
    "@ex.config\n",
    "def my_config():\n",
    "    recipient = \"Świecie\"\n",
    "    greeting = \"Witaj\"\n",
    "\n",
    "@ex.capture\n",
    "def prepare_message(recipient, greeting):\n",
    "    return \"{0} {1}!\".format(greeting, recipient)\n",
    "\n",
    "@ex.automain\n",
    "def my_main(recipient, greeting):\n",
    "    print(prepare_message()) ## Nie musimy przekazywać wartości"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Dodawanie własnych informacji\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO - file_observer - Running command 'my_main'\n",
      "INFO - file_observer - Started run with ID \"2\"\n",
      "INFO - file_observer - Completed after 0:00:00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Witaj Świecie!\n"
     ]
    }
   ],
   "source": [
    "from sacred.observers import FileStorageObserver\n",
    "from sacred import Experiment\n",
    "from datetime import datetime\n",
    "\n",
    "ex = Experiment(\"file_observer\", interactive=True)\n",
    "\n",
    "ex.observers.append(FileStorageObserver('my_runs'))\n",
    "\n",
    "@ex.config\n",
    "def my_config():\n",
    "    recipient = \"Świecie\"\n",
    "    greeting = \"Witaj\"\n",
    "\n",
    "### - Do \"przechwyconej\" przez @ex.capture funkcji prepare_message dodaliśmy specjalny parametr _run\n",
    "### - Daje on dostęp do obiektu wywołania eksperymentu w trakcie jego wywołania\n",
    "### - umożliwia m.in. zapisywanie dodatkowych informacji w słowniku info\n",
    "@ex.capture\n",
    "def prepare_message(recipient, greeting, _run):\n",
    "    _run.info[\"prepare_message_ts\"] = str(datetime.now())\n",
    "    return \"{0} {1}!\".format(greeting, recipient)\n",
    "\n",
    "@ex.main\n",
    "def my_main(recipient, greeting):\n",
    "    print(prepare_message()) ## Nie musimy przekazywać wartości\n",
    "    \n",
    "r = ex.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "  \"prepare_message_ts\": \"2021-04-26 10:39:59.268539\"\r\n",
      "}"
     ]
    }
   ],
   "source": [
    "cat my_runs/6/info.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Artefakty\n",
    "\n",
    "- Artefakty służą do zapisywania plików, np. z wytrenowanym modelem\n",
    "- Plik można zapisać jako artefakt korzystając z : [ex.add_artifact()](https://sacred.readthedocs.io/en/stable/apidoc.html?highlight=artifact#sacred.Experiment.add_artifact)\n",
    "```python\n",
    "ex.add_artifact(\"model.pb\")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    " ## Obserwator mongo\n",
    " - Żeby skorzystać z obserwatora Mongo, musimy mieć dostęp do bazy Mongo.\n",
    " - Można ją łatwo \"postawić\" za pomocą [docker-compose ](https://docs.docker.com/compose/).\n",
    " - W tym celu wystarczy skopiować katalog [examples/docker](https://github.com/IDSIA/sacred/tree/master/examples/docker) z repozytorium SACRED i uruchomić `docker-compose up` - dostaniemy uruchomioną bazę MongoDB i dodatkowo [Omniboard ](https://vivekratnavel.github.io/omniboard/#/). Więcej informacji w [dokumentacji](https://sacred.readthedocs.io/en/stable/examples.html#docker-setup)\n",
    " - Baza taka została już postawiona na serwerze Jenkins, więc pracując na Jenkinsie można skorzystać z lokalnej bazy (`localhost:27017`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pymongo\n",
      "  Downloading https://files.pythonhosted.org/packages/10/3b/46541b4ee3000019b8ef5b1847292ddc77f492c162bc4d49c424db7fc97a/pymongo-4.1.1-cp36-cp36m-manylinux1_x86_64.whl (464kB)\n",
      "\u001b[K    100% |████████████████████████████████| 471kB 959kB/s ta 0:00:01\n",
      "\u001b[?25hInstalling collected packages: pymongo\n",
      "Successfully installed pymongo-4.1.1\n"
     ]
    }
   ],
   "source": [
    "!pip3 install pymongo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO - sacred_scopes - Running command 'my_main'\n",
      "ERROR - sacred_scopes - Failed after 0:00:30!\n",
      "ERROR - sacred_scopes - Traceback (most recent call last):\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/sacred/run.py\", line 235, in __call__\n",
      "    self._emit_started()\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/sacred/run.py\", line 333, in _emit_started\n",
      "    _id=self._id,\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/sacred/observers/mongo.py\", line 258, in started_event\n",
      "    self.insert()\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/sacred/observers/mongo.py\", line 367, in insert\n",
      "    c.next()[\"_id\"] + 1 if self.runs.count_documents({}, limit=1) else 1\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/pymongo/collection.py\", line 1811, in count_documents\n",
      "    return self._retryable_non_cursor_read(_cmd, session)\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/pymongo/collection.py\", line 1816, in _retryable_non_cursor_read\n",
      "    with client._tmp_session(session) as s:\n",
      "  File \"/usr/lib/python3.6/contextlib.py\", line 81, in __enter__\n",
      "    return next(self.gen)\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/pymongo/mongo_client.py\", line 1676, in _tmp_session\n",
      "    s = self._ensure_session(session)\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/pymongo/mongo_client.py\", line 1663, in _ensure_session\n",
      "    return self.__start_session(True, causal_consistency=False)\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/pymongo/mongo_client.py\", line 1608, in __start_session\n",
      "    self._topology._check_implicit_session_support()\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/pymongo/topology.py\", line 519, in _check_implicit_session_support\n",
      "    self._check_session_support()\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/pymongo/topology.py\", line 536, in _check_session_support\n",
      "    readable_server_selector, self._settings.server_selection_timeout, None\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/pymongo/topology.py\", line 229, in _select_servers_loop\n",
      "    % (self._error_message(selector), timeout, self.description)\n",
      "pymongo.errors.ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 6266603d3269f60a67f61383, topology_type: Unknown, servers: [<ServerDescription ('localhost', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:27017: [Errno 111] Connection refused',)>]>\n",
      "\n",
      "During handling of the above exception, another exception occurred:\n",
      "\n",
      "Traceback (most recent call last):\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/sacred/run.py\", line 429, in _final_call\n",
      "    getattr(observer, method)(**kwargs)\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/sacred/observers/mongo.py\", line 283, in failed_event\n",
      "    self.final_save(attempts=1)\n",
      "  File \"/home/tomek/.local/lib/python3.6/site-packages/sacred/observers/mongo.py\", line 421, in final_save\n",
      "    os.makedirs(self.failure_dir, exist_ok=True)\n",
      "  File \"/usr/lib/python3.6/os.py\", line 205, in makedirs\n",
      "    head, tail = path.split(name)\n",
      "  File \"/usr/lib/python3.6/posixpath.py\", line 107, in split\n",
      "    p = os.fspath(p)\n",
      "TypeError: expected str, bytes or os.PathLike object, not NoneType\n",
      "\n"
     ]
    },
    {
     "ename": "ServerSelectionTimeoutError",
     "evalue": "localhost:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 6266603d3269f60a67f61383, topology_type: Unknown, servers: [<ServerDescription ('localhost', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:27017: [Errno 111] Connection refused',)>]>",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mServerSelectionTimeoutError\u001b[0m               Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-23-95f1b404dde7>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     19\u001b[0m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprepare_message\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m## Nie musimy przekazywać wartości\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     20\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 21\u001b[0;31m \u001b[0mex\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/sacred/experiment.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(self, command_name, config_updates, named_configs, info, meta_info, options)\u001b[0m\n\u001b[1;32m    274\u001b[0m             \u001b[0mcommand_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconfig_updates\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnamed_configs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minfo\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmeta_info\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moptions\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    275\u001b[0m         )\n\u001b[0;32m--> 276\u001b[0;31m         \u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    277\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mrun\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    278\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/sacred/run.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args)\u001b[0m\n\u001b[1;32m    233\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    234\u001b[0m             \u001b[0;32mwith\u001b[0m \u001b[0mcapture_stdout\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_output_file\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 235\u001b[0;31m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_emit_started\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    236\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_start_heartbeat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    237\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_execute_pre_run_hooks\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/sacred/run.py\u001b[0m in \u001b[0;36m_emit_started\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    331\u001b[0m                 \u001b[0mconfig\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    332\u001b[0m                 \u001b[0mmeta_info\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmeta_info\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 333\u001b[0;31m                 \u001b[0m_id\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_id\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    334\u001b[0m             )\n\u001b[1;32m    335\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_id\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/sacred/observers/mongo.py\u001b[0m in \u001b[0;36mstarted_event\u001b[0;34m(self, ex_info, command, host_info, start_time, config, meta_info, _id)\u001b[0m\n\u001b[1;32m    256\u001b[0m         \u001b[0;31m# save sources\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    257\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_entry\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"experiment\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"sources\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msave_sources\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mex_info\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 258\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minsert\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    259\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_entry\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"_id\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    260\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/sacred/observers/mongo.py\u001b[0m in \u001b[0;36minsert\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    365\u001b[0m                 \u001b[0mc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msort\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"_id\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpymongo\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDESCENDING\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlimit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    366\u001b[0m                 self.run_entry[\"_id\"] = (\n\u001b[0;32m--> 367\u001b[0;31m                     \u001b[0mc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"_id\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mruns\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcount_documents\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlimit\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    368\u001b[0m                 )\n\u001b[1;32m    369\u001b[0m             \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/pymongo/collection.py\u001b[0m in \u001b[0;36mcount_documents\u001b[0;34m(self, filter, session, comment, **kwargs)\u001b[0m\n\u001b[1;32m   1809\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"n\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1810\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1811\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_retryable_non_cursor_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0m_cmd\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msession\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1812\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1813\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_retryable_non_cursor_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msession\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/pymongo/collection.py\u001b[0m in \u001b[0;36m_retryable_non_cursor_read\u001b[0;34m(self, func, session)\u001b[0m\n\u001b[1;32m   1814\u001b[0m         \u001b[0;34m\"\"\"Non-cursor read helper to handle implicit session creation.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1815\u001b[0m         \u001b[0mclient\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__database\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclient\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1816\u001b[0;31m         \u001b[0;32mwith\u001b[0m \u001b[0mclient\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_tmp_session\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msession\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1817\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mclient\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_retryable_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_read_preference_for\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1818\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/lib/python3.6/contextlib.py\u001b[0m in \u001b[0;36m__enter__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m     79\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__enter__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     80\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 81\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mnext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgen\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     82\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mStopIteration\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     83\u001b[0m             \u001b[0;32mraise\u001b[0m \u001b[0mRuntimeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"generator didn't yield\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/pymongo/mongo_client.py\u001b[0m in \u001b[0;36m_tmp_session\u001b[0;34m(self, session, close)\u001b[0m\n\u001b[1;32m   1674\u001b[0m             \u001b[0;32mreturn\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1675\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1676\u001b[0;31m         \u001b[0ms\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_ensure_session\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msession\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1677\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1678\u001b[0m             \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/pymongo/mongo_client.py\u001b[0m in \u001b[0;36m_ensure_session\u001b[0;34m(self, session)\u001b[0m\n\u001b[1;32m   1661\u001b[0m             \u001b[0;31m# Don't make implicit sessions causally consistent. Applications\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1662\u001b[0m             \u001b[0;31m# should always opt-in.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1663\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__start_session\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcausal_consistency\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1664\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mConfigurationError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mInvalidOperation\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1665\u001b[0m             \u001b[0;31m# Sessions not supported.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/pymongo/mongo_client.py\u001b[0m in \u001b[0;36m__start_session\u001b[0;34m(self, implicit, **kwargs)\u001b[0m\n\u001b[1;32m   1606\u001b[0m         \u001b[0;31m# Raises ConfigurationError if sessions are not supported.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1607\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mimplicit\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1608\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_topology\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_check_implicit_session_support\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1609\u001b[0m             \u001b[0mserver_session\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_EmptyServerSession\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1610\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/pymongo/topology.py\u001b[0m in \u001b[0;36m_check_implicit_session_support\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    517\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_check_implicit_session_support\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    518\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_lock\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 519\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_check_session_support\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    520\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    521\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_check_session_support\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/pymongo/topology.py\u001b[0m in \u001b[0;36m_check_session_support\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    534\u001b[0m             \u001b[0;32melif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_description\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreadable_servers\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    535\u001b[0m                 self._select_servers_loop(\n\u001b[0;32m--> 536\u001b[0;31m                     \u001b[0mreadable_server_selector\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_settings\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mserver_selection_timeout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    537\u001b[0m                 )\n\u001b[1;32m    538\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/.local/lib/python3.6/site-packages/pymongo/topology.py\u001b[0m in \u001b[0;36m_select_servers_loop\u001b[0;34m(self, selector, timeout, address)\u001b[0m\n\u001b[1;32m    227\u001b[0m                 raise ServerSelectionTimeoutError(\n\u001b[1;32m    228\u001b[0m                     \u001b[0;34m\"%s, Timeout: %ss, Topology Description: %r\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 229\u001b[0;31m                     \u001b[0;34m%\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_error_message\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mselector\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdescription\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    230\u001b[0m                 )\n\u001b[1;32m    231\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mServerSelectionTimeoutError\u001b[0m: localhost:27017: [Errno 111] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 6266603d3269f60a67f61383, topology_type: Unknown, servers: [<ServerDescription ('localhost', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:27017: [Errno 111] Connection refused',)>]>"
     ]
    }
   ],
   "source": [
    "from sacred.observers import MongoObserver\n",
    "from sacred import Experiment\n",
    "\n",
    "ex = Experiment(\"sacred_scopes\", interactive=True)\n",
    "ex.observers.append(MongoObserver(url='mongodb://mongo_user:mongo_password@localhost:27017',\n",
    "                                  db_name='sacred'))  # Tutaj podajemy dane uwierzytelniające i nazwę bazy skonfigurowane w pliku .env podczas uruchamiania bazy.\n",
    "# W przypadku instancji na Jenkinsie url będzie wyglądał następująco: mongodb://mongo_user:mongo_password_IUM_2021@localhost:27017\n",
    "@ex.config\n",
    "def my_config():\n",
    "    recipient = \"Świecie\"\n",
    "    greeting = \"Witaj\"\n",
    "\n",
    "@ex.capture\n",
    "def prepare_message(recipient, greeting):\n",
    "    return \"{0} {1}!\".format(greeting, recipient)\n",
    "\n",
    "@ex.main\n",
    "def my_main(recipient, greeting):\n",
    "    print(prepare_message()) ## Nie musimy przekazywać wartości\n",
    "    \n",
    "ex.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- Informacje o eksperymencie można obejrzeć na Omniboard: http://127.0.0.1:9000/sacred\n",
    "- Instancja na Jenkinsie: http://tzietkiewicz.vm.wmi.amu.edu.pl:9000/sacred\n",
    "<img width=\"75%\" src=\"IUM_07/omniboard.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Metryki\n",
    "\n",
    "- W trakcie eksperymentu możemy śledzić [metryki](https://sacred.readthedocs.io/en/stable/collected_information.html#metrics-api), np. aktualny loss\n",
    "- W tym celu wystarczy:\n",
    "  - dodać do funkcji udekorowanej `@ex.main` albo `@ex.capure` parametr `_run`\n",
    "  - potem wywołać np. `_run.log_scalar()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO - sacred_scopes - Running command 'my_main'\n",
      "INFO - sacred_scopes - Started run with ID \"9\"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Witaj Świecie!\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO - sacred_scopes - Completed after 0:00:50\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<sacred.run.Run at 0x7f423c2de550>"
      ]
     },
     "execution_count": 192,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sacred.observers import MongoObserver\n",
    "from sacred import Experiment\n",
    "import random\n",
    "import time\n",
    "\n",
    "ex = Experiment(\"sacred_scopes\", interactive=True)\n",
    "ex.observers.append(MongoObserver(url='mongodb://mongo_user:mongo_password@localhost:27017',\n",
    "                                  db_name='sacred'))  # Tutaj podajemy dane uwierzytelniające i nazwę bazy skonfigurowane w pliku .env podczas uruchamiania bazy.\n",
    "# W przypadku instancji na Jenkinsie url będzie wyglądał następująco: mongodb://mongo_user:mongo_password_IUM_2021@localhost:27017\n",
    "@ex.config\n",
    "def my_config():\n",
    "    recipient = \"Świecie\"\n",
    "    greeting = \"Witaj\"\n",
    "\n",
    "@ex.capture\n",
    "def prepare_message(recipient, greeting):\n",
    "    return \"{0} {1}!\".format(greeting, recipient)\n",
    "\n",
    "@ex.main\n",
    "def my_main(recipient, greeting, _run):\n",
    "    print(prepare_message()) ## Nie musimy przekazywać wartości  \n",
    "    counter = 0\n",
    "    while counter < 20:\n",
    "        counter+=1\n",
    "        value = counter\n",
    "        ms_to_wait = random.randint(5, 5000)\n",
    "        time.sleep(ms_to_wait/1000)\n",
    "        noise = 1.0 + 0.1 * (random.randint(0, 10) - 5)\n",
    "        # This will add an entry for training.loss metric in every second iteration.\n",
    "        # The resulting sequence of steps for training.loss will be 0, 2, 4, ...\n",
    "        if counter % 2 == 0:\n",
    "           _run.log_scalar(\"training.loss\", value * 1.5 * noise, counter)\n",
    "        # Implicit step counter (0, 1, 2, 3, ...)\n",
    "        # incremented with each call for training.accuracy:\n",
    "        _run.log_scalar(\"training.accuracy\", value * 2 * noise)\n",
    "\n",
    "ex.run()    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Wartości metryk możemy na żywo śledzić w Omniboard\n",
    "<img src=\"IUM_07/metrics.png\" width=\"75%\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Zadanie  [15 pkt] (do 9 V 2021)\n",
    "1. \"Owiń\" wywołanie swojego eksperymentu za pomocą Sacred, w ten sposób, żeby zapisane zostały [10pkt]:\n",
    " - parametry, z którymi wywołany był trening\n",
    " - powstały plik z modelem (jako artefakt)\n",
    " - kod źródłowy użyty do przeprowadzenia treningu\n",
    " - metryki\n",
    " \n",
    "Jako nazwę eksperymentu użyj swojego numeru indeksu tak, żebyś mogła/mógł je odnaleźć w Omniboard\n",
    "\n",
    "2. Wykorzystaj 2 obserwatory [5pkt]: \n",
    " - MongoObserver, skorzytaj nastęþującego URL: `mongodb://admin:IUM_2021@172.17.0.1:27017` (będziesz mógł przeglądać wyniki na http://tzietkiewicz.vm.wmi.amu.edu.pl:9000/sacred)\n",
    " - FileObserver - zapisane pliki zarchiwizuj na Jenkinsie jako jego artefakty\n"
   ]
  }
 ],
 "metadata": {
  "author": "Tomasz Ziętkiewicz",
  "celltoolbar": "Slideshow",
  "email": "tomasz.zietkiewicz@amu.edu.pl",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "lang": "pl",
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "slideshow": {
   "slide_type": "slide"
  },
  "subtitle": "7.Sacred[laboratoria]",
  "title": "Inżynieria uczenia maszynowego",
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": false,
   "toc_window_display": false
  },
  "year": "2021"
 },
 "nbformat": 4,
 "nbformat_minor": 4
}