From e2422914001f63e3b69ab5de0fed43506a8129cf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Pawe=C5=82=20Sk=C3=B3rzewski?= Date: Tue, 6 Apr 2021 11:16:04 +0200 Subject: [PATCH] =?UTF-8?q?Wyk=C5=82ad=203=20-=20nadmierne=20dopasowanie?= =?UTF-8?q?=20i=20regularyzacja?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- wyk/2a_Regresja_logistyczna.ipynb | 8 +- ...waluacja_regularyzacja_optymalizacja.ipynb | 2692 +++++++++++++++++ wyk/3_Metody_ewaluacji_i_optymalizacji.ipynb | 1805 ----------- 3 files changed, 2696 insertions(+), 1809 deletions(-) create mode 100644 wyk/3_Ewaluacja_regularyzacja_optymalizacja.ipynb delete mode 100644 wyk/3_Metody_ewaluacji_i_optymalizacji.ipynb diff --git a/wyk/2a_Regresja_logistyczna.ipynb b/wyk/2a_Regresja_logistyczna.ipynb index 534413e..04f0d92 100644 --- a/wyk/2a_Regresja_logistyczna.ipynb +++ b/wyk/2a_Regresja_logistyczna.ipynb @@ -8,8 +8,8 @@ } }, "source": [ - "### AITech — Uczenie maszynowe\n", - "# 3. Regresja logistyczna" + "### Uczenie maszynowe\n", + "# 2a. Regresja logistyczna" ] }, { @@ -86,7 +86,7 @@ } }, "source": [ - "## 3.1. Dwuklasowa regresja logistyczna" + "## 2a.1. Dwuklasowa regresja logistyczna" ] }, { @@ -5637,7 +5637,7 @@ } }, "source": [ - "## 3.2. Wieloklasowa regresja logistyczna" + "## 2a.2. Wieloklasowa regresja logistyczna" ] }, { diff --git a/wyk/3_Ewaluacja_regularyzacja_optymalizacja.ipynb b/wyk/3_Ewaluacja_regularyzacja_optymalizacja.ipynb new file mode 100644 index 0000000..fe6d613 --- /dev/null +++ b/wyk/3_Ewaluacja_regularyzacja_optymalizacja.ipynb @@ -0,0 +1,2692 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Uczenie maszynowe\n", + "# 3. Ewaluacja, regularyzacja, optymalizacja" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## 3.1. Metodologia testowania" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "W uczeniu maszynowym bardzo ważna jest ewaluacja budowanego modelu. Dlatego dobrze jest podzielić posiadane dane na odrębne zbiory – osobny zbiór danych do uczenia i osobny do testowania. W niektórych przypadkach potrzeba będzie dodatkowo wyodrębnić tzw. zbiór walidacyjny." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Zbiór uczący a zbiór testowy" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "* Na zbiorze uczącym (treningowym) uczymy algorytmy, a na zbiorze testowym sprawdzamy ich poprawność.\n", + "* Zbiór uczący powinien być kilkukrotnie większy od testowego (np. 4:1, 9:1 itp.).\n", + "* Zbiór testowy często jest nieznany.\n", + "* Należy unikać mieszania danych testowych i treningowych – nie wolno „zanieczyszczać” danych treningowych danymi testowymi!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "Czasami potrzebujemy dobrać parametry modelu, np. $\\alpha$ – który zbiór wykorzystać do tego celu?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Zbiór walidacyjny" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "Do doboru parametrów najlepiej użyć jeszcze innego zbioru – jest to tzw. **zbiór walidacyjny**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + " * Zbiór walidacyjny powinien mieć wielkość zbliżoną do wielkości zbioru testowego, czyli np. dane można podzielić na te trzy zbiory w proporcjach 3:1:1, 8:1:1 itp." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Walidacja krzyżowa" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "Którą część danych wydzielić jako zbiór walidacyjny tak, żeby było „najlepiej”?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + " * Niech każda partia danych pełni tę rolę naprzemiennie!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "\n", + "Żródło: https://chrisjmccormick.wordpress.com/2013/07/31/k-fold-cross-validation-with-matlab-code/" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Walidacja krzyżowa\n", + "\n", + "* Podziel dane $D = \\left\\{ (x^{(1)}, y^{(1)}), \\ldots, (x^{(m)}, y^{(m)})\\right\\} $ na $N$ rozłącznych zbiorów $T_1,\\ldots,T_N$\n", + "* Dla $i=1,\\ldots,N$, wykonaj:\n", + " * Użyj $T_i$ do walidacji i zbiór $S_i$ do trenowania, gdzie $S_i = D \\smallsetminus T_i$. \n", + " * Zapisz model $\\theta_i$.\n", + "* Akumuluj wyniki dla modeli $\\theta_i$ dla zbiorów $T_i$.\n", + "* Ustalaj parametry uczenia na akumulowanych wynikach." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Walidacja krzyżowa – wskazówki\n", + "\n", + "* Zazwyczaj ustala się $N$ w przedziale od $4$ do $10$, tzw. $N$-krotna walidacja krzyżowa (*$N$-fold cross validation*). \n", + "* Zbiór $D$ warto zrandomizować przed podziałem.\n", + "* W jaki sposób akumulować wyniki dla wszystkich zbiórow $T_i$?\n", + "* Po ustaleniu parametrów dla każdego $T_i$, trenujemy model na całych danych treningowych z ustalonymi parametrami.\n", + "* Testujemy na zbiorze testowym (jeśli nim dysponujemy)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### _Leave-one-out_\n", + "\n", + "Jest to szczególny przypadek walidacji krzyżowej, w której $N = m$." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "* Jaki jest rozmiar pojedynczego zbioru $T_i$?\n", + "* Jakie są zalety i wady tej metody?\n", + "* Kiedy może być przydatna?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Zbiór walidujący a algorytmy optymalizacji\n", + "\n", + "* Gdy błąd rośnie na zbiorze uczącym, mamy źle dobrany parametr $\\alpha$. Należy go wtedy zmniejszyć.\n", + "* Gdy błąd zmniejsza się na zbiorze trenującym, ale rośnie na zbiorze walidującym, mamy do czynienia ze zjawiskiem **nadmiernego dopasowania** (*overfitting*).\n", + "* Należy wtedy przerwać optymalizację. Automatyzacja tego procesu to _early stopping_." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## 3.2. Miary jakości" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "Aby przeprowadzić ewaluację modelu, musimy wybrać **miarę** (**metrykę**), jakiej będziemy używać.\n", + "\n", + "Jakiej miary użyc najlepiej?\n", + " * To zależy od rodzaju zadania.\n", + " * Innych metryk używa się do regresji, a innych do klasyfikacji" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Metryki dla zadań regresji\n", + "\n", + "Dla zadań regresji możemy zastosować np.:\n", + " * błąd średniokwadratowy (*root-mean-square error*, RMSE):\n", + " $$ \\mathrm{RMSE} \\, = \\, \\sqrt{ \\frac{1}{m} \\sum_{i=1}^{m} \\left( \\hat{y}^{(i)} - y^{(i)} \\right)^2 } $$\n", + " * średni błąd bezwzględny (*mean absolute error*, MAE):\n", + " $$ \\mathrm{MAE} \\, = \\, \\frac{1}{m} \\sum_{i=1}^{m} \\left| \\hat{y}^{(i)} - y^{(i)} \\right| $$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "W powyższych wzorach $y^{(i)}$ oznacza **oczekiwaną** wartości zmiennej $y$ w $i$-tym przykładzie, a $\\hat{y}^{(i)}$ oznacza wartość zmiennej $y$ w $i$-tym przykładzie wyliczoną (**przewidzianą**) przez nasz model." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Metryki dla zadań klasyfikacji\n", + "\n", + "Aby przedstawić kilka najpopularniejszych metryk stosowanych dla zadań klasyfikacyjnych, posłużmy się następującym przykładem:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "# Przydatne importy\n", + "\n", + "import ipywidgets as widgets\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "import pandas\n", + "import random\n", + "import seaborn\n", + "\n", + "%matplotlib inline" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "def powerme(x1,x2,n):\n", + " \"\"\"Funkcja, która generuje n potęg dla zmiennych x1 i x2 oraz ich iloczynów\"\"\"\n", + " X = []\n", + " for m in range(n+1):\n", + " for i in range(m+1):\n", + " X.append(np.multiply(np.power(x1,i),np.power(x2,(m-i))))\n", + " return np.hstack(X)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "def plot_data_for_classification(X, Y, xlabel=None, ylabel=None, Y_predicted=[], highlight=None):\n", + " \"\"\"Wykres danych dla zadania klasyfikacji\"\"\"\n", + " fig = plt.figure(figsize=(16*.6, 9*.6))\n", + " ax = fig.add_subplot(111)\n", + " fig.subplots_adjust(left=0.1, right=0.9, bottom=0.1, top=0.9)\n", + " X = X.tolist()\n", + " Y = Y.tolist()\n", + " X1n = [x[1] for x, y in zip(X, Y) if y[0] == 0]\n", + " X1p = [x[1] for x, y in zip(X, Y) if y[0] == 1]\n", + " X2n = [x[2] for x, y in zip(X, Y) if y[0] == 0]\n", + " X2p = [x[2] for x, y in zip(X, Y) if y[0] == 1]\n", + " \n", + " if len(Y_predicted) > 0:\n", + " Y_predicted = Y_predicted.tolist()\n", + " X1tn = [x[1] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 0 and yp[0] == 0]\n", + " X1fn = [x[1] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 1 and yp[0] == 0]\n", + " X1tp = [x[1] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 1 and yp[0] == 1]\n", + " X1fp = [x[1] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 0 and yp[0] == 1]\n", + " X2tn = [x[2] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 0 and yp[0] == 0]\n", + " X2fn = [x[2] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 1 and yp[0] == 0]\n", + " X2tp = [x[2] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 1 and yp[0] == 1]\n", + " X2fp = [x[2] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 0 and yp[0] == 1]\n", + " \n", + " if highlight == 'tn':\n", + " ax.scatter(X1tn, X2tn, c='r', marker='x', s=100, label='Dane')\n", + " ax.scatter(X1fn, X2fn, c='k', marker='o', s=50, label='Dane')\n", + " ax.scatter(X1tp, X2tp, c='k', marker='o', s=50, label='Dane')\n", + " ax.scatter(X1fp, X2fp, c='k', marker='x', s=50, label='Dane')\n", + " elif highlight == 'fn':\n", + " ax.scatter(X1tn, X2tn, c='k', marker='x', s=50, label='Dane')\n", + " ax.scatter(X1fn, X2fn, c='g', marker='o', s=100, label='Dane')\n", + " ax.scatter(X1tp, X2tp, c='k', marker='o', s=50, label='Dane')\n", + " ax.scatter(X1fp, X2fp, c='k', marker='x', s=50, label='Dane')\n", + " elif highlight == 'tp':\n", + " ax.scatter(X1tn, X2tn, c='k', marker='x', s=50, label='Dane')\n", + " ax.scatter(X1fn, X2fn, c='k', marker='o', s=50, label='Dane')\n", + " ax.scatter(X1tp, X2tp, c='g', marker='o', s=100, label='Dane')\n", + " ax.scatter(X1fp, X2fp, c='k', marker='x', s=50, label='Dane')\n", + " elif highlight == 'fp':\n", + " ax.scatter(X1tn, X2tn, c='k', marker='x', s=50, label='Dane')\n", + " ax.scatter(X1fn, X2fn, c='k', marker='o', s=50, label='Dane')\n", + " ax.scatter(X1tp, X2tp, c='k', marker='o', s=50, label='Dane')\n", + " ax.scatter(X1fp, X2fp, c='r', marker='x', s=100, label='Dane')\n", + " else:\n", + " ax.scatter(X1tn, X2tn, c='r', marker='x', s=50, label='Dane')\n", + " ax.scatter(X1fn, X2fn, c='g', marker='o', s=50, label='Dane')\n", + " ax.scatter(X1tp, X2tp, c='g', marker='o', s=50, label='Dane')\n", + " ax.scatter(X1fp, X2fp, c='r', marker='x', s=50, label='Dane')\n", + "\n", + " else:\n", + " ax.scatter(X1n, X2n, c='r', marker='x', s=50, label='Dane')\n", + " ax.scatter(X1p, X2p, c='g', marker='o', s=50, label='Dane')\n", + " \n", + " if xlabel:\n", + " ax.set_xlabel(xlabel)\n", + " if ylabel:\n", + " ax.set_ylabel(ylabel)\n", + " \n", + " ax.margins(.05, .05)\n", + " return fig" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "# Wczytanie danych\n", + "import pandas\n", + "import numpy as np\n", + "\n", + "alldata = pandas.read_csv('data-metrics.tsv', sep='\\t')\n", + "data = np.matrix(alldata)\n", + "\n", + "m, n_plus_1 = data.shape\n", + "n = n_plus_1 - 1\n", + "\n", + "X2 = powerme(data[:, 1], data[:, 2], n)\n", + "Y2 = np.matrix(data[:, 0]).reshape(m, 1)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm8AAAFmCAYAAAA70X3dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df3Dc913n8ddbieS5rndo7bjFdWKSYk2H2NCS6kKhmiqFJpeIay2LFiVnIDeXOZOjmXHtwMR33EGHH3PAQHzuEcqlpkPL+JrNTSXFUBU3DZRieoUomSS160ulhpC4yiWqXdq1OCQn+74/vt+v/dVqV/pK2t3v97v7fMzsaPfz/X7Xn/16V/vS9/PL3F0AAADIh660KwAAAIDkCG8AAAA5QngDAADIEcIbAABAjhDeAAAAcoTwBgAAkCNXpl2BNFx11VV+7bXXpl0NAACARZ544olvufuW5fbpyPB27bXXanJyMu1qAAAALGJm/7DSPjSbAgAA5AjhDQAAIEdSD29m9gkze8XMTtXZbmb2UTObNrNnzOyG2LZbzezZcNuh1tUaAAAgHamHN0l/LOnWZbbfJqk3vO2T9DFJMrMrJD0Qbr9e0h1mdn1TawoAAJCy1MObu39J0vlldtkt6VMe+Iqk15vZVkk3Spp29+fcfUHSQ+G+AAAAbSv18JbANkkvxh6fDcvqlQMAALStPIQ3q1Hmy5TXfhKzfWY2aWaTs7OzDascAABAK+UhvJ2VdE3s8dWSZpYpr8ndH3T3Pnfv27Jl2bnvAAAAMisP4e24pJ8LR52+U9J33P0lSY9L6jWz68ysR9Lt4b4AqrlLY2PBzyTlAIDMSj28mdmnJf1vSW81s7NmdpeZ3W1md4e7TEh6TtK0pI9L+gVJcvdXJd0j6YSkM5IedvfTLX8BQB6Mj0vDw9KBA5eDmnvweHg42A4AyIXUl8dy9ztW2O6SPlRn24SCcAdgOUND0v790pEjwePDh4PgduRIUD40lG79AACJpR7eALSAWRDYpCCwRSFu//6g3GqN/wEAZJF5B/Z16evrcxamR0dyl7pivSUqFYIbAGSImT3h7n3L7ZN6nzcALRL1cYuL94EDAOQC4Q3oBFFwi/q4VSqX+8AR4AAgV+jzBnSC8fHLwS3q4xbvAzcwIO3Zk24dAQCJEN6ATjA0JI2OBj+jPm5RgBsYYLQpAOQI4Q3oBGa1r6zVKwcAZBZ93gAAAHKE8AYAAJAjhDcAAIAcIbwBAADkCOENAAAgRwhvAAAAOUJ4AwAAyBHCGwAAQI4wSS8AADWU58sqnS5p6tyUejf3amTniIobimlXCyC8AQBQ7eQLJzV4bFAVr2ju4pwK3QUdPHFQE3sn1L+9P+3qocPRbAoAQEx5vqzBY4MqL5Q1d3FOkjR3cU7lhaD8wsKFlGuITkd4AwAgpnS6pIpXam6reEWlU6UW1whYjPAGAEDM1LmpS1fcqs1dnNP0+ekW1whYjPAGAEBM76YdKtiGmtsKtkE7Nn1/i2sELEZ4AwAgZuTvC+r65/ma27r+eV4jz72uxTUCFiO8AQAQU/ypOzQx/wEV56WCB5MyFPxKFeelifkPaONP3ZFyDdHpmCoEAIA4M/X/7sOaOfAhlf78Y5reJO04/6pG3v0ftPHwA5JZ2jVEhzN3T7sOLdfX1+eTk5NpVwMAkGXuUlesgapSIbih6czsCXfvW24fmk0BAKjmLh04sLjswIGgHEgZ4Q0AgLgouB05Iu3fH1xx278/eEyAQwbQ5w0AgLjx8cvB7fDhoKn08OFg25Ej0sCAtGdPunVER8tEeDOzWyUdkXSFpKPu/ltV239J0t7w4ZWSfkDSFnc/b2bPSypLek3Sqyu1EwMAsKyhIWl0NPgZ9XGLAtzAQFAOpCj1AQtmdoWkr0u6WdJZSY9LusPdv1Zn//dJOuDuPx4+fl5Sn7t/K+m/yYAFAACQRXkZsHCjpGl3f87dFyQ9JGn3MvvfIenTLakZAABAxmQhvG2T9GLs8dmwbAkze52kWyV9Jlbskj5vZk+Y2b56/4iZ7TOzSTObnJ2dbUC1AXQ8d2lsbGkH9nrlANAAWQhvtSbNqfcb732S/sbdz8fK3uXuN0i6TdKHzOzdtQ509wfdvc/d+7Zs2bK+GgOAFHRsHx5ePAIxGqk4PBxsB4AGy0J4OyvpmtjjqyXN1Nn3dlU1mbr7TPjzFUljCpphAaD5hoaWTiERn2KCju0AmiALo00fl9RrZtdJ+qaCgPZvqncys++RNCDpZ2JlBUld7l4O798i6ddaUmsAqJ5C4siR4H58igkAaLDUr7y5+6uS7pF0QtIZSQ+7+2kzu9vM7o7tukfS5919Llb2JkknzexpSX8n6bPu/uetqjsALApwEYIbgCbKwpU3ufuEpImqsj+sevzHkv64quw5SW9rcvUAoL56yygR4AA0SepX3gAgt1hGCUAKMnHlDQByiWWUAKSA8AYAa8UySgBSQHgDgLUyq31lrV45ADQAfd4AAAByhPAGAACQI4Q3AACAHCG8AQAA5AjhDQAAIEcIbwAAADlCeAMAAMgRwhsAAECOEN4AAAByhPAGAACQI4Q3AACAHCG8AQAA5AjhDQAAIEcIbwAAADlCeAMAAMgRwhsAAECOEN4AAAByhPAGAACQI4Q3AACAHCG8AQAA5AjhDQAAIEcIbwAAADlCeAMAAMiRTIQ3M7vVzJ41s2kzO1Rj+01m9h0zeyq8/UrSYwEAANrJlWlXwMyukPSApJslnZX0uJkdd/evVe361+7+r9d4LAAAQFvIwpW3GyVNu/tz7r4g6SFJu1twLAAAQO5kIbxtk/Ri7PHZsKzaj5rZ02b2OTPbucpjZWb7zGzSzCZnZ2cbUW8AAICWS73ZVJLVKPOqx09K+j53v2Bmg5LGJfUmPDYodH9Q0oOS1NfXV3MfIOvK82WVTpc0dW5KvZt7NbJzRMUNxbSrBQBooSyEt7OSrok9vlrSTHwHd/9u7P6Emf2BmV2V5FigXZx84aQGjw2q4hXNXZxTobuggycOamLvhPq396ddPQBAi2Sh2fRxSb1mdp2Z9Ui6XdLx+A5m9r1mZuH9GxXU+1ySY4F2UJ4va/DYoMoLZc1dnJMkzV2cU3khKL+wcCHlGgIAWiX18Obur0q6R9IJSWckPezup83sbjO7O9ztA5JOmdnTkj4q6XYP1Dy29a8CaK7S6ZIqXqm5reIVlU6VWlwjAEBastBsKnefkDRRVfaHsfu/L+n3kx4LtJupc1OXrrhVm7s4p+nz0y2uEQAgLalfeQOwst7NvSp0F2puK3QXtGPTjhbXCACQFsIbkAMjO0fUZbU/rl3WpZFdIy2uEQAgLYQ3IAeKG4qa2DuhYk/x0hW4QndBxZ6gfGPPxpRrGHKXxsaCn0nKAQCrlok+bwBW1r+9XzP3zqh0qqTp89PasWmHRnaNZCe4SdL4uDQ8LO3fLx0+LJkFge3AAenIEWl0VNqzJ+1aAkCuEd6AHNnYs1F33XBX2tWob2goCG5HjgSPDx++HNz27w+2AwDWhfAGoHHMgsAmBYEtCnHxK3EAgHWhz1u7ou8R0hIPcBGCGwA0DOGtXUV9jw4cuBzUor5Hw8PBdqAZovdZXPx9CABYF8Jbu4r3PYq+OOl7hGarfp9VKkvfhwCAdaHPW7ui7xHSMD5+ObhF77P4+3BggNGmALBO5h34l3BfX59PTk6mXY3WcJe6YhdYKxWCG5rHPQhwQ0OL32f1ygEAi5jZE+7et9w+NJu2M/oeodXMgitr1QGtXjkAYNUIb+2KvkcAALQl+ry1K/oeAQDQlghv7WpoKFiKKN7HKApwAwOMNgUAIKdoNm1X9D3CajGxMwDkAuENQICJnTtWeb6so08e1X2P3qejTx5Veb6cdpUALINmUwAqz5dV2v4tTd33DvU+ekQjBxZUPPwAEzt3gJMvnNTgsUFVvKK5i3MqdBd08MRBTeydUP/2/rSr11xMbYOcYp43oMMt+fL2K9W18Komjkn9L4iJndtYeb6sbfdvU3lh6ZW2Yk9RM/fOaGPPxhRq1iJjY8FV5fh7PD5Sf3SUgV1oOeZ5A7Cs8nxZg8cGVV4oa+7inCRpzl5VeYM0uFe60COCWxsrnS6p4pWa2ypeUelUqcU1ajGWEUROEd6ADrbsl7ek0k4xL2Abmzo3dSm0V5u7OKfp89MtrlGLRSPwowDX1bV0iiUggwhvQAdb9st7gzR9Sx8TO7ex3s29KnQXam4rdBe0Y9OOFtcoBfE5MCMEN2Qc4Q3oYCt+ef/0z1++KsFo07YzsnNEXVb7a6DLujSya6TFNUoBywgihwhvQAdb+cv79uAqRDThM9pKcUNRE3snVOwpXgrxhe6Cij1BeVsPVpBYRhC5xWhToMPVmiqiy7o6Y6oISJIuLFxQ6VRJ0+entWPTDo3sGmn/4CYx2hSZlGS0KeENQOd+eaOzMc8bMojwVgfhDZfwyxsAkCG5mefNzG41s2fNbNrMDtXYvtfMnglvXzazt8W2PW9mXzWzp8yMRIbVYUkoAEDOpL48lpldIekBSTdLOivpcTM77u5fi+3295IG3P3bZnabpAcl/Uhs+3vc/VstqzTaR3ySTino98IknQCADEs9vEm6UdK0uz8nSWb2kKTdki6FN3f/cmz/r0i6uqU1RPuKz/F05MjlEMcknQCAjMpCs+k2SS/GHp8Ny+q5S9LnYo9d0ufN7Akz29eE+qHdMUknACBHshDean1D1hxFYWbvURDe7osVv8vdb5B0m6QPmdm76xy7z8wmzWxydnZ2vXVGO2GSTgBAjmQhvJ2VdE3s8dWSZqp3MrMfknRU0m53PxeVu/tM+PMVSWMKmmGXcPcH3b3P3fu2bNnSwOoj15ikEwCQM1kIb49L6jWz68ysR9Ltko7HdzCz7ZJGJf2su389Vl4ws2J0X9Itkk61rOZ55R5MTlkdTOqVt7Px8aULUccXqma0KQAgY1IPb+7+qqR7JJ2QdEbSw+5+2szuNrO7w91+RdJmSX9QNSXImySdNLOnJf2dpM+6+5+3+CXkD9NjXDY0FMyiHu/jFgU4loQCgM6W0YsdTNLbiaqbCqunx6CzPgAAqSyhlmSS3ixMFYJWY3oMAABWltG5QLny1sncpa5Yy3mlQnADEijPl1U6XdLUuSn1bu7VyM4RFTcU064WgGaIX2mLNPFiB2ub1kF4U8vfjEC7OPnCSQ0eG1TFK5q7OKdCd0Fd1qWJvRPq396fdvUANEMLL3bkZm1TtBjTYwBrUp4va/DYoMoLZc1dnJMkzV2cU3khKL+wcCHlGgJouAzOBUp460RMjwGsSel0SRWv1NxW8YpKp0otrhGApsroxQ4GLHSiaHqMoaGl02MMDDA9BlDH1LmpS1fcqs1dnNP0+ekW1whAU9W72CEF5QMDDR9tmgThrROZ1X6z1SsHIEnq3dyrQnehZoArdBe0Y9OOFGoFoGkyerGDZlMASGhk54i6rPavzS7r0siukRbXCEBTRRc1qgcn1CtvEcIbACRU3FDUxN4JFXuKKnQXJAVX3Io9QfnGno0p1xBAJ6DZFABWoX97v2bunVHpVEnT56e1Y9MOjewaIbgBaBnCGwCs0saejbrrhrvSrgaADkWzKQAAQI4Q3gAAAHKE8AYAAJAjhDeky10aG1s6S3W9cgAAOhzhDekaH5eGhxcvMxItRzI8zFJdAABUIbwhXUNDS9eJi68jx1JdAIBGaZPWHsIb0hUtMxIFuK6upevIAQDQCG3S2kN4Q/riC/1GCG4AgEZrk9YewhvSF3144uJ/FQEA0Aht0tpDeEO6qv/qqVSW/lUEAECjtEFrD+EN6RofX/pXT/yvopz0PwAA5EQbtPYQ3pCuoSFpdHTxXz1RgBsdzU3/AwBADrRJaw8L0yNdZtKePcnLAQBYq3qtPVJQPjCQi+8ewhsAAOgMUWvP0NDS1p6Bgdy09hDeAABAZ2iT1h76vAEAAORIJsKbmd1qZs+a2bSZHaqx3czso+H2Z8zshqTHAgAAtJPUw5uZXSHpAUm3Sbpe0h1mdn3VbrdJ6g1v+yR9bBXHAgAAtI0s9Hm7UdK0uz8nSWb2kKTdkr4W22e3pE+5u0v6ipm93sy2Sro2wbEAgCYpz5dVOl3S1Lkp9W7u1cjOERU3FNOuFtDWEoc3M7tZ0k9LesDdnzKzfe7+YAPqsE3Si7HHZyX9SIJ9tiU8FgDQBCdfOKnBY4OqeEVzF+dU6C7o4ImDmtg7of7t/WlXD2hbq2k2/QVJvyTpZ8zsxyW9vUF1qLUeRfUsefX2SXJs8ARm+8xs0swmZ2dnV1lFAEBceb6swWODKi+UNXdxTpI0d3FO5YWg/MLChZRrCLSv1YS3WXf/R3f/RUm3SPqXDarDWUnXxB5fLWkm4T5JjpUkufuD7t7n7n1btmxZd6UBoJOVTpdU8UrNbRWvqHSq1OIaAZ1jNeHts9Eddz8k6VMNqsPjknrN7Doz65F0u6TjVfscl/Rz4ajTd0r6jru/lPBYAECDTZ2bunTFrdrcxTlNn59ucY2AzrFieDOz/2Zm5u6PxMvd/b83ogLu/qqkeySdkHRG0sPuftrM7jazu8PdJiQ9J2la0scVNOHWPbYR9QIA1Ne7uVeF7kLNbYXugnZs2tHiGgGdw3yFRVjN7DckvU3SiLv/k5ndIulX3f1drahgM/T19fnk5GTa1QCA3CrPl7Xt/m0qL5SXbCv2FDVz74w29mxMoWZAvpnZE+7et9w+K155c/f/LOnTkv7KzE5KulcSk+ECQAcrbihqYu+Eij3FS1fgCt0FFXuCcoIbGsJdGhsLfiYp7xArThViZj8h6d9LmpO0VdJd7v5ssysGAMi2/u39mrl3RqVTJU2fn9aOTTs0smuE4IbGGR+Xhoel/fuDxePNgsB24IB05EiwyHyO1iRtlCTzvP2ypP/i7ifN7AcllczsoLv/RZPrBgDIuI09G3XXDXelXQ20q6GhILgdORI8Pnz4cnDbvz/Y3oFWDG/u/uOx+181s9skfUbSjzWzYgAAoMOZBYFNCgJbFOLiV+I60IoDFmoeZPYv3P3/NaE+LcGABQAAcsRd6op1069U2ja4NWTAQi15Dm4AsB7l+bKOPnlU9z16n44+eVTl+aWjLQE0UNTHLe7AgY4drCBlY2F6AMgF1vIEWiw+OCFqKo0eSx3bdLqmK28A0GlYyxNIwfj44uAW9YGLBjGMj6ddw1QQ3gAgAdbyBFIwNBRMBxK/whYFuNFRRpsCAOpjLU8gBWa153GrV94huPIGAAmwlieArCC8AUACIztH1GW1f2V2WZdGdo20uEYAOhXNpgCQQLSWZ/Vo0y7rYi3PBivPl1U6XdLUuSn1bu7VyM4RFTcU064WkBlrmqQ371o2Sa97MBJmaGjxUOZ65QAy78LCBdbybKJa07FEAZnpWNAJkkzSS3hrprExFtQFgITK82Vtu3+bygtLJz4u9hQ1c+8MQRltr2krLCCh+IK60WzQLKgLADUxHQuQDH3emokFdQEgMaZjAZLhyluzxQNchOAGAEswHQuQDOGt2VhQFwASYToWIBnCWzNV93GrVJb2gQMASLo8HUuxp3jpClyhu6BiT5HpWIAY+rw1U70FdaWgfGCA0abVmF4F6Gj92/s1c+8M07EAy2CqkGYiiKwe06sAADoYU4WkLVo4tzqg1SsH06sAQCdzD/6Ir76wVK+8QxHekC1R03IU4Lq6ljY9AwDa0/h40PoS7xce/RE/PBxsB+ENGcT0KgDQmWh9SYTwhuxhepXsokkDQDPR+pII4Q2tk+SLn+lVso0mDQDNRuvLilINb2a2ycweNbOp8OcbauxzjZn9pZmdMbPTZrY/tu0jZvZNM3sqvA229hVkSB6uiCT54q83vUoU4AgH6aJJA0Cz0fqyMndP7SbpdyQdCu8fkvTbNfbZKumG8H5R0tclXR8+/oikX1ztv/uOd7zD287oaHDdav9+90olKKtUgsdSsD1t8fpE9ax+XKkEdY1eQ/zYWuVovfj/W3SLv+8AYK2SfE+0OUmTvlJ+WmmHZt4kPStpq18Oac8mOOYRSTc74W2xvLzh2/WLv9NCZ6Wy+P+w3V4fgHTk4UJEkyUJb2n3eXuTu78kSeHPNy63s5ldK+mHJf1trPgeM3vGzD5Rq9m1Y+Slk2e79mXopL5g0euKo0kDQCMMDQWTsce/F6LvjdFRumaEmh7ezOwLZnaqxm33Kp9no6TPSPqwu383LP6YpO+X9HZJL0n6vWWO32dmk2Y2OTs7u8ZXk3F5CEbt+sXfKX3Bql8XA0oANBKT2yez0qW5Zt6UsNlUUrekE5IOLvNc10o6leTfbctmU/fsN0nmpWl3rbJ+/huBJg0AaCrloNn0uKQ7w/t3KujPtoiZmaQ/knTG3e+v2rY19nCPpFNNqmf25eGKSLuPJM3Dlc/1okkDAFKX6sL0ZrZZ0sOStkt6QdIH3f28mb1Z0lF3HzSzfkl/Lemrkirhof/J3SfM7E8UNJm6pOcl/byHfeiW07KF6VspDwu6uwcBbWhocaCpV5438fMdyVqfQwBApiVZmD7V8JaWtgxv7R6Msq76yufhw0sfc/4BACtIEt6ubFVl0GRRZ86k5Wisek3CUlA+MMD/AwCgIdLu8wa0B/qCAcgaz8HKO1gTwhvQCAxvB5A1nTT/ZIeh2RQAgHYUn39SWtoXlxaB3CK8AQDQjqr73kYhjkFUucdoUwAA2pl7sGRipFIhuGVYktGm9HkDAKBdteuShB2O8AYAQDvKw8o7WBP6vAEA0I6Yf7JtEd4AAGhH0fyT8RV2ogA3MMBo0xwjvKHtlOfLKp0uaerclHo392pk54iKG4ppVwsAWouVd9oW4Q1t5eQLJzV4bFAVr2ju4pwK3QUdPHFQE3sn1L+9P+3qAQCwbgxYQNsoz5c1eGxQ5YWy5i7OSZLmLs6pvBCUX1i4kHINAQBYP8Ib2kbpdEkVr9TcVvGKSqdKLa4RAACNR3hD25g6N3Xpilu1uYtzmj4/3eIaAQDQeIQ3tI3ezb0qdBdqbit0F7Rj044W1wgAgMYjvKFtjOwcUZfVfkt3WZdGdo20uEYAADQe4Q1to7ihqIm9Eyr2FC9dgSt0F1TsCco39mxMuYYAAKwfU4WgrfRv79fMvTMqnSpp+vy0dmzaoZFdIwQ3AB2BeS47g3kHrm3W19fnk5OTaVejtdyDpVLiM20vVw4AyJVa81x2WRfzXOaMmT3h7n3L7UOzaacYH5eGhxcvRhwtWjw8HGwHAOQS81x2FsJbpxgaChYnPnLkcoA7cODyosWscQcAucU8l52FPm+dIlqMWAoC25Ejwf39+4NymkwBILeY57KzcOWtk8QDXITgBgC5xzyXnYXw1kmiptK4eB84AEAuMc9lZyG8dYrqPm6VytI+cACAXGKey85Cn7dOMT5+ObhFTaXxPnADA9KePa2tE9OXAEDDMM9l50h1njcz2ySpJOlaSc9L+ml3/3aN/Z6XVJb0mqRXo/lPkh5fjXneMhKUxsaCaUrigTJ+hXB0tPWBEgCAFOVhnrdDkh5z915Jj4WP63mPu7+96gWt5vjOZhYEoeqAVq+8FZi+BACAVUu72XS3pJvC+5+U9EVJ97XweKSJ6UsAAFi1tJtN/9HdXx97/G13f0ON/f5e0rcluaT/4e4Prub4ah3ZbJpl7lJX7CJwpUJwAwB0pEw0m5rZF8zsVI3b7lU8zbvc/QZJt0n6kJm9ew312Gdmk2Y2OTs7u9rD0SxMXwIAwKo0Pby5+3vdfVeN2yOSXjazrZIU/nylznPMhD9fkTQm6cZwU6Ljw2MfdPc+d+/bsmVL414g1o7pSwAAWLW0Bywcl3RneP9OSY9U72BmBTMrRvcl3SLpVNLjkWH1pi+JAtz4eNo1BAAgc9Lu87ZZ0sOStkt6QdIH3f28mb1Z0lF3HzSztyi42iYFAyz+p7v/5nLHr/Tv0uctI7I4fQkAAClK0uct1fCWFsIbAACoKeULC5kYsAAAAJAb4+PBBPLxvtdRH+3h4Ux06Ul7njcAGVOeL6t0uqSpc1Pq3dyrkZ0jKm4opl0tAGiN+ATyUtAXO2MTyNNsCuCSky+c1OCxQVW8ormLcyp0F9RlXZrYO6H+7f1pVw8AWiM+G0KkRRPI0+etDsIbsFR5vqxt929TeaG8ZFuxp6iZe2dY4BpA50hpAnn6vAFIrHS6pIpXam6reEWlU6UW1wgAUpLxCeQJbwAkSVPnpjR3ca7mtrmLc5o+P93iGgFACnIwgTzhrVHcpbGxpf+p9cqBjOnd3KtCd6HmtkJ3QTs27WhxjQAgBTmYQJ7w1ig5GFoMLGdk54i6rPavhC7r0siukRbXCABSMDQkjY4uHpwQBbjR0UyMNiW8NUp8aHEU4DI2tBhYTnFDURN7J1TsKV66AlfoLqjYE5QzWAFARzCT9uxZOjihXnkKGG3aSCkOLQYa5cLCBZVOlTR9flo7Nu3QyK4RglsjsSwcgGUwVUgdTZ0qJKWhxQByYmws6EoR/8Mu/off6Gjw1z2yiwCOJmKqkFbL+NBiABlAF4vGSmOwGH2ckTLCW6PkYGgxgAyoHrnW1bV0ZBuSSyNIEcCRMppNG4WmEACrQReLxqgOTtXrUDYrENPHGU1Cn7c6mhLe6AMBICm++BsrrfNJAEcT0OetlXIwtBhABtDFovGipui4VgQ3+jgjJYQ3AGilHMzenjutDlIEcKSM8AYArZSD2dtzJY0gRQBHyujzBgDIrzQGi9HHGU3EgIU6CG8A0CYIUmgzScLbla2qDAAADRcNCktaDrQB+rwBAADkCOENAAC0VhrLmrURwhsAAGgt1oddF/q8AQCA1oqvDystXdaMKXOWRXgDAACtFV8V48iRyyGOZeISYaoQAACQDtaHXSLza5ua2SYze9TMpsKfb6ixz1vN7KnY7btm9uFw20fM7JuxbYOtfxUAAGDVWB92zdIesHBI0mPu3ivpsfDxIu7+rLu/3d3fLukdkv5J0lhsl8PRdnefaEmtAQDA2rE+7Lqk3edtt6SbwvuflPRFSfcts/9PSPqGu/9Dc6sFAACapt76sFJQPjDAJMvLSPvK25vc/SVJCnBYF1gAAAxZSURBVH++cYX9b5f06aqye8zsGTP7RK1mVwAAkDFDQ8G6s/HBCVGAGx1ltOkKmh7ezOwLZnaqxm33Kp+nR9L7Jf2vWPHHJH2/pLdLeknS7y1z/D4zmzSzydnZ2TW8EmAVmIASqI3PBqTLy5dVD06oV45Fmh7e3P297r6rxu0RSS+b2VZJCn++ssxT3SbpSXd/OfbcL7v7a+5ekfRxSTcuU48H3b3P3fu2bNnSmBcH1MMElEBtfDaAdUu72fS4pDvD+3dKemSZfe9QVZNpFPxCeySdamjtgLWKT0AZfUkxASU6WXRlbffuxZ+NSkV63/v4bACrkOo8b2a2WdLDkrZLekHSB939vJm9WdJRdx8M93udpBclvcXdvxM7/k8UNJm6pOcl/XzUh245zPOGlogHtggTUKJTjY0FV9b275fuv186eHDxZ+Mnf1L60z/ls4GOl2SeNybpBZqJCSiBQPXV5/vvl6644vL2115b/FkBOlTmJ+kF2hoTUAKXRSMJoybTeHCTgitxfDaARAhvQDMwASWwlFlwxS3utdf4bACrlPYkvUB7YgJKYCl36f3vX1x28ODlQMdnA0iE8AY0QzQB5dDQ0gkoBwYYUYfOE12N/uxng8EJx48vHrRw//18NoCECG9AM0QTTSYtB9odV6OBhiG8AQCaj6vRQMMQ3gAAzcfVaKBhGG0KAACQI4Q3AACAHCG8AUAzRGt5Vs9bVq8cABIivAFAM4yPB2t5xieejabLGB4OtgPAGjBgAQCaYWjo8soBUjCqMr7qBqMrAawR4Q0AmqF6HrMoxMXnOQOANaDZFACaJR7gInkIbvTXAzKN8AYAzRL1cYvLw+Lr9NcDMo3wBgDNEIWdqI9bpXK5D1zWA1y8v15UV/rrAZlBnzcAaIY8r+VJfz0g08yz/Ndfk/T19fnk5GTa1QDQztyDABdfy3O58ixyl7piDTSVSvbrjNZoh/d3RpnZE+7et9w+NJsCQDNEa3ZWf4HVK8+avPbXQ2vQLzJVhDcAwGJ57q+H1mhmv0hGO6+I8AYAWKxef73oy5qrKqh+T3R1LX3PrBVX9VZEnzcAwGL0Z0JSzegXWX0Vr3p1kjYfNJOkzxujTQEAi0X98pKWozPV6xe53nDFaOcV0WwKAJ2CvkRolGb3i8zr6iQtQngDgE5BXyI0SrP7RTLaeVmENwDoFKycgEYZGpJGRxdfDYsC3Ojo+kebMtp5WQxYAIBOEv9ijNCXCFkyNhZcCY6/L+Pv29HRtu57mWTAAuENALKglSM8WTkBWdbho50zv8KCmX3QzE6bWcXM6lbUzG41s2fNbNrMDsXKN5nZo2Y2Ff58Q2tqDgAN1qr+aPQlQtblfXWSFki7z9spScOSvlRvBzO7QtIDkm6TdL2kO8zs+nDzIUmPuXuvpMfCxwCQP63oj0ZfIqAtpDrPm7ufkSRbPkXfKGna3Z8L931I0m5JXwt/3hTu90lJX5R0X3NqCwBN1Iq5reqNEIz+zYGBtu5LBLSLtK+8JbFN0ouxx2fDMkl6k7u/JEnhzzfWexIz22dmk2Y2OTs727TKAsCaNXtuq2aOEATQMk0Pb2b2BTM7VeO2O+lT1Chb9bV9d3/Q3fvcvW/Lli2rPRwAmq/Z/dHoSwS0haY3m7r7e9f5FGclXRN7fLWkmfD+y2a21d1fMrOtkl5Z578FAOlYbj1Hiak8AFySh2bTxyX1mtl1ZtYj6XZJx8NtxyXdGd6/U9IjKdQPANav2TPWA2gbaU8VssfMzkr6UUmfNbMTYfmbzWxCktz9VUn3SDoh6Yykh939dPgUvyXpZjObknRz+BgA8of+aAASYpJeAACAjMj8JL0AAABYHcIbAABAjhDeAAAAcoTwBgAAkCOENwAAgBwhvAEAAOQI4Q0AACBHCG8AAAA50pGT9JrZrKR/aNDTXSXpWw16rjzjPHAOIpyHAOeBcxDhPHAOIknOw/e5+5bldujI8NZIZja50kzInYDzwDmIcB4CnAfOQYTzwDmINOo80GwKAACQI4Q3AACAHCG8rd+DaVcgIzgPnIMI5yHAeeAcRDgPnINIQ84Dfd4AAAByhCtvAAAAOUJ4S8DMPmhmp82sYmZ1R4mY2a1m9qyZTZvZoVj5JjN71Mymwp9vaE3NGyvJ6zCzt5rZU7Hbd83sw+G2j5jZN2PbBlv/KtYn6f+lmT1vZl8NX+fkao/PuoTvhWvM7C/N7Ez4+dkf25bb90K9z3lsu5nZR8Ptz5jZDUmPzZME52Fv+PqfMbMvm9nbYttqfj7yJsE5uMnMvhN7n/9K0mPzJMF5+KXYOThlZq+Z2aZwW7u8Fz5hZq+Y2ak62xv7e8Hdua1wk/QDkt4q6YuS+ursc4Wkb0h6i6QeSU9Luj7c9juSDoX3D0n67bRf0xrPw6peR3hO/q+COWsk6SOSfjHt19GKcyDpeUlXrfccZvWW5HVI2irphvB+UdLXY5+JXL4Xlvucx/YZlPQ5SSbpnZL+NumxebklPA8/JukN4f3bovMQPq75+cjTLeE5uEnSn63l2LzcVvtaJL1P0l+003shfB3vlnSDpFN1tjf09wJX3hJw9zPu/uwKu90oadrdn3P3BUkPSdodbtst6ZPh/U9KGmpOTZtuta/jJyR9w90bNSFyFqz3/7Jj3gvu/pK7PxneL0s6I2lby2rYHMt9ziO7JX3KA1+R9Hoz25rw2LxY8bW4+5fd/dvhw69IurrFdWy29fx/dtR7ocodkj7dkpq1kLt/SdL5ZXZp6O8FwlvjbJP0YuzxWV3+onqTu78kBV9okt7Y4ro1ympfx+1a+iG9J7xk/ImcNhkmPQcu6fNm9oSZ7VvD8Vm3qtdhZtdK+mFJfxsrzuN7YbnP+Ur7JDk2L1b7Wu5ScNUhUu/zkSdJz8GPmtnTZvY5M9u5ymPzIPFrMbPXSbpV0mdixe3wXkiiob8Xrmxo1XLMzL4g6XtrbPpld38kyVPUKMvdUN7lzsMqn6dH0vsl/cdY8cck/bqC8/Lrkn5P0r9bW02bp0Hn4F3uPmNmb5T0qJn9n/Avs9xo4Htho4Jf1h929++Gxbl4L9SQ5HNeb5+2+B0RSvxazOw9CsJbf6w4958PJTsHTyroNnIh7Nc5Lqk34bF5sZrX8j5Jf+Pu8StU7fBeSKKhvxcIbyF3f+86n+KspGtij6+WNBPef9nMtrr7S+Fl0lfW+W81zXLnwcxW8zpuk/Sku78ce+5L983s45L+rBF1brRGnAN3nwl/vmJmYwoujX9JHfZeMLNuBcHtmLuPxp47F++FGpb7nK+0T0+CY/MiyXmQmf2QpKOSbnP3c1H5Mp+PPFnxHMT+WJG7T5jZH5jZVUmOzZHVvJYlrTFt8l5IoqG/F2g2bZzHJfWa2XXhVafbJR0Ptx2XdGd4/05JSa7kZdFqXseSfg3hl3xkj6Sao3IybsVzYGYFMytG9yXdosuvtWPeC2Zmkv5I0hl3v79qW17fC8t9ziPHJf1cOLrsnZK+EzYtJzk2L1Z8LWa2XdKopJ9196/Hypf7fORJknPwveHnQGZ2o4Lv3HNJjs2RRK/FzL5H0oBivyva6L2QRGN/L6Q9QiMPNwVfLmclzUt6WdKJsPzNkiZi+w0qGFH3DQXNrVH5ZkmPSZoKf25K+zWt8TzUfB01zsPrFPyC+p6q4/9E0lclPRO+Obem/ZqacQ4UjBp6Oryd7tT3goJmMg//v58Kb4N5fy/U+pxLulvS3eF9k/RAuP2rio1Qr/c7Io+3BOfhqKRvx/7vJ8Pyup+PvN0SnIN7wtf4tIJBGz/Wie+F8PG/lfRQ1XHt9F74tKSXJF1UkBfuaubvBVZYAAAAyBGaTQEAAHKE8AYAAJAjhDcAAIAcIbwBAADkCOENAAAgRwhvAAAAOUJ4AwAAyBHCGwCsgpn9pZndHN7/DTP7aNp1AtBZWNsUAFbnVyX9WriQ9g9Len/K9QHQYVhhAQBWycz+StJGSTe5e9nM3iLplxUsCfeBdGsHoN3RbAoAq2BmPyhpq6R5dy9Lkrs/5+53pVszAJ2C8AYACZnZVknHJO2WNGdm/yrlKgHoQIQ3AEjAzF4naVTSve5+RtKvS/pIqpUC0JHo8wYA62RmmyX9pqSbJR119/+acpUAtDHCGwAAQI7QbAoAAJAjhDcAAIAcIbwBAADkCOENAAAgRwhvAAAAOUJ4AwAAyBHCGwAAQI4Q3gAAAHKE8AYAAJAj/x84am1cPs4OtwAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig = plot_data_for_classification(X2, Y2, xlabel=r'$x_1$', ylabel=r'$x_2$')" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "def safeSigmoid(x, eps=0):\n", + " \"\"\"Funkcja sigmoidalna zmodyfikowana w taki sposób, \n", + " żeby wartości zawsze były odległe od asymptot o co najmniej eps\n", + " \"\"\"\n", + " y = 1.0/(1.0 + np.exp(-x))\n", + " if eps > 0:\n", + " y[y < eps] = eps\n", + " y[y > 1 - eps] = 1 - eps\n", + " return y\n", + "\n", + "def h(theta, X, eps=0.0):\n", + " \"\"\"Funkcja hipotezy (regresja logistyczna)\"\"\"\n", + " return safeSigmoid(X*theta, eps)\n", + "\n", + "def J(h,theta,X,y, lamb=0):\n", + " \"\"\"Funkcja kosztu dla regresji logistycznej\"\"\"\n", + " m = len(y)\n", + " f = h(theta, X, eps=10**-7)\n", + " j = -np.sum(np.multiply(y, np.log(f)) + \n", + " np.multiply(1 - y, np.log(1 - f)), axis=0)/m\n", + " if lamb > 0:\n", + " j += lamb/(2*m) * np.sum(np.power(theta[1:],2))\n", + " return j\n", + "\n", + "def dJ(h,theta,X,y,lamb=0):\n", + " \"\"\"Gradient funkcji kosztu\"\"\"\n", + " g = 1.0/y.shape[0]*(X.T*(h(theta,X)-y))\n", + " if lamb > 0:\n", + " g[1:] += lamb/float(y.shape[0]) * theta[1:] \n", + " return g\n", + "\n", + "def classifyBi(theta, X):\n", + " \"\"\"Funkcja predykcji - klasyfikacja dwuklasowa\"\"\"\n", + " prob = h(theta, X)\n", + " return prob" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "def GD(h, fJ, fdJ, theta, X, y, alpha=0.01, eps=10**-3, maxSteps=10000):\n", + " \"\"\"Metoda gradientu prostego dla regresji logistycznej\"\"\"\n", + " errorCurr = fJ(h, theta, X, y)\n", + " errors = [[errorCurr, theta]]\n", + " while True:\n", + " # oblicz nowe theta\n", + " theta = theta - alpha * fdJ(h, theta, X, y)\n", + " # raportuj poziom błędu\n", + " errorCurr, errorPrev = fJ(h, theta, X, y), errorCurr\n", + " # kryteria stopu\n", + " if abs(errorPrev - errorCurr) <= eps:\n", + " break\n", + " if len(errors) > maxSteps:\n", + " break\n", + " errors.append([errorCurr, theta]) \n", + " return theta, errors" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "theta = [[ 1.37136167]\n", + " [ 0.90128948]\n", + " [ 0.54708112]\n", + " [-5.9929264 ]\n", + " [ 2.64435168]\n", + " [-4.27978238]]\n" + ] + } + ], + "source": [ + "# Uruchomienie metody gradientu prostego dla regresji logistycznej\n", + "theta_start = np.matrix(np.zeros(X2.shape[1])).reshape(X2.shape[1],1)\n", + "theta, errors = GD(h, J, dJ, theta_start, X2, Y2, \n", + " alpha=0.1, eps=10**-7, maxSteps=10000)\n", + "print('theta = {}'.format(theta))" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "def plot_decision_boundary(fig, theta, X):\n", + " \"\"\"Wykres granicy klas\"\"\"\n", + " ax = fig.axes[0]\n", + " xx, yy = np.meshgrid(np.arange(-1.0, 1.0, 0.02),\n", + " np.arange(-1.0, 1.0, 0.02))\n", + " l = len(xx.ravel())\n", + " C = powerme(xx.reshape(l, 1), yy.reshape(l, 1), n)\n", + " z = classifyBi(theta, C).reshape(int(np.sqrt(l)), int(np.sqrt(l)))\n", + "\n", + " plt.contour(xx, yy, z, levels=[0.5], lw=3);" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [], + "source": [ + "Y_expected = Y2.astype(int)\n", + "Y_predicted = (classifyBi(theta, X2) > 0.5).astype(int)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "# Przygotowanie interaktywnego wykresu\n", + "\n", + "dropdown_highlight = widgets.Dropdown(options=['all', 'tp', 'fp', 'tn', 'fn'], value='all', description='highlight')\n", + "\n", + "def interactive_classification(highlight):\n", + " fig = plot_data_for_classification(X2, Y2, xlabel=r'$x_1$', ylabel=r'$x_2$',\n", + " Y_predicted=Y_predicted, highlight=highlight)\n", + " plot_decision_boundary(fig, theta, X2)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "b208b75eb3484bef8d52e8aec9b89448", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "interactive(children=(Dropdown(description='highlight', options=('all', 'tp', 'fp', 'tn', 'fn'), value='all'),…" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "widgets.interact(interactive_classification, highlight=dropdown_highlight)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "Zadanie klasyfikacyjne z powyższego przykładu polega na przypisaniu punktów do jednej z dwóch kategorii:\n", + " 0. czerwone krzyżyki\n", + " 1. zielone kółka\n", + "\n", + "W tym celu zastosowano regresję logistyczną." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "W rezultacie otrzymano model, który dzieli płaszczyznę na dwa obszary:\n", + " 0. na zewnątrz granatowej krzywej\n", + " 1. wewnątrz granatowej krzywej\n", + " \n", + "Model przewiduje klasę 0 („czerwoną”) dla punktów znajdujący się w obszarze na zewnątrz krzywej, natomiast klasę 1 („zieloną”) dla punktów znajdujących sie w obszarze wewnąrz krzywej." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "Wszysktie obserwacje możemy podzielić zatem na cztery grupy:\n", + " * **true positives (TP)** – prawidłowo sklasyfikowane pozytywne przykłady (zielone kółka w wewnętrznym obszarze)\n", + " * **true negatives (TN)** – prawidłowo sklasyfikowane negatywne przykłady (czerwone krzyżyki w zewnętrznym obszarze)\n", + " * **false positives (FP)** – negatywne przykłady sklasyfikowane jako pozytywne (czerwone krzyżyki w wewnętrznym obszarze)\n", + " * **false negatives (FN)** – pozytywne przykłady sklasyfikowane jako negatywne (zielone kółka w zewnętrznym obszarze)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "Innymi słowy:\n", + "\n", + "" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "slideshow": { + "slide_type": "skip" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "TP = 5\n", + "TN = 35\n", + "FP = 3\n", + "FN = 6\n" + ] + } + ], + "source": [ + "# Obliczmy TP, TN, FP i FN\n", + "\n", + "tp = 0\n", + "tn = 0\n", + "fp = 0\n", + "fn = 0\n", + "\n", + "for i in range(len(Y_expected)):\n", + " if Y_expected[i] == 1 and Y_predicted[i] == 1:\n", + " tp += 1\n", + " elif Y_expected[i] == 0 and Y_predicted[i] == 0:\n", + " tn += 1\n", + " elif Y_expected[i] == 0 and Y_predicted[i] == 1:\n", + " fp += 1\n", + " elif Y_expected[i] == 1 and Y_predicted[i] == 0:\n", + " fn += 1\n", + " \n", + "print('TP =', tp)\n", + "print('TN =', tn)\n", + "print('FP =', fp)\n", + "print('FN =', fn)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "skip" + } + }, + "source": [ + "Możemy teraz zdefiniować następujące metryki:" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Dokładność (*accuracy*)\n", + "$$ \\mbox{accuracy} = \\frac{\\mbox{przypadki poprawnie sklasyfikowane}}{\\mbox{wszystkie przypadki}} = \\frac{TP + TN}{TP + TN + FP + FN} $$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "Dokładność otrzymujemy przez podzielenie liczby przypadków poprawnie sklasyfikowanych przez liczbę wszystkich przypadków:" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Accuracy: 0.8163265306122449\n" + ] + } + ], + "source": [ + "accuracy = (tp + tn) / (tp + tn + fp + fn)\n", + "print('Accuracy:', accuracy)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "**Uwaga:** Nie zawsze dokładność będzie dobrą miarą, zwłaszcza gdy klasy są bardzo asymetryczne!\n", + "\n", + "*Przykład:* Wyobraźmy sobie test na koronawirusa, który **zawsze** zwraca wynik negatywny. Jaką przydatność będzie miał taki test w praktyce? Żadną. A jaka będzie jego *dokładność*? Policzmy:\n", + "$$ \\mbox{accuracy} \\, = \\, \\frac{\\mbox{szacowana liczba osób zdrowych na świecie}}{\\mbox{populacja Ziemi}} \\, \\approx \\, \\frac{7\\,700\\,000\\,000 - 600\\,000}{7\\,700\\,000\\,000} \\, \\approx \\, 0.99992 $$\n", + "(zaokrąglone dane z 27 marca 2020)\n", + "\n", + "Powyższy wynik jest tak wysoki, ponieważ zdecydowana większość osób na świecie nie jest zakażona, więc biorąc losowego Ziemianina możemy w ciemno strzelać, że nie ma koronawirusa.\n", + "\n", + "W tym przypadku duża różnica w liczności obu zbiorów (zakażeni/niezakażeni) powoduje, że *accuracy* nie jest dobrą metryką.\n", + "\n", + "Dlatego dysponujemy również innymi metrykami:" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Precyzja (*precision*)\n", + "$$ \\mbox{precision} = \\frac{TP}{TP + FP} $$" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Precision: 0.625\n" + ] + } + ], + "source": [ + "precision = tp / (tp + fp)\n", + "print('Precision:', precision)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "Precyzja określa, jaka część przykładów sklasyfikowanych jako pozytywne to faktycznie przykłady pozytywne." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### Pokrycie (czułość, *recall*)\n", + "$$ \\mbox{recall} = \\frac{TP}{TP + FN} $$" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Recall: 0.45454545454545453\n" + ] + } + ], + "source": [ + "recall = tp / (tp + fn)\n", + "print('Recall:', recall)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "Pokrycie mówi nam, jaka część przykładów pozytywnych została poprawnie sklasyfikowana." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "#### *$F$-measure* (*$F$-score*)\n", + "$$ F = \\frac{2 \\cdot \\mbox{precision} \\cdot \\mbox{recall}}{\\mbox{precision} + \\mbox{recall}} $$" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "F-score: 0.5263157894736842\n" + ] + } + ], + "source": [ + "fscore = (2 * precision * recall) / (precision + recall)\n", + "print('F-score:', fscore)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "$F$-_measure_ jest kompromisem między precyzją a pokryciem (a ściślej: jest średnią harmoniczną precyzji i pokrycia)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "$F$-_measure_ jest szczególnym przypadkiem ogólniejszej miary:" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "*$F_\\beta$-measure*:\n", + "$$ F_\\beta = \\frac{(1 + \\beta) \\cdot \\mbox{precision} \\cdot \\mbox{recall}}{\\beta^2 \\cdot \\mbox{precision} + \\mbox{recall}} $$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "Dla $\\beta = 1$ otrzymujemy:\n", + "$$ F_1 \\, = \\, \\frac{(1 + 1) \\cdot \\mbox{precision} \\cdot \\mbox{recall}}{1^2 \\cdot \\mbox{precision} + \\mbox{recall}} \\, = \\, \\frac{2 \\cdot \\mbox{precision} \\cdot \\mbox{recall}}{\\mbox{precision} + \\mbox{recall}} \\, = \\, F $$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## 3.3. Obserwacje odstające" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "**Obserwacje odstające** (*outliers*) – to wszelkie obserwacje posiadające nietypową wartość.\n", + "\n", + "Mogą być na przykład rezultatem błędnego pomiaru albo pomyłki przy wprowadzaniu danych do bazy, ale nie tylko.\n", + "\n", + "Obserwacje odstające mogą niekiedy znacząco wpłynąć na parametry modelu, dlatego ważne jest, żeby takie obserwacje odrzucić zanim przystąpi się do tworzenia modelu." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "W poniższym przykładzie można zobaczyć wpływ obserwacji odstających na wynik modelowania na przykładzie danych dotyczących cen mieszkań zebranych z ogłoszeń na portalu Gratka.pl: tutaj przykładem obserwacji odstającej może być ogłoszenie, w którym podano cenę w tys. zł zamiast ceny w zł." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "# Przydatne funkcje\n", + "\n", + "def h_linear(Theta, x):\n", + " \"\"\"Funkcja regresji liniowej\"\"\"\n", + " return x * Theta\n", + "\n", + "def linear_regression(theta):\n", + " \"\"\"Ta funkcja zwraca funkcję regresji liniowej dla danego wektora parametrów theta\"\"\"\n", + " return lambda x: h_linear(theta, x)\n", + "\n", + "def cost(theta, X, y):\n", + " \"\"\"Wersja macierzowa funkcji kosztu\"\"\"\n", + " m = len(y)\n", + " J = 1.0 / (2.0 * m) * ((X * theta - y).T * (X * theta - y))\n", + " return J.item()\n", + "\n", + "def gradient(theta, X, y):\n", + " \"\"\"Wersja macierzowa gradientu funkcji kosztu\"\"\"\n", + " return 1.0 / len(y) * (X.T * (X * theta - y)) \n", + "\n", + "def gradient_descent(fJ, fdJ, theta, X, y, alpha=0.1, eps=10**-5):\n", + " \"\"\"Algorytm gradientu prostego (wersja macierzowa)\"\"\"\n", + " current_cost = fJ(theta, X, y)\n", + " logs = [[current_cost, theta]]\n", + " while True:\n", + " theta = theta - alpha * fdJ(theta, X, y)\n", + " current_cost, prev_cost = fJ(theta, X, y), current_cost\n", + " if abs(prev_cost - current_cost) > 10**15:\n", + " print('Algorithm does not converge!')\n", + " break\n", + " if abs(prev_cost - current_cost) <= eps:\n", + " break\n", + " logs.append([current_cost, theta]) \n", + " return theta, logs\n", + "\n", + "def plot_data(X, y, xlabel, ylabel):\n", + " \"\"\"Wykres danych (wersja macierzowa)\"\"\"\n", + " fig = plt.figure(figsize=(16*.6, 9*.6))\n", + " ax = fig.add_subplot(111)\n", + " fig.subplots_adjust(left=0.1, right=0.9, bottom=0.1, top=0.9)\n", + " ax.scatter([X[:, 1]], [y], c='r', s=50, label='Dane')\n", + " \n", + " ax.set_xlabel(xlabel)\n", + " ax.set_ylabel(ylabel)\n", + " ax.margins(.05, .05)\n", + " plt.ylim(y.min() - 1, y.max() + 1)\n", + " plt.xlim(np.min(X[:, 1]) - 1, np.max(X[:, 1]) + 1)\n", + " return fig\n", + "\n", + "def plot_regression(fig, fun, theta, X):\n", + " \"\"\"Wykres krzywej regresji (wersja macierzowa)\"\"\"\n", + " ax = fig.axes[0]\n", + " x0 = np.min(X[:, 1]) - 1.0\n", + " x1 = np.max(X[:, 1]) + 1.0\n", + " L = [x0, x1]\n", + " LX = np.matrix([1, x0, 1, x1]).reshape(2, 2)\n", + " ax.plot(L, fun(theta, LX), linewidth='2',\n", + " label=(r'$y={theta0:.2}{op}{theta1:.2}x$'.format(\n", + " theta0=float(theta[0][0]),\n", + " theta1=(float(theta[1][0]) if theta[1][0] >= 0 else float(-theta[1][0])),\n", + " op='+' if theta[1][0] >= 0 else '-')))" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "# Wczytanie danych (mieszkania) przy pomocy biblioteki pandas\n", + "\n", + "alldata = pandas.read_csv('data_flats_with_outliers.tsv', sep='\\t',\n", + " names=['price', 'isNew', 'rooms', 'floor', 'location', 'sqrMetres'])\n", + "data = np.matrix(alldata[['price', 'sqrMetres']])\n", + "\n", + "m, n_plus_1 = data.shape\n", + "n = n_plus_1 - 1\n", + "Xn = data[:, 0:n]\n", + "\n", + "Xo = np.matrix(np.concatenate((np.ones((m, 1)), Xn), axis=1)).reshape(m, n + 1)\n", + "yo = np.matrix(data[:, -1]).reshape(m, 1)\n", + "\n", + "Xo /= np.amax(Xo, axis=0)\n", + "yo /= np.amax(yo, axis=0)" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmwAAAFoCAYAAADq7KeuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAcJUlEQVR4nO3dfbBkZ10n8O9vJkOik9EISQjkBbQyC6IrAa9JkNRuYMWFKZboLu7E3ZKIWQIKFi/KGrVKFP9YastXJJCKvGZXcXwDUsUARgqNKTaGSUyAvOBMAco4kYSXDTcDiZnMs390z2a46Zn0vbdv93N7Pp+qW919znNO/+65p3u+85xznlOttQAA0K8Nsy4AAICjE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMzC2xVdWZVfayq7qiq26rq1SPaVFW9uar2VNUnq+qZs6gVAGCWjpvhex9I8nOttZurakuSm6rq2tba7Ye1eUGSrcOf85K8bfgIAHDMmFkPW2vtrtbazcPni0nuSHL6kmYXJbm6DdyQ5KSqesKUSwUAmKkuzmGrqicneUaSv10y6/QkXzjs9d48MtQBAMy1WR4STZJU1YlJ/izJa1prX1s6e8QiI++lVVWXJbksSTZv3vz9T33qUydaJwDAat10001faq2dstzlZhrYqmpTBmHtD1prfz6iyd4kZx72+owk+0atq7V2VZKrkmRhYaHt2rVrwtUCAKxOVf3DSpab5VWileQdSe5orf3WEZpdk+Qlw6tFz09yb2vtrqkVCQDQgVn2sD07yU8k+VRV3TKc9ktJzkqS1tqVSXYm2ZZkT5KvJ3npDOoEAJipmQW21tr1GX2O2uFtWpJXTqciAIA+dXGVKAAARyawAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDo3EwDW1W9s6rurqpPH2H+hVV1b1XdMvz5lWnXCAAwa8fN+P3fneQtSa4+Spu/aa29cDrlAAD0Z6Y9bK2165J8ZZY1AAD0bj2cw/asqrq1qj5UVd8z62IAAKZt1odEH83NSZ7UWruvqrYleX+SraMaVtVlSS5LkrPOOmt6FQIArLGue9haa19rrd03fL4zyaaqOvkIba9qrS201hZOOeWUqdYJALCWug5sVXVaVdXw+bkZ1Pvl2VYFADBdMz0kWlXvTXJhkpOram+SNyTZlCSttSuTvDjJT1fVgSTfSHJxa63NqFwAgJmYaWBrrf34o8x/SwbDfgAAHLO6PiQKAIDABgDQPYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgczMNbFX1zqq6u6o+fYT5VVVvrqo9VfXJqnrmtGuEri0uJm9/e/ILvzB4XFycdUUArIHjZvz+707yliRXH2H+C5JsHf6cl+Rtw0fg+uuTbduSgweT/fuTzZuT170u2bkzueCCWVcHwATNtIettXZdkq8cpclFSa5uAzckOamqnjCd6qBji4uDsLa4OAhryeDx0PT77pttfQBMVO/nsJ2e5AuHvd47nAbHth07Bj1roxw8OJgPwNzoPbDViGltZMOqy6pqV1Xtuueee9a4LJix3bsf7llbav/+ZM+e6dYDwJrqPbDtTXLmYa/PSLJvVMPW2lWttYXW2sIpp5wyleJgZrZuHZyzNsrmzcnZZ0+3HgDWVO+B7ZokLxleLXp+kntba3fNuiiYue3bkw1H+Phu2DCYD8DcmOlVolX13iQXJjm5qvYmeUOSTUnSWrsyyc4k25LsSfL1JC+dTaXQmS1bBleDLr1KdMOGwfQTT5x1hQBM0EwDW2vtxx9lfkvyyimVA+vLBRck+/YNLjDYs2dwGHT7dmENYA7Nehw2YDVOPDG59NJZVwHAGuv9HDYAgGOewAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA546bdQHAKi0uJjt2JLt3J1u3Jtu3J1u2zLoqACZIYIP17Prrk23bkoMHk/37k82bk9e9Ltm5M7nggllXB8CEOCQK69Xi4iCsLS4OwloyeDw0/b77ZlsfABMjsMF6tWPHoGdtlIMHB/MBmAsCG6xXu3c/3LO21P79yZ49060HgDUjsMF6tXXr4Jy1UTZvTs4+e7r1ALBmBDZYr7ZvTzYc4SO8YcNgPgBzQWCD9WrLlsHVoFu2PNzTtnnzw9NPPHG29QEwMYb1gPXsgguSffsGFxjs2TM4DLp9u7AGMGcENljvTjwxufTSh18vLiZvf7uBdAHmiMAG88RAugBzyTlsMC8MpAswtwQ2mBcG0gWYWwIbzAsD6QLMLYEN5oWBdAHmlsAG88JAugBza+yrRKvqO5JsTXLCoWmttevWoihgBQ4NmLv0KtENGwykC7DOjRXYquq/JXl1kjOS3JLk/CT/J8lz1640YNkMpAswl8btYXt1kh9IckNr7TlV9dQkv7Z2ZQErtnQgXQDWvXHPYbu/tXZ/klTV8a21O5M8Ze3KAgDgkHF72PZW1UlJ3p/k2qr6apJ9a1cWsGKLi4NDom5NBTA3qrW2vAWq/m2Sb0/y4dbav6xJVau0sLDQdu3aNesyYPpG3Zrq0EUHbk0FMHNVdVNrbWG5y409rEdVbayqJyb5XAYXHpy23Dcbsc7nV9VnqmpPVV0+Yv6FVXVvVd0y/PmV1b4nzC23pgKYW+NeJfqzSd6Q5ItJDt37piX5vpW+cVVtTHJFkucl2ZvkE1V1TWvt9iVN/6a19sKVvg/MtcMPf/7zPycPPji63YMPDtq5GAFgXVrOVaJPaa19eYLvfW6SPa21zyZJVf1RkouSLA1swChLD38ed1xy4MDotvffn9zuowWwXo17SPQLSe6d8HufPlzvIXuH05Z6VlXdWlUfqqrvOdLKquqyqtpVVbvuueeeCZcKnRl1+PNIYe2QL0/y/1sATNO4PWyfTfJXVfXBJA8cmtha+61VvHeNmLb0CoibkzyptXZfVW3L4CrVraNW1lq7KslVyeCig1XUBf3bsWPQs7Ycj3vc2tQCwJobt4ftH5Ncm+QxSbYc9rMae5OcedjrM7JkqJDW2tdaa/cNn+9MsqmqTl7l+8L6t3v3wz1r4zjhhORpT1u7egBYU2P1sLXWfi1Jqmpza20Z/0oc1SeSbK2q70zyT0kuTvJfDm9QVacl+WJrrVXVuRkETMd1YOvWZOPG5KGHxmu/aZObvwOsY2P1sFXVs6rq9iR3DF8/vareupo3bq0dSPKqJB8ZrvePW2u3VdUrquoVw2YvTvLpqro1yZuTXNyWO3AczKNt28YLa5s3P3xTePcTBVi3xj2H7XeS/Psk1yRJa+3Wqvo3q33z4WHOnUumXXnY87ckectq3wfmzu/+7qO3qUouuyx54xuFNYB1buyBc1trX1gyacxjMcBELS4mv/M7j96uteS3fzv5+MfXviYA1tTYw3pU1Q8maVX1mKr6+QwPjwJTtmPH+OeuJcmLXuQuBwDr3LiB7RVJXpnBOGl7k5yT5GfWqijgKP7u75YX2A4cGIQ8ANatcQPbbyZ5VWvt8a21U5P8bJLfWLuygCP62MeW1/6hh5Ibb1ybWgCYinED2/e11r566MXw+TPWpiTgiBYXkzvvXP5yV12VvHVVF3YDMEPjBrYNVfUdh15U1WMz/hWmwKS85z2DiwlW4pWvHNwgHoB1ZzmHRD9eVb9eVW9M8vEk/3PtygJG+sAHVrf85ZdPpg4ApmrcOx1cXVW7kjw3g3uA/sfW2u1rWhnwSHv3rm75231sAdajsQ9rDgOab3uYpeXe8H2pTZsmUwcAUzX2wLlAB+6/f3XLP/OZk6kDgKkS2GC9WFxc3SHRxzwmOeecydUDwNQIbLAeXH99cvrpqzskevzxyfbtk6sJgKkR2KB3i4vJtm2Dx5U6/vhk5043gQdYp4ylBr3bsWN1PWsnnJB87nPJaadNriYApkoPG/Ru9+5k//6VLbt5c3LttcIawDonsEHvtm4dBK+VuOii5IILJlsPAFMnsEHvtm9PNqzwo/rZz062FgBmQmCD3m3ZMrhgYCVOOmmytQAwEwIbrAdPf/rKlvvrv07uu2+ytQAwdQIbrAevfe3KlnvwwcFVpgCsawIbrAfveMfKljtwILnttkdO37cvueSS5LzzBo/79q2uPgDWlMAG8+5tbxvcKeGQt751cNeEq69Obrxx8Hj66YPpAHSpWmuzrmHiFhYW2q5du2ZdBkxO1eqW37Jl0Iv2ta8NwtmR3HWXMdsA1lBV3dRaW1jucnrY4Fhw8ODgXLZf/MWjt7v88unUA8CyuDUV9G4S55ft35/s2ZPceefR233mM6t/L1iJxcXBfyp27x4MFr19+6BnGEgisEH/Xv3q1a9j48bk7LMH4e/GG4/c7ilPWf17wXJdf32ybdugJ3j//sGdPV73usH4g+7UAUmcwwb9e9zjkq98ZfXrWVx0Dhv9WVwc7JOLi4+cd+jcyxNPnH5dsEacwwbz6oEHVr+O7/3ewT96T3xicsUVo9tccYWwxvTt2DHoWRvl0LmXgMAG3Ttaj9i4Lrzw4ec/8zODnrRLLknOP3/weNddg+kwbbt3Dw6DjnLo3EvAOWzQvZe9LHn961e+fFVyzjnfPO2005J3v3tVZcFEbN06OGdtVGjbvHlw7iWghw269/KXr275jRsHV9xBj7ZvTzYc4Z+iDRvsuzAksEHvRp2MvRx/+qdO2qZfW7YMrgbdsmXQo5YMHg9Nt+9CEodEoX+PNtjt0TzpSclFF02uFlgLF1wwuBp0x47BOWtnnz3oWRPW4P8T2KB3jzbY7dHcd9/k6oC1dOKJaT/1U980qR185LBTowaiGjU81eh2I6aNaDnuaFdL2427rkn/DqMarrSWSW/LMSet+G8z9rZc4d903DqWU8tKzWVg+/yX9ucn3/Xw4KCT3rCr+oOOsey49U7yQzpot7IP6uja1v4LdJJ/12l8SMf/Gy5xwWvSnv7NwauNuLVoyyMntqrkTR9NNjw8b9xtOcokv8zH35YjKxnzPUetb4X7+ST/pquoY9BuZf+Q9fKdASzfXAa2xQcO5K8+c8+sy4DJ2PRtyWO/beXL/9/7J1cLTFmN+M/JiEmpEQ1Htxu1vvHeZJz1jVrXpH+H8WtbWS2r2ZajWo7/+49q9+jrG/d3H2Xke66wjnFr+YexKnukuQxsT37ct+bKn/yBb544ow/9uH/QpRMn/qFfxQdmnA/grLblKCv+0E/8QzqyuuWv7zd+I/n933/kl+qIbosa0b9RL3958vr/Pnixa1dy6U+lHnoo+cY3km/5lsGVeO96V7LwyIG3R3+ZTe7vutLPx6h1LWd9K/0dxv6HvaPvm0mua+K/w7gfapgjtcJRmuYysG05YVOe89RTZ10GTMbWM5KvrvAG8Mcfn9z6t8n7/nBwr8Yfe+E3X3V67/DxxS90CyCAjhnWA3q3mnGoHngg+fCHk9e8Jvmu70oefHB0O7cAAujaXPawwVzZsmX16zjSrX8On+8WQADd0sMGvVvtwLnjcAsggK4JbNC7aRyqdAsggK45JAq92717cus64YTB48aNg8OgmzcPwppbAAF0baaBraqen+R3k2xM8vbW2puWzK/h/G1Jvp7kJ1trN0+9UJilrVsnt65Nm5K///vkgx90CyCAdWRmga2qNia5IsnzkuxN8omquqa1dvthzV6QZOvw57wkbxs+wrFj+/bkZS9b/nIbNgzGWVvak3baacmll06+TgDWzCx72M5Nsqe19tkkqao/SnJRksMD20VJrm6D+5/cUFUnVdUTWmt3Tb9cmJGVXiV68cXJc5+rJw1gDswysJ2e5AuHvd6bR/aejWpzepJHBLaquizJZUly1llnTbRQWJd+8zcHvWkArHuzvEp01D1Jlt5XZ5w2g4mtXdVaW2itLZxyyimrLg66snHj8tqfd56wBjBHZhnY9iY587DXZyRZev+dcdrA/PuTP1le+/e/f23qAGAmZhnYPpFka1V9Z1U9JsnFSa5Z0uaaJC+pgfOT3Ov8NY5JP/qj418tesUVetcA5szMzmFrrR2oqlcl+UgGw3q8s7V2W1W9Yjj/yiQ7MxjSY08Gw3q8dFb1wswdGo7jhS/85unHHZc89anJ939/8qY3CWsAc6gGF2DOl4WFhbZr165ZlwEA8E2q6qbW2sJyl3NrKgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6d9ws3rSqHptkR5InJ/l8kv/cWvvqiHafT7KY5KEkB1prC9OrEgCgD7PqYbs8yUdba1uTfHT4+kie01o7R1gDAI5VswpsFyV5z/D5e5L8yIzqAADo3qwC2+Nba3clyfDx1CO0a0n+oqpuqqrLjrbCqrqsqnZV1a577rlnwuUCAMzOmp3DVlV/meS0EbN+eRmreXZrbV9VnZrk2qq6s7V23aiGrbWrklyVJAsLC23ZBQMAdGrNAltr7YeONK+qvlhVT2it3VVVT0hy9xHWsW/4eHdVvS/JuUlGBjYAgHk1q0Oi1yS5ZPj8kiQfWNqgqjZX1ZZDz5P8cJJPT61CAIBOzCqwvSnJ86pqd5LnDV+nqp5YVTuHbR6f5PqqujXJjUk+2Fr78EyqBQCYoZmMw9Za+3KSfzdi+r4k24bPP5vk6VMuDQCgO+50AADQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6V621WdcwcVV1T5J/mHUdx5CTk3xp1kUcQ2zv6bK9p8v2ni7be/qe0lrbstyFjluLSmattXbKrGs4llTVrtbawqzrOFbY3tNle0+X7T1dtvf0VdWulSznkCgAQOcENgCAzglsTMJVsy7gGGN7T5ftPV2293TZ3tO3om0+lxcdAADMEz1sAACdE9hYtqp6bFVdW1W7h4/fcYR2n6+qT1XVLSu9KuZYVlXPr6rPVNWeqrp8xPyqqjcP53+yqp45izrnxRjb+8Kqune4P99SVb8yizrnQVW9s6rurqpPH2G+fXuCxtje9u0Jqqozq+pjVXVHVd1WVa8e0WbZ+7jAxkpcnuSjrbWtST46fH0kz2mtneOy8eWpqo1JrkjygiRPS/LjVfW0Jc1ekGTr8OeyJG+bapFzZMztnSR/M9yfz2mtvXGqRc6Xdyd5/lHm27cn6905+vZO7NuTdCDJz7XWvjvJ+UleOYnvb4GNlbgoyXuGz9+T5EdmWMu8OjfJntbaZ1tr/5LkjzLY7oe7KMnVbeCGJCdV1ROmXeicGGd7MyGtteuSfOUoTezbEzTG9maCWmt3tdZuHj5fTHJHktOXNFv2Pi6wsRKPb63dlQx2zCSnHqFdS/IXVXVTVV02termw+lJvnDY67155Ad+nDaMZ9xt+ayqurWqPlRV3zOd0o5J9u3ps2+vgap6cpJnJPnbJbOWvY/P5Z0OWL2q+sskp42Y9cvLWM2zW2v7qurUJNdW1Z3D/+nx6GrEtKWXdI/ThvGMsy1vTvKk1tp9VbUtyfszOJzB5Nm3p8u+vQaq6sQkf5bkNa21ry2dPWKRo+7jetgYqbX2Q6217x3x84EkXzzUdTt8vPsI69g3fLw7yfsyOOzEePYmOfOw12ck2beCNoznUbdla+1rrbX7hs93JtlUVSdPr8Rjin17iuzbk1dVmzIIa3/QWvvzEU2WvY8LbKzENUkuGT6/JMkHljaoqs1VteXQ8yQ/nGTkFUqM9IkkW6vqO6vqMUkuzmC7H+6aJC8ZXm10fpJ7Dx2qZtkedXtX1WlVVcPn52bw/fnlqVd6bLBvT5F9e7KG2/IdSe5orf3WEZotex93SJSVeFOSP66qS5P8Y5IfS5KqemKSt7fWtiV5fJL3Db8Djkvyh621D8+o3nWntXagql6V5CNJNiZ5Z2vttqp6xXD+lUl2JtmWZE+Sryd56azqXe/G3N4vTvLTVXUgyTeSXNyMPL4iVfXeJBcmObmq9iZ5Q5JNiX17LYyxve3bk/XsJD+R5FNVdctw2i8lOStZ+T7uTgcAAJ1zSBQAoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAYxQVecMR31f6fIfn2Q9wLFNYAMY7ZwMxkl6hKp61DEsW2s/OPGKgGOWcdiAuTW88fKHk1yf5PwktyZ5V5JfS3Jqkv+a5LYkv5fkX2cwyPOvJvlQBgNafkuSf0ryP5J8d5InJnlyki9lMBDm/0qyefh2r2qtfbyq3pjkRcNpj01yU2vtR9fslwSOCQIbMLeGgW1PkmdkEMw+kUFouzSDUPXSJLcnub219r+r6qQkNw7b/1iShdbaq4br+tUk/yHJBa21b1TVtyY52Fq7v6q2Jnlva23hsPfenOS6JK9trV03hV8XmGNuTQXMu8+11j6VJFV1W5KPttZaVX0qg96yM5K8qKp+ftj+hAxvITPCNa21bwyfb0rylqo6J8lDSf7VkrbvSvJuYQ2YBIENmHcPHPb84GGvD2bwHfhQkv/UWvvM4QtV1Xkj1rX/sOevTfLFJE/P4Hzg+w9b9peTfL219nurrh4gLjoA+EiSn62qSpKqesZw+mKSLUdZ7tuT3NVaO5jBjZ43DpfflsHh1lesWcXAMUdgA451v57B4c1PVtWnh6+T5GNJnlZVt1TV9hHLvTXJJVV1QwaHQw/1vr0+yWlJbhgu+9trWz5wLHDRAQBA5/SwAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA69/8AXoLYKINHTj4AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig = plot_data(Xo, yo, xlabel=u'metraż', ylabel=u'cena')\n", + "theta_start = np.matrix([0.0, 0.0]).reshape(2, 1)\n", + "theta, logs = gradient_descent(cost, gradient, theta_start, Xo, yo, alpha=0.01)\n", + "plot_regression(fig, h_linear, theta, Xo)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "Na powyższym przykładzie obserwacja odstająca jawi sie jako pojedynczy punkt po prawej stronie wykresu. Widzimy, że otrzymana krzywa regresji zamiast odwzorowywać ogólny trend, próbuje „dopasować się” do tej pojedynczej obserwacji.\n", + "\n", + "Dlatego taką obserwację należy usunąć ze zbioru danych (zobacz ponizej)." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [], + "source": [ + "# Odrzućmy obserwacje odstające\n", + "alldata_no_outliers = [\n", + " (index, item) for index, item in alldata.iterrows() \n", + " if item.price > 100 and item.sqrMetres > 10]\n", + "\n", + "alldata_no_outliers = alldata.loc[(alldata['price'] > 100) & (alldata['sqrMetres'] > 100)]" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "data = np.matrix(alldata_no_outliers[['price', 'sqrMetres']])\n", + "\n", + "m, n_plus_1 = data.shape\n", + "n = n_plus_1 - 1\n", + "Xn = data[:, 0:n]\n", + "\n", + "Xo = np.matrix(np.concatenate((np.ones((m, 1)), Xn), axis=1)).reshape(m, n + 1)\n", + "yo = np.matrix(data[:, -1]).reshape(m, 1)\n", + "\n", + "Xo /= np.amax(Xo, axis=0)\n", + "yo /= np.amax(yo, axis=0)" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": { + "scrolled": true, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmwAAAFoCAYAAADq7KeuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5BcZ3nn8e8zmtFtZmSbsoxsWeayaO2YXWxYxZjgWi4JBKvYOAvEgs3GCsuW17GcNbdUTLIFhCSLUxtMAsgYr+PY3k3AW4GAKwgIIUk5TkJAJjbgu+IkWEjYigHP6K6Rnv3j9Eg9Mz0zPZfu82rm+6nq6tPnnO5+Z1ot/fSe533fyEwkSZJUrp66GyBJkqSpGdgkSZIKZ2CTJEkqnIFNkiSpcAY2SZKkwhnYJEmSCldbYIuIdRHxFxHxUEQ8EBHXtjgnIuIjEbEjIr4ZES+po62SJEl16q3xvUeAd2XmNyJiELg3Ir6cmQ82nXMpsL5xeynw8ca9JEnSolFbD1tm7s7MbzS2h4GHgLXjTrsMuCMrXwVOjYgzu9xUSZKkWhVRwxYRzwVeDPzduENrgSeaHu9kYqiTJEla0Oq8JApARAwAnwbenplD4w+3eErLtbQi4krgSoD+/v5/d955581rOyVJkubq3nvv/ZfMXD3T59Ua2CKijyqs/UFmfqbFKTuBdU2PzwZ2tXqtzLwZuBlgw4YNuX379nlurSRJ0txExD/P5nl1jhIN4PeAhzLzhklOuwu4ojFa9GLgmczc3bVGSpIkFaDOHraXAz8HfCsi7mvs+xXgHIDMvAnYBmwEdgD7gbfW0E5JkqRa1RbYMvMeWteoNZ+TwJbutEiSJKlMRYwSlSRJ0uQMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVLhaA1tE3BoRT0XEtyc5/sqIeCYi7mvc3tvtNkqSJNWtt+b3vw34GHDHFOf8VWa+vjvNkSRJKk+tPWyZeTfw/TrbIEmSVLqToYbtZRFxf0R8ISJeONlJEXFlRGyPiO179uzpZvskSZI6qvTA9g3gOZl5AfBR4LOTnZiZN2fmhszcsHr16q41UJIkqdOKDmyZOZSZexvb24C+iDi95mZJkiR1VdGBLSLWREQ0ti+iau/T9bZKkiSpu2odJRoRnwReCZweETuB9wF9AJl5E/Am4BciYgQ4ALw5M7Om5kqSJNWi1sCWmW+Z5vjHqKb9kCRJWrSKviQqSZIkA5skSVLxDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYWrNbBFxK0R8VREfHuS4xERH4mIHRHxzYh4SbfbKKlmw8Nwyy3wy79c3Q8P190iSeq63prf/zbgY8Adkxy/FFjfuL0U+HjjXtJicM89sHEjHDsG+/ZBfz+8852wbRtcckndrZOkrqm1hy0z7wa+P8UplwF3ZOWrwKkRcWZ3WiepVsPDVVgbHq7CGlT3o/v37q23fZLURaXXsK0Fnmh6vLOxT9JCd+edVc9aK8eOVcclaZEoPbBFi33Z8sSIKyNie0Rs37NnT4ebJanjHnvsRM/aePv2wY4d3W2PJNWo9MC2E1jX9PhsYFerEzPz5szckJkbVq9e3ZXGSeqg9eurmrVW+vvhBS/obnskqUalB7a7gCsao0UvBp7JzN11N0pSF2zaBD2T/BXV01Mdl6RFotZRohHxSeCVwOkRsRN4H9AHkJk3AduAjcAOYD/w1npaKqnrBger0aDjR4n29FT7BwbqbqEkdU2tgS0z3zLN8QS2dKk5kkpzySWwa1c1wGDHjuoy6KZNhjVJi07d87BJ0tQGBuBtb6u7FZJUq9Jr2CRJkhY9A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYXrrbsBknRSGB6GO++Exx6D9eth0yYYHKy7VZIWCQObJE3nnntg40Y4dgz27YP+fnjnO2HbNrjkkrpbJ2kR8JKoJE1leLgKa8PDVViD6n50/9699bZP0qJgYJOkqdx5Z9Wz1sqxY9VxSeqwti+JRsRpwHpg+ei+zLy7E42SpGI89tiJnrXx9u2DHTu62x5Ji1JbgS0i/itwLXA2cB9wMfC3wKs71zRJKsD69VXNWqvQ1t8PL3hB99skadFp95LotcCPAv+cma8CXgzs6VirJKkUmzZBzyR/Vfb0VMclqcPaDWwHM/MgQEQsy8yHgXM71yxJKsTgYDUadHCw6lGD6n50/8BAve2TtCi0W8O2MyJOBT4LfDkifgDs6lyzJKkgl1wCu3ZVAwx27Kgug27aZFiT1DWRmTN7QsQrgFOAL2bm4Y60ao42bNiQ27dvr7sZkiRJY0TEvZm5YabPm8ko0SXAs4F/bOxaA3xnpm8oSZKkmWl3lOgvAu8DngRGJyRK4EUdapckqRQuyyXVrt0etmuBczPz6U42RpJUGJflkorQ7ijRJ4BnOtkQSVJhXJZLKka7PWyPA38ZEZ8HDo3uzMwbOtIqSVL92lmW621v626bpEWq3cD2ncZtaeMmSVroXJZLKkZbgS0zfw0gIvozc5JvryRpQXFZLqkYbdWwRcTLIuJB4KHG4wsi4saOtkySVC+X5ZKK0e6gg98BfhJ4GiAz7wf+facaJUkqgMtyScVoe+LczHwiIpp3HZ3/5kiSiuKyXFIR2g1sT0TEjwEZEUuB/07j8qgkaYEbGHA0qFSzdi+JXgVsAdYCO4ELgas71ShJkiSd0G4P24eAazLzBwARcVpj33/pVMMkSQVxeSqpVu0GtheNhjWAzPxBRLy4Q22SJJXE5amk2rV7SbSn0asGQEQ8ixkMWJhMRLwuIh6JiB0RcV2L46+MiGci4r7G7b1zfU9J0gy4PJVUhJlcEv2biPgjIIHLgd+cyxtHxBJgK/Aaqrq4r0fEXZn54LhT/yozXz+X95IkzZLLU0lFaHelgzsiYjvwaiCAN7QIVjN1EbAjMx8HiIhPAZcBc31dSYuZtVbzy+WppCLMZB62B5nfMLUWeKLp8U7gpS3Oe1lE3A/sAt6dmQ/MYxskLSSdrLXatQve8x54+GE47zz44AfhrLPmp90lW7du6uNnn92ddkiLXLs1bJ0QLfbluMffAJ6TmRcAHwU+O+mLRVwZEdsjYvuePXvmsZmSTgqdrLW68UZYuxbuuAO+9rXqfu3aar8kdUGdgW0n0Pxft7OpetGOy8yhzNzb2N4G9EXE6a1eLDNvzswNmblh9erVnWqzpFK1U2s1G7t2wZYtrY9t2QLf+97sXvdk8cQTUx/fubM77ZAWuTmP9JyDrwPrI+J5wHeBNwP/qfmEiFgDPJmZGREXUQXMp7veUkllaq5Xu//+ztRavec9Ux+/7jq47bbZvfbJYP366tJyq99tf3+1VJW0yB05eozhgyMMHThS3R88wtCBIwwdPHJ8/1DjfrZqC2yZORIR1wBfApYAt2bmAxFxVeP4TcCbgF+IiBHgAPDmzBx/2VTSYjS+Xm3p0snPnUuwePjhqY8/8sjsXvdksWlTVQfYSk9PdVw6yR08crQRskYYPngiXI0NXJMf33+488ur19nDNnqZc9u4fTc1bX8M+Fi32yVpHnVi1GZzvdqow4cnP38uweK886q6tcmce+7sXne2uj0KdnCwGrQxfjBHT0+130XgVbPMZP/h8YGr2m4VuIYagWt4dN/BEQ6PTFJO0aaegFUr+hhc3suq5X3VbUUvg03bq5ZXxy//rdm9RyzEDqsNGzbk9u3b626GpFajNkf/oZ/LqM1bboG3v33yS6DLlsGhQ/Pzfrt2VQMMJrN7N6xZM7vXnqmpfp8XXNDZILd3b/X6O3ZUvZWbNhnWNC+OHUuGD7W6nNgIXwdO7Dt+fFxv19Fjc8syfUuCU1b0NQJWL6tW9B0PWNV279hANu54/9IlRLQaSzlRRNybmRtm2sZae9gkLWCtesFGA9YrXlH9g//bvz27qTGmmhsM4NWvrgLMfASLs86CrVtbDzzYurV7YW2q3+dP/mQV3DI7t3TUwIAT5KqlmdRvDR2cGL72Hhphrn1HK/qWNPVotQpcfS2Pj24v6+1pO3DVxcAmqTOmG7X5yU9Wt61b4eqrZ/ba0xXCv/GNcPnlVRt+/ddP9DiNtmumvVBXXw1veEM1wOCRR6rLoNdf372wBlP/PvfvH/t49PfymtfAVVfBC1/oBMKaVAn1W4PLeqcNV+OPj+4bXN7H0t46J73oDgObpPbNpH5qul6wUVu2VGFouvDT/N7r1sFk/xvu6amOr1079tLhtddWPVA9PbPrhVqzpt7RoO3+PpsdPAi/8zsu1r6ANddvTReumnu5hpt6u7pZvzU+cK1a3sfA8l6W9JTdu1UCA5uk9sx0FYF166qRm1MNBhg13dQYrd47E1asGBvAenrgj/4I3vSm1pcOm43u27ixqlMrvR5rql7F6ZxsP+siUkr91om6rM7Wb2n2DGySpjdV/dSP/zh86EOwefOJ3rZ77qnmL2snrAE8OMWqd1O998BAdWly584T9Wqf+tTklw5bOVkWMJ9qeo12dftnXQTLeY3Wb00arrpUvzUxXE0dvlY1ha+ToX5LBjZJ7ZiqfurwYXjXu+Dd765qx172suof6ZksBfX3f1+FvFY9dVO9dyYsX14FgVEzvXR4sixg3jy9xsgIHDhQXRZesqS6HTo0/Wt082e98caxAzVGl/SaTc1iB43Wb40vjp+ueH70eF31W6PHF0v9lgxskppNVqM2XQga7Un7wz+sLkm227M2amSkGtl5ww1je+pg6vduFUBmeunwZJqt/5JL4P3vrwIyVIF1ZKS6LV0KfX1T/9zd+FmHh+ETn4Bf+qXWx9utWWxDKfVbg+NqsqYdmWj9lmbBedgkVVrViUVUvSH33w9/+Zft9eLMxbJlVfBoroubas61/n743d8de4lveLgacNB8CXUqg4MnT13XdHPCfehD8N3vVr1YrT6rTv+so3+G9u+Ho1P0PG3eDLfddrx+a/LLiaMTnzZPeDq2t6sT9Vtja7V6xwayccet39JMzXYeNgObpJmHnE5rDha7dsHznz+zADLZIIXxo0TnYxLfbtq8ubqsONXx226bfsLieVotYUz91tPPMHTZGxk61sPwsn6GlvcztKzpNvp4+QDDq05j6NTTrd/SouTEuZJmb6o6sTocPQqvfS388IdVqFiyZOzx5cury3+TLY10ySVVkBs/Mz+c3LP1t7uu6WQ//8DAmDB38OBhhk9bzdAH/hdDWz/B0LkvnNDbNaP6rcv+R/s/y8ERAAaW9c5oZKL1W1qsDGySZjfHVyft3w9/+7cnHo+MTDzn0UenroMa7VE7duzE9uBg+aNBp9K0rmkC+/uWj+3NetErGL7vuyfC1Zk/xtBpF1Xh684HGNp3iOEHHmboihsZWt7P4d6lJ177r/fBX0+xZmoLY+q3nn6KVd95nMFD+1l1aC+rDu5j1aF9DB7ad3x71cHq8Sl3/zmr1p1l/ZY0AwY2aaEbHobbb4c/+ZPq8etfP7Gwfy5zfNVhyRL4/OdPhK/xl/jOOacasXrkSHUpddkyeMc74AtfKOby56zqt160meErLzl+afFoz5KJL/yp+6Z+49NO1MD1HT1yIkwdOcCq561j8F89Z3b1W7fcAlv/5/R/hrZuhXOfM8PfliRr2KSF7J57qkuLBw6M3b9yJXzpSyfCS2k1bO143evgRS+qtm+88cQ6mitXTlyqadTKlfDkk/NyGXRkdP3EycLVmHm3RiaMZJyP+q3lRw6eCFxnncHgc9dNXTx/682c8nufOP6cZSOHGdO/dd11Y6dImYmp/gxFwFveUg2K6OZyXlKBrGGTNNbwMFx66cSwBlWgufRS2L27Ci+Dg2Oni+iE3t7WlzZn68//HL74xYn7Jwtro8duvx22bOHgkaPtzyzf4vh8zL810/qt48eHvs/gB97L0kcentm6puvXwMEfTD7idi5TfjTPEzfZYAdJs2YPm7QQPfpoFcgef3zycyJg1So4/3z42Z+Fa67pbJuWLatC2zxcdm1Zv7Wsf+zj5QMMLVvZtN3P8LKVDPWfytCyfg7H3IrVx8+/1TJc3ft3rPqD28fWbx3ax6rfeD8DV/+37tdvTdULNl9Tfuzde3IP7JA6zGk9mhjYtKi9853w4Q/X3YoTli6tbp/+9PE1Po8R7F26ohGuBhha3ghTje0J4WvZQHW8KYy1rN+agb6jI1WQWn0aq04bnFirtazRm3U8kI09Pu38W9PNmbZ7dz2XB6eb8kNSRxnYmhjYtGg9+mh1iazDRqJnQm/W8cC1YoChpSur+76VDK0crALYGWcx1LeCocNH2bt0BTnHHq4x9VuTjEYcPbbq4F4GD+3nlKbRi8frtzo1mWy7c6bVwV4wqTbWsEmCn//5tk47uKRvTOAa35s1XY/X/qUrZt/GZdXdQNP0D9VUEFWwOh6+mgLY2DBWha+lx+apHq5TC6K3O2daHQYGTu7pTaRFyMAmnWQykwNHjrYujl/6HIYuPnvizPKjtVyNADZm/q1Z6Dl2dEzIOr49JoBNnIvrlMb5A4cPsCQLmai3UwuiN82Z1lIXekIlLRwGNqnLjh1L9h5uTO/QYrqHEwtXjzveNFpx0vUTL/7PbbWhef6t6Xqzqn37j2+vOrSP/sMHWDDTnXZqQfQPfnDqS6LXXz//7ylpwTKwSTM0fv6t8eFqqGX4muf5t/p6Wk5kuurQPgZvu6VlbdcpTb1fy0cOLZzANVc9PSeWrZpPZ51VTRK7ZcvEY1u3Oh+ZpBkxsGnROTQyyeXESSY/HX98vuffmjC56RQzy7e1fuI3PwMfvn3ObVywRueDax4d2amC+6uvhje8oZqQ9pFHZjZnmiQ1MbDppDK+fmu6meVbBa7DI3OrnWqef2tw2fThavT4KY35uTq+fuINN1TTaPzWb3XuPZpFMOcuw27p76+mFjnzzO6Njlyzpr7RoJIWDKf1UFeNr98af7lwQi3XoYnha2Sy+q029S2JFgGrxeSnxwPZ2OP9S3vpKX3B6uFhOP10OHx44rH5XnFg8+bq9e68c/rXHe3Vuv76qtdpdC6wFStar8gwlRUrqjVFR5ek6u8/sdj7wYOtn9OpKTwkqU3Ow9bEwNY5JdVvDTYt6TPd5cRTmiY/Xd7XM/WEpwvFPfdU620ePAhHj1bhZvnyajmnz3xmfibXbQ5Au3fD858/eVi6/PJqXdPRXq3xc4E980zrpbHe8Y5qYfFWE71eeOHE+cQAfuM3qp8volr83clhJRXCwNbEwDa55vqttkYmHjjSsfqtyWu1Ws8sP7i8l8HlvSzrndsM94vKVBOk/sM/wBVXwIMPwg9/eOI5vb1VwDvttKrH6siRKihlnrj8OVkAGj+Lfl9fFRQ/97kqrE3ne99rXe81m4lenRxWUoEMbE0WamCbaf1Wq0B2qMb6rcHlvQws66V3ydxmuFcHtBNu2g1ABiVJmpSBrUmpga2E+q3enjgenk5cTpwYvk7q+i1Jkgrl0lRd0Fy/1epyYevw1Z36rZbF843jp6xo9HYtpvotSZIWkEUV2OZSvzV88Aj7rN+SJEk1WJCB7Tvf388Vt35t3uu3Ihg39cPk4WqV9VuSJGmeLMjA9syBI9z96J4J+9ut35owF9fyqoje+i1JklSHBRnY1p22kv/91h+1fkuSJC0ICzKwnbqyj1ede0bdzZAkSZoXFlRJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVrtbAFhGvi4hHImJHRFzX4nhExEcax78ZES+po52SJEl1qi2wRcQSYCtwKXA+8JaIOH/caZcC6xu3K4GPd7WRkiRJBaizh+0iYEdmPp6Zh4FPAZeNO+cy4I6sfBU4NSLO7HZDJUmS6lRnYFsLPNH0eGdj30zPASAiroyI7RGxfc+ePfPaUEmSpDrVGdiixb6cxTnVzsybM3NDZm5YvXr1nBsnSZJUijoD205gXdPjs4FdszhHkiRpQaszsH0dWB8Rz4uIpcCbgbvGnXMXcEVjtOjFwDOZubvbDZUkSapTb11vnJkjEXEN8CVgCXBrZj4QEVc1jt8EbAM2AjuA/cBb62qvJElSXWoLbACZuY0qlDXvu6lpO4Et3W6XJElSSVzpQJIkqXAGNkmSpMIZ2CRJkgpnYJMkSSqcgU2SJKlwBjZJkqTCGdgkSZIKZ2CTJEkqnIFNkiSpcAY2SZKkwhnYJEmSCmdgkyRJKpyBTZIkqXAGNkmSpMIZ2CRJkgpnYJMkSSqcgU2SJKlwBjZJkqTCGdgkSZIKZ2CTJEkqnIFNkiSpcAY2SZKkwhnYJEmSCmdgkyRJKpyBTZIkqXAGNkmSpMIZ2CRJkgpnYJMkSSqcgU2SJKlwBjZJkqTCGdgkSZIKZ2CTJEkqnIFNkiSpcAY2SZKkwhnYJEmSCmdgkyRJKpyBTZIkqXAGNkmSpMIZ2CRJkgpnYJMkSSqcgU2SJKlwBjZJkqTCGdgkSZIKZ2CTJEkqnIFNkiSpcAY2SZKkwhnYJEmSCtdbx5tGxLOAO4HnAv8EXJ6ZP2hx3j8Bw8BRYCQzN3SvlZIkSWWoq4ftOuArmbke+Erj8WRelZkXGtYkSdJiVVdguwy4vbF9O/DTNbVDkiSpeHUFtmdn5m6Axv0Zk5yXwJ9GxL0RcWXXWidJklSQjtWwRcSfAWtaHPrVGbzMyzNzV0ScAXw5Ih7OzLsneb8rgSsBzjnnnBm3V5IkqVQdC2yZ+ROTHYuIJyPizMzcHRFnAk9N8hq7GvdPRcQfAxcBLQNbZt4M3AywYcOGnGv7JUmSSlHXJdG7gM2N7c3A58afEBH9ETE4ug28Fvh211ooSZJUiLoC2/XAayLiMeA1jcdExFkRsa1xzrOBeyLifuBrwOcz84u1tFaSJKlGtczDlplPAz/eYv8uYGNj+3Hggi43TZIkqTiudCBJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYUzsEmSJBXOwCZJklQ4A5skSVLhDGySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuEMbJIkSYWLzKy7DfMuIvYA/1x3O2p2OvAvdTdikfMzKIOfQ/38DMrg51CGczNzcKZP6u1ES+qWmavrbkPdImJ7Zm6oux2LmZ9BGfwc6udnUAY/hzJExPbZPM9LopIkSYUzsEmSJBXOwLZw3Vx3A+RnUAg/h/r5GZTBz6EMs/ocFuSgA0mSpIXEHjZJkqTCGdgWiIh4VkR8OSIea9yfNsl5/xQR34qI+2Y7UkVjRcTrIuKRiNgREde1OB4R8ZHG8W9GxEvqaOdC18bn8MqIeKbxZ/++iHhvHe1cyCLi1oh4KiK+Pclxvwsd1sZn4PegwyJiXUT8RUQ8FBEPRMS1Lc6Z8XfBwLZwXAd8JTPXA19pPJ7MqzLzQod3z11ELAG2ApcC5wNviYjzx512KbC+cbsS+HhXG7kItPk5APxV48/+hZn5ga42cnG4DXjdFMf9LnTebUz9GYDfg04bAd6VmT8CXAxsmY9/FwxsC8dlwO2N7duBn66xLYvJRcCOzHw8Mw8Dn6L6LJpdBtyRla8Cp0bEmd1u6ALXzuegDsvMu4HvT3GK34UOa+MzUIdl5u7M/EZjexh4CFg77rQZfxcMbAvHszNzN1R/WIAzJjkvgT+NiHsj4squtW7hWgs80fR4JxO/mO2co7lp93f8soi4PyK+EBEv7E7T1MTvQhn8HnRJRDwXeDHwd+MOzfi7sCBXOlioIuLPgDUtDv3qDF7m5Zm5KyLOAL4cEQ83/kem2YkW+8YPvW7nHM1NO7/jbwDPycy9EbER+CzV5Qh1j9+F+vk96JKIGAA+Dbw9M4fGH27xlCm/C/awnUQy8ycy89+0uH0OeHK0O7Vx/9Qkr7Grcf8U8MdUl5I0ezuBdU2PzwZ2zeIczc20v+PMHMrMvY3tbUBfRJzevSYKvwu183vQHRHRRxXW/iAzP9PilBl/FwxsC8ddwObG9mbgc+NPiIj+iBgc3QZeC7QcSaS2fR1YHxHPi4ilwJupPotmdwFXNEYFXQw8M3r5WvNm2s8hItZERDS2L6L6++/prrd0cfO7UDO/B53X+P3+HvBQZt4wyWkz/i54SXThuB74fxHxNuA7wM8ARMRZwC2ZuRF4NgCr5ogAAAKGSURBVPDHje9qL/CHmfnFmtq7IGTmSERcA3wJWALcmpkPRMRVjeM3AduAjcAOYD/w1rrau1C1+Tm8CfiFiBgBDgBvTmcOn1cR8UnglcDpEbETeB/QB34XuqWNz8DvQee9HPg54FsRcV9j368A58DsvwuudCBJklQ4L4lKkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkktRMSFjZngZ/v8v5nP9kha3AxsktTahVTzJE0QEdPOYZmZPzbvLZK0aDkPm6QFq7Hw8heBe4CLgfuB3wd+DTgD+FngAeCjwL+lmlD6/cAXqCa0XAF8F/gg8CPAWcBzgX+hmgjz/wD9jbe7JjP/JiI+APxUY9+zgHsz8z927IeUtCgY2CQtWI3AtgN4MVUw+zpVaHsbVah6K/Ag8GBm/t+IOBX4WuP8nwE2ZOY1jdd6P/AfgEsy80BErASOZebBiFgPfDIzNzS9dz9wN/COzLy7Cz+upAXMpakkLXT/mJnfAoiIB4CvZGZGxLeoesvOBn4qIt7dOH85jSVkWrgrMw80tvuAj0XEhcBR4F+PO/f3gdsMa5Lmg4FN0kJ3qGn7WNPjY1R/Bx4F3piZjzQ/KSJe2uK19jVtvwN4EriAqh74YNNzfxXYn5kfnXPrJQkHHUjSl4BfjIgAiIgXN/YPA4NTPO8UYHdmHqNa6HlJ4/kbqS63XtWxFktadAxskha7X6e6vPnNiPh24zHAXwDnR8R9EbGpxfNuBDZHxFepLoeO9r79ErAG+GrjuR/ubPMlLQYOOpAkSSqcPWySJEmFM7BJkiQVzsAmSZJUOAObJElS4QxskiRJhTOwSZIkFc7AJkmSVDgDmyRJUuH+P85m5/l5ILrVAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig = plot_data(Xo, yo, xlabel=u'metraż', ylabel=u'cena')\n", + "theta_start = np.matrix([0.0, 0.0]).reshape(2, 1)\n", + "theta, logs = gradient_descent(cost, gradient, theta_start, Xo, yo, alpha=0.01)\n", + "plot_regression(fig, h_linear, theta, Xo)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "Na powyższym wykresie widać, że po odrzuceniu obserwacji odstających otrzymujemy dużo bardziej „wiarygodną” krzywą regresji." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## 3.4. Problem nadmiernego dopasowania" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Obciążenie a wariancja" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "# Dane do prostego przykładu\n", + "\n", + "data = np.matrix([\n", + " [0.0, 0.0],\n", + " [0.5, 1.8],\n", + " [1.0, 4.8],\n", + " [1.6, 7.2],\n", + " [2.6, 8.8],\n", + " [3.0, 9.0],\n", + " ])\n", + "\n", + "m, n_plus_1 = data.shape\n", + "n = n_plus_1 - 1\n", + "Xn1 = data[:, 0:n]\n", + "Xn1 /= np.amax(Xn1, axis=0)\n", + "Xn2 = np.power(Xn1, 2) \n", + "Xn2 /= np.amax(Xn2, axis=0)\n", + "Xn3 = np.power(Xn1, 3) \n", + "Xn3 /= np.amax(Xn3, axis=0)\n", + "Xn4 = np.power(Xn1, 4) \n", + "Xn4 /= np.amax(Xn4, axis=0)\n", + "Xn5 = np.power(Xn1, 5) \n", + "Xn5 /= np.amax(Xn5, axis=0)\n", + "\n", + "X1 = np.matrix(np.concatenate((np.ones((m, 1)), Xn1), axis=1)).reshape(m, n + 1)\n", + "X2 = np.matrix(np.concatenate((np.ones((m, 1)), Xn1, Xn2), axis=1)).reshape(m, 2 * n + 1)\n", + "X5 = np.matrix(np.concatenate((np.ones((m, 1)), Xn1, Xn2, Xn3, Xn4, Xn5), axis=1)).reshape(m, 5 * n + 1)\n", + "y = np.matrix(data[:, -1]).reshape(m, 1)" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAFoCAYAAAAfEiweAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAT+0lEQVR4nO3df4zteV3f8dd79kKQmWlcwgXWhRbqnYCWP8TeEpRJQ0Xa9bZxW6OZNVFXc5NNm1Kx17RS20jSNC1pGlPbWJvNQtEUYQhi3dhblaJEb7Rk765bYbmSmVCF27u6lzbB2Wkb3M6nf5y5vdfLvXtnl5nve+6cxyPZnJnzPXPOO9987/Dk+2tqjBEAAKa10D0AAMA8EmEAAA1EGABAAxEGANBAhAEANBBhAAANDizCquq9VfVUVX3qmudeUlUfraqN3cc7D+rzAQAOs4PcE/a+JPdc99w7k3xsjLGS5GO73wMAzJ06yJu1VtWrk/ziGOP1u99/JslbxhhPVtVdST4+xnjtgQ0AAHBITX1O2MvHGE8mye7jyyb+fACAQ+FY9wA3U1UPJHkgSRYXF//86173uuaJAAD+pEcfffQLY4zjz+dnp46wP6yqu645HPnUzV44xngwyYNJcvLkyXH+/PmpZgQA2JOq+v3n+7NTH458OMn9u1/fn+QXJv58AIBD4SBvUfGBJL+V5LVVdbGqTid5d5K3VdVGkrftfg8AMHcO7HDkGOO7b7LorQf1mQAAtwt3zAcAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGx7oHAOCI2tpK1teTjY1kZSVZW0uWl7ungkNDhAGw/86dS06dSnZ2ku3tZHExOXMmOXs2WV3tng4OBYcjAdhfW1uzANvamgVYMnu88vzTT/fOB4eECANgf62vz/aA3cjOzmw5IMIA2GcbG1f3gF1vezvZ3Jx2HjikRBgA+2tlZXYO2I0sLiYnTkw7DxxSIgxgHm1tJQ89lPzIj8wet7b2773X1pKFm/zPy8LCbDng6kiAuXPQVy4uL8/e6/rPWFiYPb+09JV/BhwBIgxgnlx75eIVV87fOnUquXRpfyJpdXX2Xuvrs3PATpyY7QETYPD/iTCAebKXKxdPn96fz1pa2r/3giPIOWEA88SVi3BoiDCAeeLKRTg0RBjAPHHlIhwaIgxgnly5cnF5+eoescXFq887cR4m48R8gHnjykU4FEQYwDxy5SK0czgSAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKBBS4RV1d+tqieq6lNV9YGqelHHHAAAXSaPsKq6O8kPJjk5xnh9kjuS3Df1HAAAnboORx5L8lVVdSzJi5NcapoDAKDF5BE2xvjvSf5Fks8leTLJF8cYv3L966rqgao6X1XnL1++PPWYAAAHquNw5J1J7k3ymiRfk2Sxqr7n+teNMR4cY5wcY5w8fvz41GMCAByojsOR35rkv40xLo8x/jjJR5J8c8McAABtOiLsc0neVFUvrqpK8tYkFxrmAABo03FO2CeSfDjJY0k+uTvDg1PPAQDQ6VjHh44x3pXkXR2fDQBwGLhjPgBAAxEGANBAhAEANBBhAAANRBgAQAMRBgDQQIQBADQQYQAADUQYAEADEQYA0ECEAQA0EGEAAA1EGABAg2PdAwC029pK1teTjY1kZSVZW0uWl7unAo44EQbMt3PnklOnkp2dZHs7WVxMzpxJzp5NVle7pwOOMIcjgfm1tTULsK2tWYAls8crzz/9dO98wJEmwoD5tb4+2wN2Izs7s+UAB0SEAfNrY+PqHrDrbW8nm5vTzgPMFREGzK+Vldk5YDeyuJicODHtPMBcEWHA/FpbSxZu8mtwYWG2HOCAiDBgfi0vz66CXF6+ukdscfHq80tLvfMBR5pbVADzbXU1uXRpdhL+5ubsEOTamgADDpwIA1haSk6f7p4CmDMORwIANBBhAAANRBgAQAMRBgDQQIQBADQQYQAADUQYAEADEQYA0ECEAQA0EGEAAA1EGABAAxEGANBAhAEANBBhAAANRBgAQAMRBgDQQIQBADQQYQAADUQYAEADEQYA0ECEAQA0EGEAAA1EGABAAxEGANCgJcKq6qur6sNV9btVdaGqvqljDgCALseaPvcnkvzSGOM7q+qFSV7cNAcAQIvJI6yq/lSSv5jk+5NkjPGlJF+aeg4AgE4dhyP/bJLLSf5dVf12VT1UVYsNcwAAtOmIsGNJvjHJT40x3pBkO8k7r39RVT1QVeer6vzly5ennhEA4EB1RNjFJBfHGJ/Y/f7DmUXZnzDGeHCMcXKMcfL48eOTDggAcNAmj7Axxh8k+XxVvXb3qbcm+fTUcwAAdOq6OvLvJHn/7pWRn03yA01zAAC0aImwMcbjSU52fDYAwGHgjvkAAA1EGABAAxEGANBAhAEANBBhAAANRBgAQAMRBgDQQIQBADQQYQAADUQYAEADEQYA0ECEAQA0EGEAAA1EGABAAxEGANBAhAEANBBhAAANRBgAQAMRBgDQQIQBADQQYQAADUQYAEADEQYA0ECEAQA0EGEAAA2OdQ8AzJmtrWR9PdnYSFZWkrW1ZHm5eyqAyYkwYDrnziWnTiU7O8n2drK4mJw5k5w9m6yudk8HMCmHI4FpbG3NAmxraxZgyezxyvNPP907H8DERBgwjfX12R6wG9nZmS0HmCMiDJjGxsbVPWDX295ONjennQegmQgDprGyMjsH7EYWF5MTJ6adB6CZCAOmsbaWLNzkV87Cwmw5wBwRYcA0lpdnV0EuL1/dI7a4ePX5paXe+QAm5hYVwHRWV5NLl2Yn4W9uzg5Brq0JMGAuiTBgWktLyenT3VMAtHM4EgCgwS0jrKreXlV3TjEMAMC82MuesFckeaSqPlRV91RVHfRQAABH3S0jbIzxj5KsJHlPku9PslFV/7SqvvaAZwMAOLL2dE7YGGMk+YPd/55JcmeSD1fVPz/A2QAAjqxbXh1ZVT+Y5P4kX0jyUJK/N8b446paSLKR5O8f7IgAAEfPXm5R8dIk3zHG+P1rnxxj7FTVXzuYsQAAjrZbRtgY48eeZdmF/R0HAGA+uE8YAEADEQYA0ECEAQA0EGEAAA1EGABAg7YIq6o7quq3q+oXu2YAAOjSuSfsHUnc4gIAmEstEVZVr0zyVzO7Az8AwNzp2hP2LzP7c0c7N3tBVT1QVeer6vzly5enmwwAYAKTR9junzp6aozx6LO9bozx4Bjj5Bjj5PHjxyeaDgBgGh17wt6c5Nur6veSfDDJt1TVv2+YAwCgzeQRNsb4B2OMV44xXp3kviS/Osb4nqnnAADo5D5hAAANjnV++Bjj40k+3jkDAEAHe8IAABqIMACABiIMAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoMHkEVZVr6qqX6uqC1X1RFW9Y+oZAAC6HWv4zGeS/PAY47GqWk7yaFV9dIzx6YZZAABaTL4nbIzx5Bjjsd2vt5JcSHL31HMAAHRqPSesql6d5A1JPtE5BwDA1NoirKqWkvxckh8aY/zRDZY/UFXnq+r85cuXpx8QAOAAtURYVb0gswB7/xjjIzd6zRjjwTHGyTHGyePHj087IADAAZv8xPyqqiTvSXJhjPHjU38+kGRrK1lfTzY2kpWVZG0tWV7ungpgrnRcHfnmJN+b5JNV9fjucz86xjjbMAvMn3PnklOnkp2dZHs7WVxMzpxJzp5NVle7pwOYG5NH2BjjXJKa+nOBzPaAnTo1e7xie3v2eOpUculSsrTUMxvAnHHHfJgn6+uzPWA3srMzWw7AJEQYzJONjat7vq63vZ1sbk47D8AcE2EwT1ZWZueA3cjiYnLixLTzAMwxEQbzZG0tWbjJP/uFhdlyACYhwmCeLC/ProJcXr66R2xx8erzTsoHmEzHLSqATqurs6sg19dn54CdODHbAybAACYlwmAeLS0lp093TwEw1xyOBABoIMIAABqIMACABiIMAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKCBCAMAaHCs40Or6p4kP5HkjiQPjTHe3TEHtNraStbXk42NZGUlWVtLlpe7pwJgIpNHWFXdkeQnk7wtycUkj1TVw2OMT089C7Q5dy45dSrZ2Um2t5PFxeTMmeTs2WR1tXs6ACbQcTjyjUk2xxifHWN8KckHk9zbMAf02NqaBdjW1izAktnjleeffrp3PgAm0RFhdyf5/DXfX9x9DubD+vpsD9iN7OzMlgNw5HVEWN3gufFlL6p6oKrOV9X5y5cvTzAWTGRj4+oesOttbyebm9POA0CLjgi7mORV13z/yiSXrn/RGOPBMcbJMcbJ48ePTzYcHLiVldk5YDeyuJicODHtPAC06IiwR5KsVNVrquqFSe5L8nDDHNBjbS1ZuMk/vYWF2XIAjrzJI2yM8UyStyf55SQXknxojPHE1HNAm+Xl2VWQy8tX94gtLl59fmmpdz4AJtFyn7AxxtkkZzs+Gw6F1dXk0qXZSfibm7NDkGtrAgxgjrREGJBZcJ0+3T0FAE382SIAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKCBCAMAaCDCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgwAoIEIAwBoIMIAABqIMACABiIMAKBBjTG6Z7ilqtpK8pnuOebIS5N8oXuIOWJ9T8v6np51Pi3re1qvHWMsP58fPLbfkxyQz4wxTnYPMS+q6rz1PR3re1rW9/Ss82lZ39OqqvPP92cdjgQAaCDCAAAa3C4R9mD3AHPG+p6W9T0t63t61vm0rO9pPe/1fVucmA8AcNTcLnvCAACOlEMZYVX1XVX1RFXtVNVNr/Coqnuq6jNVtVlV75xyxqOkql5SVR+tqo3dxztv8rrfq6pPVtXjX8nVIPPqVttrzfyr3eW/U1Xf2DHnUbGH9f2Wqvri7vb8eFX9WMecR0VVvbeqnqqqT91kue17H+1hfdu+91FVvaqqfq2qLuz2yTtu8JrnvI0fyghL8qkk35Hk12/2gqq6I8lPJvm2JF+f5Lur6uunGe/IeWeSj40xVpJ8bPf7m/lLY4xvcPnzc7PH7fXbkqzs/vdAkp+adMgj5Dn8fviN3e35G8YY/3jSIY+e9yW551mW27731/vy7Os7sX3vp2eS/PAY4+uSvCnJ396P3+GHMsLGGBfGGLe6Oesbk2yOMT47xvhSkg8muffgpzuS7k3y07tf/3SSv944y1G1l+313iQ/M2b+S5Kvrqq7ph70iPD7YWJjjF9P8j+f5SW27320h/XNPhpjPDnGeGz3660kF5Lcfd3LnvM2figjbI/uTvL5a76/mC9fIezNy8cYTyazDS3Jy27yupHkV6rq0ap6YLLpjoa9bK+26f2z13X5TVX1X6vqP1XVn5tmtLll+56e7fsAVNWrk7whySeuW/Sct/G2O+ZX1X9O8oobLPqHY4xf2Mtb3OA5l3rexLOt7+fwNm8eY1yqqpcl+WhV/e7u/xvj1vayvdqm989e1uVjSf7MGOPpqjqV5D9kdhiBg2H7npbt+wBU1VKSn0vyQ2OMP7p+8Q1+5Fm38bYIG2N861f4FheTvOqa71+Z5NJX+J5H1rOt76r6w6q6a4zx5O6u06du8h6Xdh+fqqqfz+yQjwjbm71sr7bp/XPLdXntL9Axxtmq+jdV9dIxhr+5dzBs3xOyfe+/qnpBZgH2/jHGR27wkue8jd/OhyMfSbJSVa+pqhcmuS/Jw80z3a4eTnL/7tf3J/myPZFVtVhVy1e+TvKXM7uAgr3Zy/b6cJLv273C5k1JvnjlMDHP2S3Xd1W9oqpq9+s3Zvb78H9MPun8sH1PyPa9v3bX5XuSXBhj/PhNXvact/FD+Qe8q+pvJPnXSY4n+Y9V9fgY469U1dckeWiMcWqM8UxVvT3JLye5I8l7xxhPNI59O3t3kg9V1ekkn0vyXUly7fpO8vIkP7/7b/pYkp8dY/xS07y3nZttr1X1N3eX/9skZ5OcSrKZ5H8l+YGueW93e1zf35nkb1XVM0n+d5L7hrtXP29V9YEkb0ny0qq6mORdSV6Q2L4Pwh7Wt+17f705yfcm+WRVPb773I8m+dPJ89/G3TEfAKDB7Xw4EgDgtiXCAAAaiDAAgAYiDACggQgDAGggwgAAGogwAIAGIgyYC1X1F6rqd6rqRbt/AeKJqnp991zA/HKzVmBuVNU/SfKiJF+V5OIY4581jwTMMREGzI3dvyP5SJL/k+Sbxxj/t3kkYI45HAnMk5ckWUqynNkeMYA29oQBc6OqHk7ywSSvSXLXGOPtzSMBc+xY9wAAU6iq70vyzBjjZ6vqjiS/WVXfMsb41e7ZgPlkTxgAQAPnhAEANBBhAAANRBgAQAMRBgDQQIQBADQQYQAADUQYAEADEQYA0OD/AWCU6vzjnoR/AAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig = plot_data(X1, y, xlabel='x', ylabel='y')" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "# Funkcja regresji wielomianowej\n", + "\n", + "def h_poly(Theta, x):\n", + " \"\"\"Funkcja wielomianowa\"\"\"\n", + " return sum(theta * np.power(x, i) for i, theta in enumerate(Theta.tolist()))\n", + "\n", + "def polynomial_regression(theta):\n", + " \"\"\"Funkcja regresji wielomianowej\"\"\"\n", + " return lambda x: h_poly(theta, x)" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [], + "source": [ + "def plot_fun(fig, fun, X):\n", + " \"\"\"Wykres funkcji `fun`\"\"\"\n", + " ax = fig.axes[0]\n", + " x0 = np.min(X[:, 1]) - 1.0\n", + " x1 = np.max(X[:, 1]) + 1.0\n", + " Arg = np.arange(x0, x1, 0.1)\n", + " Val = fun(Arg)\n", + " return ax.plot(Arg, Val, linewidth='2')" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": { + "scrolled": true, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[]" + ] + }, + "execution_count": 31, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAFoCAYAAAAfEiweAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXiU9b3+8fubHSZhD/sOkR0SS13Ro3VHLXXBhJ722Nbfse2pBY0b7latlbpC29MeThfb05YEXFHRui9oXdCEsJOwh7AkLGGyL/P9/TGxUJpAEmbmO8v7dV1eSeaZzHNfz/VkvHmWzxhrrQAAABBaca4DAAAAxCJKGAAAgAOUMAAAAAcoYQAAAA5QwgAAAByghAEAADgQtBJmjPm9MWavMWb1EY/1Msa8YYwpbvnaM1jrBwAACGfBPBL2tKSLj3psrqS3rLUZkt5q+RkAACDmmGAOazXGDJf0srV2YsvPGySdY63dZYwZIOlda+2YoAUAAAAIU6G+JqyftXaXJLV87Rvi9QMAAISFBNcB2mKMuV7S9ZLk8Xi+MnbsWMeJAAAdUVZZq31VDUpKiFNG31TFGeM6EhBwn3/+eYW1Nr0zvxvqErbHGDPgiNORe9t6orV2oaSFkjR16lS7YsWKUGUEAJygdzbs1Xf/8JmGxBk9+8MzNGVID9eRgKAwxmzr7O+G+nTkUknXtnx/raQXQ7x+AECQlXvrdeuSlZKkmy8cQwED2hDMERWLJP1d0hhjTKkx5jpJj0i6wBhTLOmClp8BAFHCWqtbn1mpiqoGnT6yt75/9kjXkYCwFbTTkdbaWW0sOi9Y6wQAuPX0R1v17oZy9eiaqCeypygujuvAgLYwMR8AEBDrdh3Sz15dL0l65MrJGtC9i+NEQHijhAEATlhdY7NmLypQQ5NPs04Zqosn9ncdCQh7lDAAwAn76SvrVLy3SiPTPbrnsnGu4wARgRIGADghb67do//7eJsS440W5GSpa1LYjqAEwgolDADQaXsP1em2Z4skSbddNFYTB3V3nAiIHJQwAECn+HxWNy9Zqf3VDToro4+umzbCdSQgolDCAACd8rvlW/RBcYV6eZL0+EzGUQAdRQkDAHTY6p2V+vnf/OMo5l01WX27pThOBEQeShgAoENqGpo0O69Ajc1W3z5tmC4Y3891JCAiUcIAAB3y4MvrtLm8Whl9U3XXpYyjADqLEgYAaLfXVu/Wok+3KykhTgtmZSklMd51JCBiUcIAAO2yq7JWc5/zj6OYe/FYjRvQzXEiILJRwgAAx9Xss8rNX6mDNY06Z0y6vnvmcNeRgIhHCQMAHNfC9zfr75v3qU9qkh69eoqMYRwFcKIoYQCAY1q546Aef32DJOnRq6coPS3ZcSIgOlDCAABtqq5v0py8AjX5rL5zxnCdO7av60hA1KCEAQDadP/SNdq6r0Zj+6dp7iVjXccBogolDADQqpeLyrTk81IlM44CCApKGADgX5QeqNEdz62SJN196Tid1C/NcSIg+iS4DgAACC9fjqPw1jXp/HF99a3ThnXuhbxeKT9fKi6WMjKk7GwpjTIHfIkSBgD4J//9Tok+3bpf6WnJmnfV5M6No1i+XJo+XfL5pOpqyeORcnOlZcukadMCHxqIQJyOBAD8w+fbDuipt4olSY/PnKLeqZ0YR+H1+guY1+svYJL/65ePV1UFMDEQuShhAABJkreuUTfmF6jZZ/WfZ43Q2Seld+6F8vP9R8Ba4/P5lwOghAEA/O59cY127K/V+AHddMtFYzr/QsXFh4+AHa26Wiop6fxrA1GEEgYA0AsFO/V8wU6lJPrHUSQnnMA4iowM/zVgrfF4pNGjO//aQBThwnwAiEVH3Lm4Y/hY3b1rgCTp3ssmaHTf1BN77exs/0X4rYmL8y8HQAkDgJhzxJ2LTTW1mvPtR1U1oK8u6p+gWacMOfHXT0vz3wV59N2RcXH+x1NPsOQBUYISBgCx5Mg7FyUtmPZNfTFgjPp7K/TI7+6Q+X8bA1OSpk2Tysr8R9tKSvynILOzKWDAEShhABBLjrhz8bNB4/XL07NlrE9PvPyEetYe8i+/7rrArCs1NXCvBUQhShgAxJKWOxcrkz268fJb5IuL1w8+XqIzthf5l3PnIhAylDAAiCUZGbIej+4670fa2b2vJu/aqNwP/uJfxp2LQEgxogIAYkl2tp4dd45eHne2ujbUav5LjynJ1+Rfxp2LQEhxJAwAYsjW+jjdd+EPpWbp/g+e1ogDZdy5CDhCCQOAGNHY7NOc/EJVN0uXjkvXzNFXSOdP4s5FwBFKGADEiKfe3KiVOw5qYPcUPTwzS6brKa4jATGNa8IAIAZ8vHmf/vvdTYoz0pPZmereNdF1JCDmUcIAIModrGnQTfmFslb60bmjderI3q4jARAlDACimrVWdz6/Srsq65Q5pIdmn5fhOhKAFpQwAIhii1fs0LJVu5WanKAFOVlKjOdtHwgX/DUCQJTaVF6l+5eulSQ9MGOChvbu6jgRgCNRwgAgCjU0+XRjXqFqG5s1I3Ogrsga5DoSgKNQwgAgCj3+xgat2lmpwT276MFvTJQxxnUkAEehhAFAlPmwpEL/895mxRlpfk6muqUwjgIIR5QwAIgi+6sblLu4UJI0+7wMfWVYL8eJALSFEgYAUcJaq9ufLdKeQ/WaOqynbjh3tOtIAI6BEgYAUeKvn27XG2v3KC05QU9mZyqBcRRAWOMvFACiQMlerx582T+O4qdXTtKQXoyjAMIdJQwAIlx9U7N+vKhQdY0+XXnyIH19ykDXkQC0g5MSZoy5yRizxhiz2hizyBiT4iIHAESDn7+2Qet2HdLQXl31wIyJruMAaKeQlzBjzCBJsyVNtdZOlBQvKSfUOQAgGry3sVy/W75F8XFG83MylZqc4DoSgHZydToyQVIXY0yCpK6SyhzlAICIVVFVr5sXr5Qk5V5wkrKG9nScCEBHhLyEWWt3SnpM0nZJuyRVWmtfP/p5xpjrjTErjDErysvLQx0TAMKatVa3PVOkiqp6nTqil37wb6NcRwLQQS5OR/aUNEPSCEkDJXmMMd86+nnW2oXW2qnW2qnp6emhjgkAYe1Pf9+mt9fvVbcU/ziK+Dg+lgiINC5OR54vaYu1ttxa2yjpOUlnOMgBABFpw26vfrpsnSTpkasma2CPLo4TAegMFyVsu6TTjDFdjf8TZc+TtM5BDgCIOHWNzZq9qEANTT5lTx2i6ZMGuI4EoJNcXBP2iaRnJH0haVVLhoWhzgEAkeiRV9drwx6vRvTx6N7Lx7uOA+AEOLmX2Vp7n6T7XKwbACLV2+v36OmPtiox3mhBTpY8jKMAIhoT8wEgAuz11unWJUWSpJsvHKNJg7s7TgTgRFHCACDM+XxWtywp0r7qBp0xqreuP2uk60gAAoASBgBh7g8fbdX7G8vVo2uinrgmU3GMowCiAiUMAMLYmrJKzXt1vSRp3lWT1b87H7ULRAtKGACEqdqGZs3JK1RDs0/fPHWoLprQ33UkAAFECQOAMPXQK2tVsrdKo9I9uudSxlEA0YYSBgBh6PU1u/WXT7YrKT5OC2ZlqUtSvOtIAAKMEgYAYWbPoTrd/qx/HMVtF4/RhIGMowCiESUMAMKIz2eVu7hQB2oadVZGH33vzBGuIwEIEkoYAISR//1gsz4s2afeniQ9fs0UxlEAUYwSBgBhYlVppR57fYMk6edXT1bfNMZRANGMEgYAYaCmoUlz8grU2Gx17enDdN64fq4jAQgyShgAhIEHXlqrzRXVGtMvTXdMH+c6DoAQSHAdAACc83ql/HypuFjKyJCys6W0tJCt/tVVu5T32Q4lJcRp/qxMpSQyjgKIBZQwALFt+XJp+nTJ55OqqyWPR8rNlZYtk6ZNC/rqyw7Wau5zqyRJd14yVmP7dwv6OgGEB05HAohdXq+/gHm9/gIm+b9++XhVVVBX3+yzuim/UJW1jTp3TLquPWN4UNcHILxQwgDErvx8/xGw1vh8/uVB9Jv3NumTLfvVJzVZj86cImMYRwHEEkoYgNhVXHz4CNjRqqulkpKgrbpwx0E9+cZGSdJjMyerT2py0NYFIDxRwgDErowM/zVgrfF4pNGjg7Laqnr/OIomn9X3zhyhc8b0Dcp6AIQ3ShiA2JWdLcW18TYYF+dfHgT3vbhG2/bVaNyAbrr9kjFBWQeA8EcJAxC70tL8d0GmpR0+IubxHH48NTXgq1y6skzPflGq5IQ4LcjJVHIC4yiAWMWICgCxbdo0qazMfxF+SYn/FGR2dlAKWOmBGt31vH8cxT2XjVdGv9DNIgMQfihhAJCaKl13XVBX0dTs0415hfLWNemC8f3076cODer6AIQ/TkcCQAj86p1NWrHtgPqmJWveVZMZRwGAEgYAwfb5tv1a8HaxJOmJazLVy5PkOBGAcEAJA4AgOlTXqDl5hWr2WX3/7JGaltHHdSQAYYISBgBBdO8Lq1V6oFYTB3XTzRcyjgLAYZQwAAiS5wtK9UJhmbokxmt+TpaSEnjLBXAY7wgAEATb99XonhfWSJLuu3y8RqUHfuQFgMhGCQOAAGtq9mlOfoGq6pt0ycT+yv7qENeRAIQhShgABNiCt4pVsP2gBnRP0c+unMQ4CgCtooQBQAB9umW/fvlOiYzxj6Po0ZVxFABaRwkDgACprG3UTfmF8lnpv84ZpdNH9XYdCUAYo4QBQABYa3Xn86u082CtpgzpoRvPP8l1JABhjhIGAAHwzOeleqVolzxJ8ZqfnanEeN5eARwb7xIAcIK2VlTrvqX+cRQ/mTFRw/t4HCcCEAkoYQBwAhqbfZqTV6CahmZdPmWgrjp5kOtIACIEJQwATsCTb2zUytJKDerRRQ99YyLjKAC0GyUMADrpo00V+vV7mxRnpKdyMtW9S6LrSAAiCCUMADrhYE2DcvNXylrphq9l6KvDe7mOBCDCUMIAoIOstZr77CrtPlSnk4f20OyvjXYdCUAEooQBQAflf7ZDr63ZrbTkBM3PyVIC4ygAdALvHADQAZvKq/STl9ZKkh78xkQN6dXVcSIAkYoSBgDtVN/UrNmLClTb2KwrsgbpG1mMowDQeZQwAGinx1/fqDVlhzSkVxc9MGOC6zgAIhwlDADaYXlxhRa+v1nxcUbzc7KUlsI4CgAnhhIGAMexv7pBuYsLJUk3npehk4f2dJwIQDRwUsKMMT2MMc8YY9YbY9YZY053kQMAjsdaq9ueKdJeb71OGd5L/3Uu4ygABEaCo/XOl/SatfZqY0ySJG4vAhCW/vzJdr25bo/SUhL0ZE6m4uP4WCIAgRHyEmaM6SbpbEnfkSRrbYOkhlDnAIDjKd7j1UMv+8dR/OzKSRrUo4vjRACiiYvTkSMllUv6gzGmwBjzW2OMx0EOAGhTXWOzfryoQPVNPs38ymBdNnmg60gAooyLEpYg6WRJv7bWZkmqljT36CcZY643xqwwxqwoLy8PdUYAMe7nr23Q+t1eDe/dVfd/nXEUAALPRQkrlVRqrf2k5edn5C9l/8Rau9BaO9VaOzU9PT2kAQHEtnc37NXvP9yihJZxFJ5kV5fPAohmIS9h1trdknYYY8a0PHSepLWhzgEArSn31uuWJSslSbkXnqQpQ3o4TgQgWrn6592PJf2l5c7IzZK+6ygHAPyDfxzFSlVUNej0kb31/bNHuY4EIIo5KWHW2kJJU12sGwDa8sePtuqdDeXq3iVRT2RPYRwFgKBiYj4ASFq/+5AefnW9JGneVZM0oDvjKAAEFyUMQMyra2zW7EUFamjyadYpQ3TxxAGuIwGIAZQwADHv4WXrtHFPlUame3TPZeNdxwEQIyhhAGLam2v36E9/36bEeKMFOVnqmsQ4CgChQQkDELP2HqrTbc8WSZJuu2isJg7q7jgRgFhCCQMQk3w+q5uXrNT+6gadldFH100b4ToSgBhDCQMQk37/4RZ9UFyhXp4kPT5ziuIYRwEgxChhAGLO6p2Vmvfal+MoJqtvtxTHiQDEIkoYgJhS09CkOXkFamy2+vZpw3TB+H6uIwGIUZQwADHlwZfXaVN5tTL6puquS8e5jgMghlHCAMSM11bv1qJPtyspIU4LZmUpJTHedSQAMYwSBiAm7K6s09zn/OMo5l48VuMGdHOcCECso4QBiHrNPqub8gt1sKZR54xJ13fPHO46EgBQwgBEv4Xvb9bfN+9Tn9QkPXr1FBnDOAoA7lHCAES1otKDevz1DZKkR6+eovS0ZMeJAMCPEgYgalXXN2lOXqGafFbfOWO4zh3b13UkAPgHShiAqPWTl9ZoS0W1xvZP09xLxrqOAwD/hBIGICq9UrRLi1eUKplxFADCFCUMQNTZebBWd7SMo7j70nE6qV+a40QA8K8oYQCiSrPP6qa8Qh2qa9L54/rqW6cNcx0JAFpFCQMQVX79bok+3bpf6WnJmnfVZMZRAAhblDAAUeOL7Qf05JvFkqQnrpmi3qmMowAQvihhAKKCt65RN+YVqtln9Z9njdBZGemuIwHAMVHCAESF+15co+37azR+QDfdctEY13EA4LgoYQAi3ouFO/VcwU6lJPrHUSQnMI4CQPhLcB0AQIzxeqX8fKm4WMrIkLKzpbTOj5DYsb9Gdz+/WpJ072UTNLpvaqCSAkBQUcIAhM7y5dL06ZLPJ1VXSx6PlJsrLVsmTZvW4ZdravbpxvxCeeubdNGEfpp1ypAghAaA4OB0JIDQ8Hr9Bczr9Rcwyf/1y8erqjr8kr94u0Sfbzug/t1S9MiVjKMAEFkoYQBCIz/ffwSsNT6ff3kHrNi6X794u1jG+MdR9PQkBSAkAIQOJQxAaBQXHz4CdrTqaqmkpN0vVVnbqDl5hfJZ6ftnj9IZo/sEKCQAhA4lDEBoZGT4rwFrjccjjR7drpex1uruF1Zr58FaTR7cXbkXnBTAkAAQOpQwAKGRnS3FtfGWExfnX94Oz32xUy+tLFPXpHjNz8lSUgJvYwAiE+9eAEIjLc1/F2Ra2uEjYh7P4cdTjz9aYtu+at37on8cxf2XT9CIPm0cWQOACMCICgChM22aVFbmvwi/pMR/CjI7u10FrLHZp9l5hapuaNalkwZo5tTBIQgMAMFDCQMQWqmp0nXXdfjX5r9ZrJU7Dmpg9xQ9fMUkxlEAiHicjgQQ9j7evE+/erdExkhPZmeqe9dE15EA4IQdt4QZY24wxvQMRRgAOFplTaNy8wtlrfSjc0br1JG9XUcCgIBoz5Gw/pI+M8YsNsZcbDgHACBErLW64/kilVXWKXNID805P8N1JAAImOOWMGvt3ZIyJP1O0nckFRtjHjbGjApyNgAxbsmKUi1btVuepHjNz8lUYjxXUACIHu16R7PWWkm7W/5rktRT0jPGmJ8HMRuAGLa5vEr3LV0jSXrwGxM1rDfjKABEl+PeHWmMmS3pWkkVkn4r6VZrbaMxJk5SsaTbghsRQKxpaPJpTl6hahub9fUpA3VF1iDXkQAg4NozoqKPpCuttduOfNBa6zPGXBacWABi2eNvbNCqnZUa3LOLHrpiIuMoAESl45Ywa+29x1i2LrBxAMS6D0sqtPD9zYoz0lPZmeqWwjgKANGJq1wBhI0D1Q3KXewfRzH7vAxNHd7LdSQACBpKGICwYK3V7c8Wac+hek0d1lM3nDvadSQACCpKGICw8NdPt+v1tXuUlpygJ7MzlcA4CgBRjnc5AM6V7PXqwZfXSpIeumKihvTq6jgRAAQfJQyAU/VNzfrxokLVNfp05cmDNCOTcRQAYoOzEmaMiTfGFBhjXnaVAYB7j762Qet2HdLQXl31wIyJruMAQMi4PBI2RxIjLoAY9v7Gcv12+RbFxxnNz8lUanJ7RhcCQHRwUsKMMYMlXSr/BH4AMaiiql65i1dKknIvOElZQ3s6TgQAoeXqSNhT8n/cka+tJxhjrjfGrDDGrCgvLw9dMgBBZ63V7c8UqaKqXqeO6KUf/Nso15EAIORCXsJaPupor7X282M9z1q70Fo71Vo7NT09PUTpAITC/328TW+t36tuKf5xFPFxfCwRgNjj4kjYmZK+bozZKilP0teMMX92kAOAAxt2e/XQK/7LQR+5arIG9ujiOBEAuBHyEmatvcNaO9haO1xSjqS3rbXfCnUOAKFX19is2YsK1NDkU/bUIZo+aYDrSADgDHPCAITMI6+u14Y9Xo3o49G9l493HQcAnHJ6P7i19l1J77rMACA03l6/R09/tFWJ8UYLcrLkYRwFgBjHkTAAQbfXW6dblxRJkm6+cIwmDe7uOBEAuEcJAxBUPp/VrUuKtK+6QWeM6q3rzxrpOhIAhAVKGICg+sNHW/XexnL16JqoJ67JVBzjKABAEiUMQBCtLTukea+ulyTNu2qy+ndPcZwIAMIHJQxAUNQ2NGt2XoEamn365qlDddGE/q4jAUBYoYQBCIqfLlurkr1VGpXu0T2XMo4CAI5GCQMQcK+v2a0/f7xdSfFxWjArS12S4l1HAoCwQwkDEFB7DtXp9mf94yhuu3iMJgxkHAUAtIYSBiBgfD6r3MWFOlDTqLMy+uh7Z45wHQkAwhYlDEDA/O8Hm/VhyT719iTp8WumMI4CAI6BEgYgIFaVVuqx1zdIkn5+9WT1TWMcBQAcCyUMwAmraWjSnLwCNTZbXXv6MJ03rp/rSAAQ9ihhAE7YAy+t1eaKao3pl6Y7po9zHQcAIgIlDMAJeXXVLuV9tkNJCXGaPytTKYmMowCA9qCEAei0soO1mvvcKknSnZeM1dj+3RwnAoDIQQkD0CnNPqub8gtVWduoc8ek69ozhruOBAARhRIGoFN+894mfbJlv/qkJuvRmVNkDOMoAKAjKGEAOqxwx0E9+cZGSdJjMyerT2qy40QAEHkoYQA6pKreP46iyWf1vTNH6JwxfV1HAoCIRAkD0CH3vbhG2/bVaNyAbrr9kjGu4wBAxKKEAWi3pSvL9OwXpUpOiNOCnEwlJzCOAgA6ixIGoF1KD9Toruf94yjuuWy8MvqlOU4EAJGNEgbguJqafboxr1DeuiZdML6f/v3Uoa4jAUDEo4QBOK5fvbNJK7YdUN+0ZM27ajLjKAAgAChhAI7p8237teDtYknSE9dkqpcnyXEiAIgOlDAAbTpU16g5eYVq9ll9/+yRmpbRx3UkAIgalDAAbbr3hdUqPVCriYO66eYLGUcBAIFECQPQqucLSvVCYZm6JMZrfk6WkhJ4uwCAQOJdFcC/2L6vRve8sEaSdN/l4zUqPdVxIgCIPpQwAP+ksdmnOfkFqqpv0iUT+yv7q0NcRwKAqEQJA/BPfvFWsQq2H9SA7in62ZWTGEcBAEFCCQPwD59u2a9fvlMiY/zjKHp0ZRwFAAQLJQyAJKmyplE35hXIZ6Uf/tsonT6qt+tIABDVKGEAZK3VnS+sUlllnaYM7q6bLjjJdSQAiHqUMAB65vNSvVK0S54k/ziKxHjeGgAg2HinBWLclopq3bfUP47iJzMmangfj+NEABAbKGFADGto8mlOXoFqGpp12eQBuurkQa4jAUDMoIQBMezJNzeqqLRSg3p00U+vYBwFAIQSJQyIUR9tqtBv3tukOCM9lZOp7l0SXUcCgJhCCQNi0IHqBuXmr5S10g1fy9BXh/dyHQkAYg4lDIgx1lrd8dwq7T5Up5OH9tDsr412HQkAYhIlDIgxeZ/t0GtrdistOUHzc7KUwDgKAHCCd18ghmwqr9IDL62VJD10xUQN6dXVcSIAiF2UMCBG1Dc1a/aiAtU2NuuKrEGakck4CgBwiRIGxIjHX9+oNWWHNKRXFz0wY4LrOAAQ8yhhQAxYXlyhhe9vVnyc0fycLKWlMI4CAFyjhAFRbn91g3IXF0qSbjwvQycP7ek4EQBAclDCjDFDjDHvGGPWGWPWGGPmhDoDECustbrtmSLt9dbrlOG99F/nMo4CAMJFgoN1Nkm62Vr7hTEmTdLnxpg3rLVrHWQBotqfP9muN9ftUVpKgp7MyVR8HB9LBADhIuRHwqy1u6y1X7R875W0ThK3aQEBVrzHq4de9v/b5mdXTtKgHl0cJwIAHMnpNWHGmOGSsiR94jIHEG3qGpv140UFqm/yaeZXBuuyyQNdRwIAHMVZCTPGpEp6VtKN1tpDrSy/3hizwhizory8PPQBgQj289c2aP1ur4b37qr7v844CgAIR05KmDEmUf4C9hdr7XOtPcdau9BaO9VaOzU9PT20AYEI9u6Gvfr9h1uU0DKOwpPs4tJPAMDxhPzd2RhjJP1O0jpr7ROhXj8Qzcq99bplyUpJUu6FJ2nKkB6tP9HrlfLzpeJiKSNDys6W0tJCmBQA4OKfyGdK+rakVcaYwpbH7rTWLnOQBYga/nEUK1VR1aDTR/bW988e1foTly+Xpk+XfD6pulryeKTcXGnZMmnatNCGBoAYFvISZq1dLon75IEA++NHW/XOhnJ175KoJ7KntD6Owuv1FzCv9/Bj1dX+r9OnS2VlUmpqaAIDQIxjYj4QBdbvPqSHX10vSZp31SQN6N7GOIr8fP8RsNb4fP7lAICQoIQBEa6usVmzFxWoocmnWacM0cUTB7T95OLiw0e+jlZdLZWUBCckAOBfUMKACPfwsnXauKdKI9M9uuey8cd+ckaG/xqw1ng80mg+1ggAQoUSBkSwt9bt0Z/+vk2J8UYLcrLUNek4l3lmZ0txbfzZx8X5lwMAQoISBkSovYfqdOszRZKk2y4aq4mDuh//l9LS/HdBpqUdPiLm8Rx+nIvyASBkmOIIRCCfz+rmJSu1v7pBZ2X00XXTRrT/l6dN898FmZ/vvwZs9Gj/ETAKGACEFCUMiEC//3CLPiiuUC9Pkh6fOUVxrY2jOJbUVOm664ITDgDQLpyOBCLM6p2Vmvfal+MoJqtvtxTHiQAAnUEJAyJIbUOz5uQVqLHZ6tunDdMF4/u5jgQA6CRKGBBBHnxlrTaVVyujb6ruunSc6zgAgBNACQMixGurd+uvn2xXUkKcFszKUkpivOtIAIATQAkDIsDuyjrNfc4/jmLuxWM1bkA3x4kAACeKEgaEOZ/PKndxoQ7WNOqcMen67pnDXUcCAAQAJQwIcws/2KyPNu1Tn9QkPXr1FBnTwXEUAICwRAkDwlhR6UE99rcNkiv47rIAAA0DSURBVKRHr56i9LRkx4kAAIFCCQPCVHV9k+bkFarJZ/WdM4br3LF9XUcCAAQQJQwIUz95aY22VFRrbP80zb1krOs4AIAAo4QBYeiVol1avKJUyYyjAICoRQkDwszOg7W6o2Ucxd2XjtNJ/dIcJwIABAMlDAgjzT6rm/ILdaiuSeeP66tvnTbMdSQAQJBQwoAw8ut3S/Tplv1KT0vWvKsmM44CAKIYJQwIEwXbD+jJN4slSU9cM0W9UxlHAQDRjBIGhAFvXaPm5BWq2Wf1n2eN0FkZ6a4jAQCCjBIGhIH7lq7R9v01Gj+gm265aIzrOACAEKCEAY69WLhTz32xUymJ/nEUyQmMowCAWEAJAxzasb9Gdz+/WpJ072UTNLpvquNEAIBQoYQBjjQ1+3RTfqG89U26aEI/zTpliOtIAIAQooQBjvzynRKt2HZA/bol65ErGUcBALGGEgY4sGLrfi14q1jGSE9ek6meniTXkQAAIUYJA0LsUMs4Cp+Vvn/2KJ0xuo/rSAAAByhhQAhZa3XX86u182CtJg/urtwLTnIdCQDgCCUMCKHnvtipl1aWqWtSvObnZCkpgT9BAIhV/B8ACJFt+6p174v+cRT3f32CRvTxOE4EAHCJEgaEQGOzT7PzClXd0KxLJw3QzK8Mdh0JAOAYJQwIgflvFmvljoMa2D1FD18xiXEUAABKGBBsH2/ep1+9W6I4Iz2ZnanuXRNdRwIAhAFKGBBElTWNuim/UNZKPzp3tE4d2dt1JABAmKCEAUFirdUdzxdpV2WdMof00OzzMlxHAgCEEUoYECRLVpRq2ardSk1O0IKcLCXG8+cGADiM/ysAQbC5vEr3LV0jSXpgxgQN7d3VcSIAQLihhAEB1tDk05y8QtU2NuvrUwbqiqxBriMBAMIQJQwIsMff2KBVOys1uGcXPXTFRMZRAABaRQkDAujDkgotfH+z4oz0VHamuqUwjgIA0DpKGBAgB6oblLvYP45i9nkZmjq8l+tIAIAwRgkDAsBaq9ufLdKeQ/WaOqynbjh3tOtIAIAwRwkDAuCvn27X62v3KC05QU9mZyqBcRQAgOPg/xTACSrZ69WDL6+VJD10xUQN6cU4CgDA8VHCgBNQ39SsHy8qVF2jT1eePEgzMhlHAQBonwQXKzXGXCxpvqR4Sb+11j7iIgdwoh59bYPW7Tqkob266oEZEzv2y16vlJ8vFRdLGRlSdraUlhacoACAsBPyEmaMiZf0K0kXSCqV9JkxZqm1dm2oswAn4v2N5frt8i2KjzOan5Op1OQO/DktXy5Nny75fFJ1teTxSLm50rJl0rRpwQsNAAgbLk5HniKpxFq72VrbIClP0gwHOYBOq6iqV+7ilZKkm87PUNbQnu3/Za/XX8C8Xn8Bk/xfv3y8qioIiQEA4cZFCRskaccRP5e2PAZEBGutbn+mSBVV9TplRC/98JwOjqPIz/cfAWuNz+dfDgCIei5KWGuf4WL/5UnGXG+MWWGMWVFeXh6CWED7/N/H2/TW+r3qlpKgp7IzFR/XwY8lKi4+fATsaNXVUknJiYcEAIQ9FyWsVNKQI34eLKns6CdZaxdaa6daa6emp6eHLBxwLBt2e/XQK+skST+7crIG9ujS8RfJyPBfA9Yaj0cazaBXAIgFLkrYZ5IyjDEjjDFJknIkLXWQA+iQusZmzV5UoIYmn66ZOliXTh7QuRfKzpbi2vjTi4vzLwcARL2QlzBrbZOkGyT9TdI6SYuttWtCnQPoqEdeXa8Ne7wa0cej+y6f0PkXSkvz3wWZlnb4iJjHc/jx1NTABAYAhDUnc8KstcskLXOxbqAz3l6/R09/tFUJLeMoPB0ZR9GaadOksjL/RfglJf5TkNnZFDAAiCFOShgQSfZ663TrkiJJ0i0XjdHkwT0C88KpqdJ11wXmtQAAEYePLQKOweezunVJkfZVN+iMUb11/VkjXUcCAEQJShhwDH/4aKve21iuHl0T9cQ1mYrr6DgKAADaQAkD2rC27JDmvbpekjTvqsnq3z3FcSIAQDShhAGtqG1o1uy8AjU0+/TNU4fqogn9XUcCAEQZShjQip8uW6uSvVUale7RPZeOdx0HABCFKGHAUd5Yu0d//ni7kuLjND8nS12S4l1HAgBEIUoYcIQ9h+p02zMrJUm3XTxGEwd1d5wIABCtKGFAC5/P6ubFK3WgplFnZfTR984c4ToSACCKUcKAFr9dvlnLSyrUy5Okx2dOYRwFACCoKGGApNU7K/Xo3zZIkh69erL6dmMcBQAguChhiHk1DU2avahAjc1W/3H6MJ03rp/rSACAGEAJQ8x78OW12lxRrZP6perO6eNcxwEAxAhKGGLaq6t2adGnO5SUEKcFs7KUksg4CgBAaFDCELN2VdZq7nOrJEl3XjJWY/t3c5wIABBLKGGISc0+q5vyC1VZ26hzx6Tr2jOGu44EAIgxlDDEpP95f5M+3rxffVKT9ejMKTKGcRQAgNCihCHmrNxxUE+8vlGS9NjMyeqTmuw4EQAgFlHCEFOq6ps0J69ATT6r7505QueM6es6EgAgRlHCEFPuX7pGW/fVaGz/NN128RjXcQAAMYwShpjxclGZnvm8VMkJcfoF4ygAAI5RwhATSg/U6I6WcRR3XzZeGf3SHCcCAMQ6ShiiXrPPKjd/pbx1TbpgfD9969ShriMBACBjrXWd4biMMV5JG1zniCF9JFW4DhFD2N6hxfYOPbZ5aLG9Q2uMtbZTp1cSAp0kSDZYa6e6DhErjDEr2N6hw/YOLbZ36LHNQ4vtHVrGmBWd/V1ORwIAADhACQMAAHAgUkrYQtcBYgzbO7TY3qHF9g49tnlosb1Dq9PbOyIuzAcAAIg2kXIkDAAAIKqEZQkzxsw0xqwxxviMMW3e4WGMudgYs8EYU2KMmRvKjNHEGNPLGPOGMaa45WvPNp631RizyhhTeCJ3g8Sq4+2vxm9By/IiY8zJLnJGi3Zs73OMMZUt+3OhMeZeFzmjhTHm98aYvcaY1W0sZ/8OoHZsb/bvADLGDDHGvGOMWdfST+a08pwO7+NhWcIkrZZ0paT323qCMSZe0q8kXSJpvKRZxpjxoYkXdeZKestamyHprZaf23KutTaT2587pp376yWSMlr+u17Sr0MaMop04P3hg5b9OdNa+0BIQ0afpyVdfIzl7N+B9bSOvb0l9u9AapJ0s7V2nKTTJP0oEO/hYVnCrLXrrLXHG856iqQSa+1ma22DpDxJM4KfLirNkPTHlu//KOkbDrNEq/bsrzMk/cn6fSyphzFmQKiDRgneH0LMWvu+pP3HeAr7dwC1Y3sjgKy1u6y1X7R875W0TtKgo57W4X08LEtYOw2StOOIn0v1rxsE7dPPWrtL8u9okvq28Twr6XVjzOfGmOtDli46tGd/ZZ8OnPZuy9ONMSuNMa8aYyaEJlrMYv8OPfbvIDDGDJeUJemToxZ1eB93NjHfGPOmpP6tLLrLWvtie16ilce41bMNx9reHXiZM621ZcaYvpLeMMasb/nXGI6vPfsr+3TgtGdbfiFpmLW2yhgzXdIL8p9GQHCwf4cW+3cQGGNSJT0r6UZr7aGjF7fyK8fcx52VMGvt+Sf4EqWShhzx82BJZSf4mlHrWNvbGLPHGDPAWrur5dDp3jZeo6zl615jzPPyn/KhhLVPe/ZX9unAOe62PPIN1Fq7zBjz38aYPtZaPnMvONi/Q4j9O/CMMYnyF7C/WGufa+UpHd7HI/l05GeSMowxI4wxSZJyJC11nClSLZV0bcv310r6lyORxhiPMSbty+8lXSj/DRRon/bsr0sl/UfLHTanSar88jQxOuy429sY098YY1q+P0X+98N9IU8aO9i/Q4j9O7BatuXvJK2z1j7RxtM6vI+H5Qd4G2OukPQLSemSXjHGFFprLzLGDJT0W2vtdGttkzHmBkl/kxQv6ffW2jUOY0eyRyQtNsZcJ2m7pJmSdOT2ltRP0vMtf9MJkv5qrX3NUd6I09b+aoz5Qcvy30haJmm6pBJJNZK+6ypvpGvn9r5a0g+NMU2SaiXlWKZXd5oxZpGkcyT1McaUSrpPUqLE/h0M7dje7N+Bdaakb0taZYwpbHnsTklDpc7v40zMBwAAcCCST0cCAABELEoYAACAA5QwAAAAByhhAAAADlDCAAAAHKCEAQAAOEAJAwAAcIASBiAmGGO+aowpMsaktHwCxBpjzETXuQDELoa1AogZxpiHJKVI6iKp1Fr7M8eRAMQwShiAmNHyOZKfSaqTdIa1ttlxJAAxjNORAGJJL0mpktLkPyIGAM5wJAxAzDDGLJWUJ2mEpAHW2hscRwIQwxJcBwCAUDDG/IekJmvtX40x8ZI+MsZ8zVr7tutsAGITR8IAAAAc4JowAAAAByhhAAAADlDCAAAAHKCEAQAAOEAJAwAAcIASBgAA4AAlDAAAwAFKGAAAgAP/HxTdPmMVvbemAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig = plot_data(X1, y, xlabel='x', ylabel='y')\n", + "theta_start = np.matrix([0, 0]).reshape(2, 1)\n", + "theta, _ = gradient_descent(cost, gradient, theta_start, X1, y, eps=0.00001)\n", + "plot_fun(fig, polynomial_regression(theta), X1)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "Ten model ma duże **obciążenie** (**błąd systematyczny**, *bias*) – zachodzi **niedostateczne dopasowanie** (*underfitting*)." + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[]" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAFoCAYAAAAfEiweAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3yV5cH/8e91ssmAACFA2CTsKWGDIm4caBVxVlstWldt+1StHc/z9Pm1tba1tVatVq0LFQsOFMRRQUWGhr1JGBkEQoCQBZnn+v1xAiKCBEjOdcbn/XrxSnLOCfl6v26O31zXfV+XsdYKAAAA/uVxHQAAACAcUcIAAAAcoIQBAAA4QAkDAABwgBIGAADgACUMAADAgWYrYcaY54wxu40xa494rLUx5kNjTHbDx+Tm+vkAAACBrDlHwp6XdOFRjz0g6T/W2gxJ/2n4GgAAIOyY5lys1RjTTdK71toBDV9vkjTBWrvTGNNB0gJrbe9mCwAAABCg/H1NWKq1dqckNXxs5+efDwAAEBAiXQc4HmPMNEnTJCk+Pn5Ynz59HCcCAAD4umXLlu2x1qacyvf6u4QVGWM6HDEduft4L7TWPi3paUnKzMy0WVlZ/soIAADQKMaY3FP9Xn9PR86WdFPD5zdJetvPPx8AACAgNOcSFa9KWiyptzGmwBhzi6SHJJ1njMmWdF7D1wAAAGGn2aYjrbXXHuepc5rrZwIAAAQLVswHAABwgBIGAADgACUMAADAAUoYAACAA5QwAAAAByhhAAAADlDCAAAAHKCEAQAAOEAJAwAAcIASBgAA4AAlDAAAwAFKGAAAgAOUMAAAAAcoYQAAAA5QwgAAAByghAEAADhACQMAAHCAEgYAAOBApOsAAIDwYa1Vndeqps7r+1Pv+1h91Ne+z+u/9lyrFtE6o0srtUmIcf2fATQJShgAoEmVHqzV5zl79MnaQn2xvkDlNV5VR0SqxhOpmnqvrD29v79H23id0TVZmV2TNaxrsnqmJMjjMU0THvAjShgA4LRYa7WusEyfbC7WJ5uKtSyvRPXeQ00rUjKSvJK8XklShMcoOsKj6MiGPxEexUR+/etjfb6j5KBWFezX1j2V2rqnUjOXFUiSWsZFaVhDIRvWNVmDO7VSXHSEk2MBnAxKGADgpJUeqNVnOcVasKlYn2wuVnF59eHnIow0YscGTcheqjO3r1C7in2Krq/1/YmLVeSOAikh4ZR+bm29V+sLy7Qst0TLckuUlbtPRWXV+njjbn28cbckKdJj1L9jkoZ1ba3Mbr5ilpoU2yT/3UBTMvZ0x4X9IDMz02ZlZbmOAQBhy+v1jXYt2LRbCzYXa0VeibxH/O+jfVKsJvRO0Vm9UjR20Vwl/fRHUmXlN/+i+Hjp0UelW25pklzWWu3Yf/CrUra9RBt3lX0tmyR1So47PH05vHtr9U5NlDFMYeL0GWOWWWszT+V7GQkDABxTSWWNPs32jXR9urlYeypqDj8X6TEa2b21zuqdogm9U75eal7efOwCJvkez8lpsozGGHVKbqFOyS00eUiaJKmiuk4r8/YrK3efluWWaEXefhWUHFRByUG9tbJQknRGl1a6e2KGJvROoYzBGUoYAOCwmjqvZmTl643lBVqVv/9rI0odW8bqrN7tNKF3isamt1VCzHH+F5KR4RvxOt5IWHp684RvkBATqXEZbTUuo60kqd5rtbmoXFm5JVqeW6L5m3Zred5+fe/5LzUgLUl3nZ2h8/ulcnE//I7pSAAIR+Xl0owZUna2lJEh79VXa862Cv3pg03K3XtAkhQVYTSie2tN6OUrXuntEho3alReLqWl+T4eLTFRKiw85WvCmkJldZ1eWZqnpz7dqj0VvmvZeqcm6s6J6bp4YAdFUMZwEk5nOpISBgDhZuFCadIk392KlZVa1GuEHhp7vVa36ylJ6pkSrx+d20vn9Gmn+OONdp3kz1B8vOTxSHPnSuPGNeF/zKmrqq3XjC/z9Y9PtmhnaZUk3/IXd5ydrslDOioqgvXMcWKUMABA4xwxSrU+pbv+MOEmfdLD9/+PdpUl+vGUkZoytqcim6KAVFT4RttycnxTkFOnOh0BO56aOq9mLS/QEwtylL/voCTfhfw/nNBTVw3rpJhIlrvA8VHCAACN88wzyv/Vb/XIsO/orf4TZI1HidWVun3JTH1vw3/U4s8PN9mdi8Gmtt6r2SsL9fiCHG0t9l3P1j4pVred1UPXDO/C2mM4Ju6OBACcUElljR7PrteL1/9FNZFRiqqv1Y3LZ+uuxa+r9cEy34ua8M7FYBMV4dGVwzrp8qFpmrtmp/7+cY42FZXrf99Zr8fn5+gH43vo+lFdj39DAnCSOJMAIMQdrKnXc59v0z8WbFG56SRFSpevm6+ffvayOpcWffVCP9y5GAwiPEaXDu6oiwd20EcbivTYxzlas6NUv39vo578ZIu+P7a7bhrTTS3jolxHRZBjOhIAQlRdvVczlxXoLx9tVlGZ7y7A8T2Sdf9Dt2vAtjXf/IYAuHMxEFlr9cnmYj32cY6W5ZZIkhJjInXTmG66ZVx3JcdHO04Il7gmDABwmLVWH64v0sPvb1LO7gpJ0oC0JD1wYV/f2llBcOdiILLWavHWvfr7xzlatGWvJKltQrT+MnWIxmekOE4HVyhhAABJUtb2fXrovY3Kahix6dw6Tv91fm9dOqjj1xcjDZI7FwPVstx9+sO8Tfpi2z4ZI90xoad+fG6vprmrFEGFEgYAYa64vFq/fGuN3l/nu8ardXy07p6YrutHdlV0JMWgOdR7rR6fn6O/frRZXisN75asv107VB1axrmOBj+ihAFAGFuVv1+3v7xMO0urFBcVoVvHd9e0M3soMZYLx/1hyda9+tFrK1RUVq3kFlH689WDNbFPqutY8JPTKWH8egQAQez1rHxNeWqxdpZW6YwurfTxf52ln57fmwLmR6N6tNHce8ZrQu8UlRyo1fefz9Jv56xXTZ3XdTQEOEoYAAShmjqvfv32Wt03c7Vq6ry6fmQXvTZtNFNhjrRJiNFzNw3Xzy/qowiP0T8/26YpTy1W/r4DrqMhgFHCACDI7C6v0vXPLNGLi3MVHeHRQ98ZqN9eMZBrvxzzeIxuO6unXr9ttNJaxWlV/n5N+ttnem/NTtfREKD4FwsAQWRFXokufWyhvtxeotSkGM24bZSuGdHFdSwcYVjXZM25Z5zO75eq8qo6/XD6cv367bWqqq13HQ0BhhIGAEFixpd5mvrUEhWVVWt4t2S9c/c4De2S7DoWjqFVi2g9deMw/c+l/RQd4dGLi3P1nScWaWtxhetoCCCUMAAIcDV1Xv3izTW6f9Ya1dR79d3RXTX91lFqlxjrOhq+hTFGN4/trlk/HKOubVpo/c4yXfrYQr29cofraAgQlDAACGC7y6p07T+XaPrSPEVHevTwVYP0m8kDuP4riAzs1FLv3j1OlwzqoMqaev3otZW6f+ZqHaxhejLc8a8YAALUstx9uuSxhVqWW6IOLWP179tG6+rMzq5j4RQkxkbpsWuH6ndXDFRMpEczsvI1+fGF2lxU7joaHKKEAUAAemVpnq55eol2l1drRPfWeufucRrcuZXrWDgNxhhdN7KL3rpzrHqmxGtzUYUu+/tCvf5lvoJh4XQ0PUoYAASQ6rp6/fyN1XrwzTWqrbe6eUw3Tb91pNomxLiOhibSt0OSZt81Tlee0UlVtV7dN2u1fjxjJXdPhqFI1wEAAD5FZVW6/eVlWpG3X9GRHv3uioG6algn17HQDOJjIvXnqwdrdM82+tVba/XWykLtqajRMzdlKjYqwnU8+AkjYQAQALK2+67/WpG3Xx1bxmrW7WMoYGHgqmGd9NadY9U2IUYLc/bo1heyGBELI05KmDHmx8aYdcaYtcaYV40x3GcNICxZa/XSklxd8/QSFZdXa1QP3/VfAzu1dB0NftK7faJe/cFIilgY8nsJM8akSbpHUqa1doCkCEnX+DsHALhmrdVv3l2vX721VnVeq1vGddfLt4xUG67/CjsZqRSxcORqOjJSUpwxJlJSC0mFjnIAgDMPv79J//p8u6IjPPrL1MH61SX9FBnBVSLhKiM1Ua9N+3oRYy2x0Ob3f+3W2h2S/iQpT9JOSaXW2g+Ofp0xZpoxJssYk1VcXOzvmADQrB6fn6MnF2xRhMfo8evP0BVDuf4LUnq7rxexH7xIEQtlLqYjkyVNltRdUkdJ8caYG45+nbX2aWttprU2MyUlxd8xAaDZPP/5Nv3x/U0yRnrk6sE6r1+q60gIIBSx8OFi3PtcSdustcXW2lpJb0ga4yAHAPjd61n5+p931kuSfnfFQE0ekuY4EQIRRSw8uChheZJGGWNaGGOMpHMkbXCQAwD8as7qnXpg1mpJ0i8v7qtrR3RxnAiB7OgiduuLX1LEQoyLa8KWSpopabmkNQ0ZnvZ3DgDwp483FulHr62Q10o/PreXbh3fw3UkBAFfERultgkx+jxnL0UsxDi5Dcda+9/W2j7W2gHW2huttdUucgCAPyzaske3v7xcdV6raWf20D3npLuOhCCS3i6BIhaiuBcaAJrRirwS/eCFLNXUeXXdyC76+UV95LsSA2g8ilhoooQBQDNZX1imm577QpU19bp8SEf9v8kDKGA4ZYeKWEoiRSxUUMIAoBlsKa7Qjc8uVVlVnc7vl6o/TRksj4cChtOT3i5Br/7gqyJ2ywsUsWBGCQOAJpa/74BueGap9lbWaHxGWz123VBWwkeTObKILdpCEQtmvCsAQBPaXValG55dqp2lVRreLVlP35ipmMgI17EQYihioYESBgBNZF9lja5/Zqly9x7QwLSWevbm4YqLpoCheRx5jRhFLDhRwgCgCZRV1eq7zy1V9u4KZbRL0AvfH6Gk2CjXsRDieqZQxIIZJQwATtOBmjp9/19fau2OMnVt00LTbx2p1vHRrmMhTBxdxO6Yvkz1Xus6FhqBEgYAp6G6rl63vbRMWbkl6tAyVtNvHal2SbGuYyHMHCpiyS2iNH9Tsf7y4WbXkdAIlDAAOEW19V7d/coKfZa9R20TojX91pHqlNzCdSyEqZ4pCXr8ujPkMdLf5+do3tqdriPhBChhAHAKvF6rn/17lT5YX6Sk2Ei9dMtI9UhJcB0LYW5Mels9OKmvJOmnr69SdlG540T4NpQwADhJ1lr98u21emtloeKjI/TC90eob4ck17EASdIt47rr0sEdVVlTr2kvLVNZVa3rSDgOShgAnKQXF+fqlaV5ion06Jmbhmtol2TXkYDDjDH6w5UD1ad9orbtqdSPX1spLxfqByRKGACUl0vPPCPdf7/vY/nxp3BW5u/X/5uzXpL0pymDNbpnG3+lBBqtRXSknr4xUy3jovSfjbv1t4+zXUfCMVDCAIS3hQultDTp3nulhx/2fUxL8z1+lJLKGt05fblq661uGt1Vlw7u6CAw0Dhd2rTQY9cOlcdIf/0oWx+uL3IdCUehhAEIX+Xl0qRJvo+Vlb7HKiu/eryi4vBLvV6rn7y+Ujv2H9Tgzq304MV9HYUGGu/MXin62QV9JEk/mbFSW4orTvAd8CdKGIDwNWOG5PUe+zmv1/d8gyc/2aL5m4rVMi5Kj183lP0gETRuP6uHJg1sr/LqOk17MUvlXKgfMChhAMJXdvZXI2BHq6yUcnIkSYu37NWfP9gkSfrL1MGsBYagYozRH68arF6pCdpSXKmfvr6KC/UDBCUMQPjKyJDi44/9XHy8lJ6u3WVVuvvVFfJa6Y4JPTWxT6p/MwJNID4mUk/dmKnE2Eh9sL5ITyzIcR0JooQBCGdTp0qe47wNejyqm3K17n51hfZUVGtUj9b6yXm9/JsPaELd28br0WuGyBjpzx9u1vyNu11HCnuUMADhKzFRmjvX9/HQiFh8/OHHH1m0Q0u37VNKYoz+du1QRUbwlongNrFPqn5ybi9ZK93z2gpt33Oc6Xj4Be8oAMLbuHFSYaH06KPSAw/4PhYW6uO2GXpiwRZ5jPS3a4aqXSKbciM03Hl2us7vl6ryqjpNeylLldV1riOFLWNt4F+cl5mZabOyslzHABAm8vcd0CWPLVTpwVrdd2Fv3TEh3XUkoEmVV9Xq8sc/15biSl08sIP+ft1QGWNcxwpKxphl1trMU/leRsIA4AjVdfW665XlKj1Yq3P6tNPtZ/Z0HQlocomxUXrqxkwlxERqzpqdeurTra4jhSVKGAAc4XdzNmhVQanSWsXpz1cPlsfD6ABCU3q7BD1y9WBJ0sPzNurTzcWOE4UfShgANHhnVaFeWJyrqAijx68/Q61aRLuOBDSr8/u31z3nZMhrpbtfXaG8vQdcRworlDAAkLSluEIPzFotSfrVJf00pHMrx4kA/7j3nAyd06edSg/WatpLWTpQw4X6/kIJAxD2DtbU646Xl6uypl6XDu6oG0d1dR0J8BuPx+iRqUPUvW28Nu4q1wOz1igYbtoLBZQwAGHNWqtfvLVGm4rK1SMlXr//zkDuEkPYaRkXpaduHKYW0RGavapQzy7c5jpSWKCEAQhrM77M1xvLdyg2yqMnrx+mhJhI15EAJ3qlJurPU3wX6v9u7gYtytnjOFHoo4QBCFvrCkv169nrJEm/vXygerdPdJwIcOuigR10x4Se8lrpzleWq6isynWkkEYJAxCWyqpqdcf05aqp8+qa4Z115bBOriMBAeGn5/fW+Iy2KjlQq1+8uZbrw5oRJQxA2LHW6r5/r1bu3gPq1yFJ/3NZf9eRgIAR4TF6+KpBSoyJ1EcbijR7VaHrSCGLEgYg7Dy7cJvmrdulxJhIPXH9GYqNinAdCQgoHVrG6RcX95Uk/ffsddpdzrRkc6CEAQgry3L36aH3NkqS/jhlkLq1jXecCAhMU4d31viMttp/oFa/fmsd05LNgBIGIGzsq6zRXa+sUJ3X6pZx3XXhgA6uIwEByxijh64cpPjoCM1bt0tz1ux0HSnkUMIAhI1fvLlGO0urdEaXVnrgoj6u4wABL61VnH4+yTct+eu312lvRbXjRKGFEgYgLLy/bpfeW7tLLaIj9Ldrhyoqgrc/oDGuG9FFo3u00b7KmsNLuqBp8C4EIOSVVdXq12+vlSTdd0FvdUpu4TgREDw8DXdLtoiO0JzVOzVvLdOSTYUSBiDkPTxvo4rKqjWkcyvdOLqb6zhA0OncuoXuv9A3hf/Lt9aqpLLGcaLQQAkDENKytu/Ty0vyFOkxeujKgYrwsC8kcCpuHNVVI7q31p6KGv3vO0xLNgVKGICQVV1Xr/tnrZYk/XBCT/Vpn+Q4ERC8PB6jh68cpNgoj95aWagP1xe5jhT0KGEAQtYT87doS3GleqTE686z013HAYJet7bx+tkFvmnJX7y5RqUHah0nCm6UMAAhKbuoXE8syJEk/f6KgayKDzSRm8d007CuydpdXq3fvLvedZygRgkDEHK8XqsH3lij2nqra0d01sgebVxHAkLGob0lYyI9mrW8QPM37nYdKWhRwgCEnOlLc7Ust0QpiTF64KK+ruMAIadnSoJ+en4vSdLP31ijsiqmJU8FJQxASNlZelB/mLdJkvSby/qrZVyU40RAaLplXA8N6dxKu8qq9Nt3N7iOE5SclDBjTCtjzExjzEZjzAZjzGgXOQCEFmutfvXWOlVU1+m8fqm6cEB715GAkBXhMfrjVYMUHeHRjKx8fbq52HWkoONqJOxRSfOstX0kDZZEhQZw2t5bu0sfbShSYkyk/m/yABnDmmBAc8pITdS952VI8k1LljMteVL8XsKMMUmSzpT0rCRZa2ustfv9nQNAaCk9UKv/btjX7r6L+qh9y1jHiYDwMG18Dw3q1FI79h/U79/b6DpOUHExEtZDUrGkfxljVhhjnjHGxDvIASCEPDRvg4rLq5XZNVnXj+jiOg4QNiIjPPrjVYMVFWH0ytI8LcrZ4zpS0HBRwiIlnSHpSWvtUEmVkh44+kXGmGnGmCxjTFZxMfPMAI5vyda9evWLfEVFGP3+OwPlYWsiwK96t0/UPRN905L3zVqtyuo6x4mCg4sSViCpwFq7tOHrmfKVsq+x1j5trc201mampKT4NSCA4FFVW68H31gjSbrz7HRlpCY6TgSEp9sn9FT/jkkqKDmoh+cxLdkYfi9h1tpdkvKNMb0bHjpHEkvuAjglf/84R1v3VCq9XYJ+OKGn6zhA2IpqmJaM9Bi9sDhXS7budR0p4Lm6O/JuSdONMaslDZH0O0c5AASxjbvK9I9PtkiSHvrOQMVEsjUR4FK/jkmH92m9f9ZqHaypd5wosDkpYdbalQ1TjYOstZdba0tc5AAQvOq9VvfPWqM6r9WNo7oqs1tr15EAyHdZQJ/2icrde0B/fH+T6zgBjRXzAQSlFxdv16r8/WqfFKv7Lux9wtcD8I/oSI/+NGWwIjxG/1q0TVnb97mOFLAoYQCCzo79Bw//hv2byf2VGMvWREAgGZDWUref1UPWSvfNXK2qWqYlj4USBiCoWGv1yzfX6EBNvS4a0F7n92drIiAQ3XNOhjLaJWjrnko9+p9s13ECEiUMQFB5Z/VOzd9UrMTYSP3vZf1dxwFwHDGREXr4qkGSpGc/26b8fQccJwo8lDAAQaOkskb/27A10YOT+qpdElsTAYFsaJdkXTE0TTX1Xj3ElkbfQAkDEDR+O3eD9lbWaET31pqa2dl1HACN8LMLeis2yqM5a3Zykf5RKGEAgsLnOXs0c1mBoiM9bE0EBJGOreI0bXwPSdL/vbteXq91nChwUMIABLyq2no9+KZva6J7JqarZ0qC40QATsZtZ/VUu8QYrSoo1durdriOEzAoYQAC3l8/ylbu3gPqnZqoaWeyNREQbOJjIvWzC3zr+T08bxMr6TeghAEIaFuLK/TMZ1tljPTQlQMVHcnbFhCMrjyjk/p3TNLO0ir987OtruMEBN7NAAS0h+dtUp3X6uphnTW0S7LrOABOkcdj9KtL+kmSnlywRUVlVY4TuUcJAxCwluXu07x1uxQb5dFPzu/lOg6A0zSqRxtd0D9VB2vr9Sf2laSEAQhM1lr9bq5vXaEfjO+hVNYEA0LCzy/qq6gIo5nLC7R2R6nrOE5RwgAEpPfX7dKy3BK1iY/WtDN7uI4DoIl0axuvm0Z3k7W+JSusDd8lKyhhAAJObb1Xf5jnm6q499wMNugGQszd52QouUWUlm7bpw/WF7mO4wwlDEDAee2LPG3bU6nubeN1zYguruMAaGIt46J077m+6zx/P3eDauq8jhO5QQkDEFDKq2r114+yJUn3X9hbURG8TQGh6LqRXdQzJV7b9x7Qi4u3u47jBO9uAALK059u1d7KGg3rmqwL+rd3HQdAM4mK8OiXF/uWrHj0P9naV1njOJH/UcIABIyisq8WcXxwUh8Zw/6QQCib0DtF4zPaqryqTo9+tNl1HL+jhAEIGI98sFlVtV5d2L+9hnVt7ToOgGZmjNEvL+4nj5FeXpqnnN3lriP5FSUMQEDYtKtc/16Wr0iP0X0X9nYdB4Cf9G6fqGtGdFG91+q3cza4juNXlDAAAeEP8zbKa30X6/ZISXAdB4Af/eS8XkqIidT8TcX6dHOx6zh+QwkD4NyiLXv08cbdSoiJ1D3nZLiOA8DP2ibE6M6z0yVJv52zQXX14bFkBSUMgFNer9XvG7Ynuv2sHmqbEOM4EQAXvje2mzolx2lTUblmZOW7juMXlDAATr2zulBrdpQqNSlGt4xjeyIgXMVGRejnF/WV5LtJp7yq1nGi5hfpOgCAMFNeLs2YIWVnqzo9Q3/c002S75qQuOgIt9kAODVpYHtldk1WVm6JHp+/RQ9c1Md1pGbFSBgA/1m4UEpLk+69V3r4Yb30wocqKK1Wr0SPrhrW2XU6AI4ZY/SrS3wLuD63cJvy9x1wnKh5UcIA+Ed5uTRpku9jZaVKY+L12LArJEk/f/X3ijhQ6TgggEAwuHMrXTE0TTX1Xj00b6PrOM2KEgbAP2bMkLxf3fH0+OirVRqXqNG5qzRh+3Lf8wAg6WcX9FZslEdzVu9U1vZ9ruM0G0oYAP/IzpYqfaNd+Unt9PywyyRJD85/TqayUsrJcZkOQADp2CpO08b7btT5v3fXy+u1jhM1D0oYAP/IyJDi4yVJj4y/QTWRUZq8boEGFm3xPZ6e7jgggEBy21k91S4xRqsKSjV7VaHrOM2CEgbAP6ZOlTwerU3tqTcHTFR0Xa3+67OXfM95PL7nAaBBfEyk/usC3xZmf5i3UQdr6h0nanqUMAD+kZgoO2eOfnfuDyRJ313+rjrXVUiJidLcuVICWxUB+Lqrzuik/h2TtLO0Sv/8bKvrOE2OEgbAbz5p11uLOg1Qkseru0Z0kB59VCoslMaNcx0NQADyeIx+ebFvyYonF2xRUVmV40RNixIGwC/qj9ie6K4L+6nVQ/8n3XILI2AAvtXonm10fr9UHayt15/e3+Q6TpOihAHwi1nLC7SpqFxpreL03dHdXMcBEEQenNRXURFGM5cXKGd3ues4TeaEJcwYc5cxJtkfYQCEpoM19Xrkg82SDq3/w/ZEABqvW9t4XZ3ZWdZKj30cOsvZNGYkrL2kL40xrxtjLjTGmOYOBSC0PPf5Nu0qq1L/jkm6bHBH13EABKE7zk5XVITRO6sKtaW4wnWcJnHCEmat/aWkDEnPSrpZUrYx5nfGmJ7NnA1ACNhbUa0nF2yR5JtS8Hj4PQ7AyUtrFaerhnWS10qPh8hoWKOuCbPWWkm7Gv7USUqWNNMY83AzZgMQAh77OEcV1XWa0DtFY9Pbuo4DIIjdMSFdkR6jt1bu0PY9wb/fbGOuCbvHGLNM0sOSPpc00Fr7Q0nDJF3ZzPkABLFteyr18pJcGSM9cFEf13EABLnOrVvoO2ek+UbD5gf/aFhjRsLaSvqOtfYCa+2/rbW1kmSt9Uq6pFnTAQhqf3x/o+q8Vled0Ul92ie5jgMgBNx5droiPEZvrNihvL0HXMc5LY25JuzX1trc4zy3oekjAQgFK/JKNHfNLsVGefST83u5jgMgRHRtE6/Lh6Sp3mv1xILgHg1jnTAAzeLR/2RLkr43trs6tIxznAZAKLnz7J7yGGnmsgIVlATvaBglDECTW7ujVAs2FSsuKjx57UQAABYhSURBVEI/GN/DdRwAIaZHSoIuG9xRdV6rJxruvg5GlDAATe7QBbPXj+yi1vHRjtMACEV3TcyQMdK/s/JVuP+g6zinhBIGoEnl7C7XvHW7FB3h0Q/OZBQMQPNIb5egSwZ1VG291T8+Cc7RMEoYgCb1xPwtslaaktlJqUmxruMACGF3T0yXMdJrX+RrV2mV6zgnzVkJM8ZEGGNWGGPedZUBQNPK23tAb68qVITH6Paz2FQDQPPqlZqoSQM6qKbeG5SjYS5Hwn4kiSUugBDyj0+3qN5rNXlIR3Vu3cJ1HABh4O5z0iVJr36Rp91lwTUa5qSEGWM6SbpY0jMufj6AprertEozswpkjG9rEQDwhz7tk3Rh//aqrvPqqU+3uo5zUlyNhP1V0n2SvMd7gTFmmjEmyxiTVVxc7L9kAE7JPz/bqpp6ry4a0F7p7RJcxwEQRg6Nhk1fmqvi8mrHaRrP7yXMGHOJpN3W2mXf9jpr7dPW2kxrbWZKSoqf0gE4Ffsqa/TK0jxJjIIB8L/+HVvqvH6pqqr16p+fBc9omIuRsLGSLjPGbJf0mqSJxpiXHeQA0ESeW7hNB2vrdXbvFA1Ia+k6DoAwdM/EDEnSS4tztbciOEbD/F7CrLU/t9Z2stZ2k3SNpI+ttTf4OweAplFWVasXFm+XJN01kVEwAG4M7NRSE/u008Haej2zcJvrOI3COmEATstLi3NVXlWnUT1aa1jX1q7jAAhj95zjGw17cdF2lVTWOE5zYk5LmLV2gbX2EpcZAJy6AzV1erbhN867zs5wnAZAuBvSuZXO6pWiypr6w+9NgYyRMACn7NUv8rWvskaDO7fS2PQ2ruMAwOHRsOcXbdf+A4E9GkYJA3BKquvq9fSnvhWq7zo7XcYYx4kAQBrWNVnjM9qqorpOz32+3XWcb0UJA3BKZi3boaKyavVpn6hz+rRzHQcADjs0Gvavz7ep9GCt4zTHRwkDcNLqjtin7Y6z0+XxMAoGIHAM79ZaY3q2UXlVnZ4P4NEwShiAk/bO6kLl7Tug7m3jdfHADq7jAMA3HBoNe3bhVpVXBeZoGCUMwEnxeq2emO8bBfvhWT0VwSgYgAA0qkcbjejeWmVVdXpxca7rOMdECQNwUj5Yv0vZuyvUsWWsLh+a5joOABzXvQ2jYf/8bKsqquscp/kmShiARrPW6u/zcyRJt53VU9GRvIUACFyje7ZRZtdk7T9Qq5cCcDSMd1AAjfbJ5mKt3VGmtgkxmjq8s+s4APCtjDGHrw3752dbdaAmsEbDKGEAGu3xhlGwW8d3V2xUhOM0AHBi4zPaamiXVtpXWaOXlwTWaBglDECjLN26V19uL1HLuCjdMKqr6zgA0ChHjoY9/elWHaypd5zoK5QwAI1y6Fqwm8d0U0JMpOM0ANB4E3qlaHCnltpTUaPpSwNnNIwSBuCEVuXv12fZexQfHaHvje3mOg4AnJQjR8Oe+nSrqmoDYzSMEgbghA5dC3bDqK5q1SLacRoAOHkT+7TTgLQkFZdX67Uv8lzHkUQJA3ACm3aV64P1RYqO9OiW8d1dxwGAU2KM0T0TfaNhT36yJSBGwyhhAL7VEwt8o2DXDO+sdomxjtMAwKk7r1+q+nZIUlFZtf6dle86DiUMwPFt31Opd1YVKtJjdNtZPV3HAYDT4hsNS1fr+GhFRrivQNziBOC4/vHJFnmtdOUZaUprFec6DgCctgv6t9dZvVPUItp9BXJfAwEEpML9BzVreYE8RvrhBEbBAIQGj8cERAGTKGEAjuPpT7eqtt7q4kEd1SMlwXUcAAg5lDAA37Cnolqvfem7hfvOsxkFA4DmQAkD8A3PLtymqlqvzu2bqj7tk1zHAYCQRAkD8DWlB2r10mLfth53TUx3nAYAQhclDMDXPL9ouyqq6zQuva2GdG7lOg4AhCxKGIDDqmrr9cLi7ZKkO89mFAwAmhMlDMBh76wq1L7KGg1IS9KoHq1dxwGAkEYJAyBJstbq+UXbJUk3j+kuY4zbQAAQ4ihhACRJy3JLtK6wTK3jo3XJoA6u4wBAyKOEAZAk/athFOy6EV0UGxXhNgwAhAFKGADtLD2oeWt3KcJjdP2oLq7jAEBYoIQB0PQlear3Wl04oL06tGSjbgDwB0oYEOaqauv16he+LYpuHtPNbRgACCOUMCDMvbt6p/ZW1qh/xyRldk12HQcAwgYlDAhj1lq90HBB/k1jurEsBQD4ESUMCGPL80q0ZkepWsdH67LBHV3HAYCwQgkDwtjzi3wbdV8zvDPLUgCAn1HCgDBVVFal99bsVITH6IZRXV3HAYCwQwkDwtT0Jbmq81pd0D9VHVuxLAUA+BslDAhD1XX1euXwshTdHacBgPBECQPC0JzVO7WnokZ9OyRpeDeWpQAAFyhhQJix1ur5hmUpvseyFADgDCUMCDMr8vdrdUGpkltE6bIhLEsBAK5QwoAw8/zn2yVJ14zowrIUAOAQJQwII0VlVZq7Zqc8RixLAQCOUcKAMDJ9aV7DshTtlcayFADgFCUMCBPVdfV6ZalvWYqbxnRzGwYAQAkDwsXcNTu1p6JafdonamT31q7jAEDY83sJM8Z0NsbMN8ZsMMasM8b8yN8ZgHB0aJ/Im1mWAgACQqSDn1kn6afW2uXGmERJy4wxH1pr1zvIAoSFFXklWpW/X61aRGnykDTXcQAAcjASZq3daa1d3vB5uaQNkvi/AtCMXmhYnHXq8M6Ki2ZZCgAIBE6vCTPGdJM0VNJSlzmAULa7vEpzGpaluJFlKQAgYDgrYcaYBEmzJN1rrS07xvPTjDFZxpis4uJi/wcEQsQrS/NUW291Xr9UdUpu4ToOAKCBkxJmjImSr4BNt9a+cazXWGufttZmWmszU1JS/BsQCBE1dV5Nb1iW4uYx3R2nAQAcye8X5hvfbVnPStpgrX3E3z8fCCfvrd2p4vJq9U5N1KgeRyxLUV4uzZghZWdLGRnS1KlSYqK7oAAQhlzcHTlW0o2S1hhjVjY89qC1dq6DLEBI+1fDPpE3jz1iWYqFC6VJkySvV6qslOLjpZ/8RJo7Vxo3zl1YAAgzfi9h1tqFklikCGhmq/L3a2X+frWMi9Llh5alKC/3FbDy8q9eWFnp+zhpklRYKCUk+D8sAIQhVswHQtShZSmuOXJZihkzfCNgx+L1+p4HAPgFJQwIQcXl1XpndaE8RrrhyGUpsrO/Gvk6WmWllJPjn4AAAEoYEIpe/cK3LMW5fVPVufURy1JkZPiuATuW+HgpPd0/AQEAlDAg1NTUefXykq/2ifyaqVMlz3H+2Xs8vucBAH5BCQNCzLx1u7S7vFq9UhM0umebrz+ZmOi7CzIx8asRsfj4rx7nonwA8BsXS1QAaEbPf75NknTTmCOWpTjSuHG+uyBnzPBdA5ae7hsBo4ABgF9RwoAQsrpgv5bn7VdSbKSuGJp2/BcmJEi33OK/YACAb2A6EgghzzcsSzF1eGe1iOZ3LAAIZJQwIETsqajWu6t2yhjpu6O7uY4DADgBShgQIl5dmqeaeq/O6XPUshQAgIBECQNCQG29Vy8v9S1L8b2x3dyGAQA0CiUMCAEfri9SUVm1MtolaMzRy1IAAAISJQwIATO+zJckXTeyy7GXpQAABBxKGBDkCvcf1KfZxYqO8OjyId+yLAUAIKBQwoAgN2tZgayVzuufquT4aNdxAACNRAkDgpjXa/X6Mt9U5NTMzo7TAABOBiUMCGJLtu1V/r6DSmsVp7HpbV3HAQCcBEoYEMReb7gg/8phnRTh4YJ8AAgmlDAgSJUerNV7a3fJGGnKsE6u4wAAThIlDAhSs1cVqrrOq7E927JCPgAEIUoYEKQOTUVOyWQUDACCESUMCELrC8u0ZkepWsZF6YL+7V3HAQCcAkoYEIRez/KNgl0+pKNioyIcpwEAnApKGBBkqmrr9eaKHZKkKawNBgBBixIGBJkP1xep9GCt+ndM0oC0lq7jAABOESUMCDKHpiKnDmcUDACCGSUMCCIFJQe0MGePoiM9mjyYzboBIJhRwoAgMrNhs+4L+7dXyxZRruMAAE4DJQwIEl6v1b+zCiQxFQkAoYASBgSJRVv2asf+g+qUHKfRPdq4jgMAOE2UMCBIzGi4IH/KsM7ysFk3AAQ9ShgQBPYfqNH763ybdV/FNkUAEBIoYUAQeHtloWrqvBqfkaK0VnGu4wAAmgAlDAgCMxo2676aUTAACBmUMCDArd1RqvU7y5TcIkrn9Ut1HQcA0EQoYUCAO7xZ99A0xUSyWTcAhApKGBDAqmrr9VbDZt1Xs1k3AIQUShgQwN5ft0tlVXUa1Kml+nZIch0HANCEKGFAADs0FckoGACEHkoYEKDy9x3Q5zl7FRPp0aWDO7qOAwBoYpQwIED9u2EUbNLADmoZx2bdABBqKGFAAKr3Ws1c5tusm6lIAAhNlDAgAC3M2aPC0ip1ad1CI7u3dh0HANAMKGFAAHr9iBXy2awbAEITJQwIMPsqa/TB+l3yGOnKYWxTBAChihIGBJi3VuxQbb3Vmb1S1KElm3UDQKiihAEBxFp7eG2wqVyQDwAhjRIGBJDVBaXauKtcreOjdU5fNusGgFAW6eKHGmMulPSopAhJz1hrH3KRA3CqvFyaMUPKzpYyMqSpUw+Pgl0xNE3RkfyOBAChzO8lzBgTIelxSedJKpD0pTFmtrV2vb+zAM4sXChNmiR5vVJlpRQfr4P3PaDZd7wgSZo6nKlIAAh1Ln7VHiEpx1q71VpbI+k1SZMd5ADcKC/3FbDycl8Bk6TKSr3XcZDK66QhaYnqlZroNiMAoNm5KGFpkvKP+Lqg4TEgPMyY4RsBO8rrA8+TJF1dW+DvRAAAB1yUsGOtPGm/8SJjphljsowxWcXFxX6IBfhJdvZXI2ANclu115KugxRXU6VLdzMzDwDhwEUJK5B05AUvnSQVHv0ia+3T1tpMa21mSkqK38IBzS4jQ4qP/9pDh0bBJm1ZosSM7i5SAQD8zEUJ+1JShjGmuzEmWtI1kmY7yAG4MXWq5Pnqn1698WjmwHMlSVdv/MT3PAAg5Pm9hFlr6yTdJel9SRskvW6tXefvHIAziYnS3Lm+j/Hx+rT7GSpKbKPu+3dqxDN/lhISXCcEAPiBk3XCrLVzJc118bOBgDBunFRYKM2YoRnZvn+GU64YIzO+n+NgAAB/YTVIwJWEBO2deoM+8qT4Nuse3cN1IgCAH1HCAIfeXLFDdV6rs3u3U2pSrOs4AAA/ooQBDs1c5lsTbAqbdQNA2KGEAY5sLirXxl3lahkXpYl92rmOAwDwM0oY4Mjslb7l8SYNbM9m3QAQhnjnBxyw1urtVTskSZcNZtcuAAhHlDDAgRX5+5W/76BSk2I0ontr13EAAA5QwgAHDk1FXjqooyI8x9pOFQAQ6ihhgJ/V1Xv17mpfCZs8hKlIAAhXlDDAzxZv3as9FTXq0TZeA9KSXMcBADhCCQP87O1DU5GDO8oYpiIBIFxRwgA/qqqt17y1uyRJlw3p6DgNAMAlShjgR/M37lZFdZ0GprVUz5QE13EAAA5RwgA/OjQVOZlRMAAIe5QwwE/Kqmr18abdMka6ZBAlDADCHSUM8JP31+5STZ1XI7u3VvuWsa7jAAAco4QBfjJ7FWuDAQC+QgkD/GB3eZU+z9mjqAijiwa0dx0HABAAKGGAH8xZvVNeK53VK0WtWkS7jgMACACUMMAPDk1FXsZUJACgASUMaGZ5ew9oRd5+tYiO0Ll927mOAwAIEJQwoJnNXrVDknR+v1S1iI50nAYAECgoYUAzstYeXqCVbYoAAEcy1lrXGU7IGFMuaZPrHGGkraQ9rkOEEY63f3G8/Y9j7l8cb//qba1NPJVvDJa5kU3W2kzXIcKFMSaL4+0/HG//4nj7H8fcvzje/mWMyTrV72U6EgAAwAFKGAAAgAPBUsKedh0gzHC8/Yvj7V8cb//jmPsXx9u/Tvl4B8WF+QAAAKEmWEbCAAAAQkpAljBjzBRjzDpjjNcYc9w7PIwxFxpjNhljcowxD/gzYygxxrQ2xnxojMlu+Jh8nNdtN8asMcasPJ27QcLVic5X4/O3hudXG2POcJEzVDTieE8wxpQ2nM8rjTG/dpEzVBhjnjPG7DbGrD3O85zfTagRx5vzuwkZYzobY+YbYzY09JMfHeM1J32OB2QJk7RW0nckfXq8FxhjIiQ9LukiSf0kXWuM6eefeCHnAUn/sdZmSPpPw9fHc7a1dgi3P5+cRp6vF0nKaPgzTdKTfg0ZQk7i/eGzhvN5iLX2N34NGXqel3ThtzzP+d20nte3H2+J87sp1Un6qbW2r6RRku5sivfwgCxh1toN1toTLc46QlKOtXartbZG0muSJjd/upA0WdILDZ+/IOlyh1lCVWPO18mSXrQ+SyS1MsZ08HfQEMH7g59Zaz+VtO9bXsL53YQacbzRhKy1O621yxs+L5e0QVLaUS876XM8IEtYI6VJyj/i6wJ984CgcVKttTsl34km6Xi7TFtJHxhjlhljpvktXWhozPnKOd10GnssRxtjVhlj3jPG9PdPtLDF+e1/nN/NwBjTTdJQSUuPeuqkz3FnK+YbYz6S1P4YT/3CWvt2Y/6KYzzGrZ7H8W3H+yT+mrHW2kJjTDtJHxpjNjb8NoYTa8z5yjnddBpzLJdL6mqtrTDGTJL0lnzTCGgenN/+xfndDIwxCZJmSbrXWlt29NPH+JZvPcedlTBr7bmn+VcUSOp8xNedJBWe5t8Zsr7teBtjiowxHay1OxuGTncf5+8obPi42xjzpnxTPpSwxmnM+co53XROeCyPfAO11s41xjxhjGlrrWXPvebB+e1HnN9NzxgTJV8Bm26tfeMYLznpczyYpyO/lJRhjOlujImWdI2k2Y4zBavZkm5q+PwmSd8YiTTGxBtjEg99Lul8+W6gQOM05nydLem7DXfYjJJUemiaGCfthMfbGNPeGGMaPh8h3/vhXr8nDR+c337E+d20Go7ls5I2WGsfOc7LTvocD8gNvI0xV0h6TFKKpDnGmJXW2guMMR0lPWOtnWStrTPG3CXpfUkRkp6z1q5zGDuYPSTpdWPMLZLyJE2RpCOPt6RUSW82/JuOlPSKtXaeo7xB53jnqzHm9obn/yFprqRJknIkHZD0PVd5g10jj/dVkn5ojKmTdFDSNZbVq0+ZMeZVSRMktTXGFEj6b0lREud3c2jE8eb8blpjJd0oaY0xZmXDYw9K6iKd+jnOivkAAAAOBPN0JAAAQNCihAEAADhACQMAAHCAEgYAAOAAJQwAAMABShgAAIADlDAAAAAHKGEAwoIxZrgxZrUxJrZhB4h1xpgBrnMBCF8s1gogbBhj/p+kWElxkgqstb93HAlAGKOEAQgbDftIfimpStIYa22940gAwhjTkQDCSWtJCZIS5RsRAwBnGAkDEDaMMbMlvSapu6QO1tq7HEcCEMYiXQcAAH8wxnxXUp219hVjTISkRcaYidbaj11nAxCeGAkDAABwgGvCAAAAHKCEAQAAOEAJAwAAcIASBgAA4AAlDAAAwAFKGAAAgAOUMAAAAAcoYQAAAA78f/lvrVq4UvfLAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig = plot_data(X2, y, xlabel='x', ylabel='y')\n", + "theta_start = np.matrix([0, 0, 0]).reshape(3, 1)\n", + "theta, _ = gradient_descent(cost, gradient, theta_start, X2, y, eps=0.000001)\n", + "plot_fun(fig, polynomial_regression(theta), X1)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "Ten model jest odpowiednio dopasowany." + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[]" + ] + }, + "execution_count": 33, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAFoCAYAAAAfEiweAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1eH///fJvhMgCzsBEkABFUFEBeqCG221tSK471pbq1Zr66efbp/+PvXbRWu1WvtRUNGqgFatVeuKG4LsyCokQIAQyAYkkz0zc35/TLBUWRLIzJnl9Xw8fCSZmcy8vY/L5c2595xrrLUCAABAaMW5DgAAABCLKGEAAAAOUMIAAAAcoIQBAAA4QAkDAABwgBIGAADgQNBKmDHmCWNMpTFmzX6P9TDGvGOMKW7/2j1Ynw8AABDOgjkS9pSk87702D2S3rPWFkl6r/1nAACAmGOCuVirMaZA0mvW2pHtP2+QdLq1dqcxprekD6y1w4IWAAAAIEyF+pqwfGvtTklq/5oX4s8HAAAICwmuAxyMMeYmSTdJUnp6+pjhw4c7TgQACFdev9WGXR75rdWgnHRlJIftX2+IMsuWLau21uYeye+Gei+tMMb03u90ZOXBXmitfUzSY5I0duxYu3Tp0lBlBABEmP9+ebX2Ltqms4bnaeY1J7mOgxhijNl6pL8b6tORr0q6uv37qyX9I8SfDwCIMiWV9Zq9ZLvijHTP+Zw1QeQI5hIVz0taKGmYMabMGHO9pN9KOtsYUyzp7PafAQA4Yr/91+fy+a2mjxugovxM13GADgva6Uhr7aUHeeqsYH0mACC2fLq5Ru+ur1BaUrzumFzkOg7QKayYDwCISH6/1b1vrJck3TxpiPIyUxwnAjqHEgYAiEj/XFWuVWW1ystM1o2TBrmOA3QaJQwAEHGa23z6/ZsbJEl3nTNUaUksSYHIQwkDAEScpxeWasfeJg3Lz9TFY/q7jgMcEUoYACCi7G1s1cPzSiRJ90wZrvg44zgRcGQoYQCAiPLneSWqa/ZqQmGOTh96RAuVA2GBEgYAiBhbaxr09MJSGSP915ThMoZRMEQuShgAIGL8/q0NavNZfXt0X43o0811HOCoUMIAABFh+bY9en3VTiUnxOlH5wxzHQc4apQwAEDYs9bq3tcDC7NeP2GQ+mSnOk4EHD1KGAAg7L21tkJLt+5Rj/Qkfff0Ia7jAF2CEgYACGsLSqr1q1fXSpLumFykrJREx4mArsESwwCAsLS3sVW/eX29XlhWJkk6oX+2Lh03wHEqoOtQwgAAYcVaq3+u2qlf/3OtqutblRQfpx+cWaibvzZEifGcwEH0oIQBAMLGjr1N+vkrazTv80pJ0rhBPfT/LhqlIbkZjpMBXY8SBgBwzue3enphqf7w1gY1tvqUmZKgn045RtPG9lcctyVClKKEAQCc+nxXne75+2qt3L5XknT+yF76nwtGKC8rxXEyILgoYQAAJ5rbfHp4Xon++uEmef1W+VnJ+v8uHKlzRvRyHQ0ICUoYACDkPt1co5++tFqbqxskSVeMH6Afnzec5ScQUyhhAIDg8HikOXOk4mKpqEiaNk21CSn67b/W6/nF2yVJhXkZ+u1FozS2oIfjsEDoUcIAAF1v/nxpyhTJ75caGmTT0/Wvh2frlxfeqaoWq8R4o++fUahbTh+i5IR412kBJyhhAICu5fEECpjHI0naldFTPz/7u3pn6ClSi9WYfln67dQTVJSf6Tgo4BYlDADQtebMkd9vVZ6Vq3cLT9Z9k65SfXKaMloa9ZOFz+nyG76huPyJrlMCzlHCAABHrMXr05bqBm2qbNCmqnqVVNZrU3GGNt/0lJqS/r3ExNkbF+rX7/5VvT010hnDHSYGwgclDABwWLWNbSqp8mhTZYNKquq1qbJeJVX12r67UX77pRebTClJyq3frcKa7bpq+es6b+MCGUlKT5cKCx38HwDhhxIGALHoADMXbUaGKj0t2rDLo5L2krWpsl6bqupVXd96wLeJM9KgnHQNyU3XkLwMDcnNUGFGnIZMHKtu1bsO8Atx0rRpQf6fAyIDJQwAYoz9+GNVX3yZNnbvq40Zedq4tlXFHz+mjQOGq67twL+TmhivIXnpgZKVm6EheRkqzMvQwJ5pB57d+PIL/zE7UunpgQL2xhtSBveBBCRKGABEtd0NrdpY4fn3f+W1Kt6wQ3uu/etXX9wmdUtJ0LBeWSrMD5StwrxA4eqdldK5ezhOmCCVlwdG20pKAqcgp02jgAH7oYQBQBSw1mr1jlqtKqtVcYVHGyvqVVzpOfBpxNRMZbY0qKh6m4ZVbVVR9TYNrd6qoY3Vyr33f2RuuL5rQmVkSNd30XsBUYgSBgARzFqrj4qr9dB7xVq2dc9Xnk9PildhfqaG5mVoWK9MFb02V0MfvU+9PDU64LjWppKgZwYQQAkDgAhkrdX7Gyr14Hsl+mz7XklSdlqizhyep6H5mRqWn6mi/Az16Zb6n6cRN+RI/uYDvykzF4GQooQBQASx1uqddRV6aF6x1uyokyT1TE/SjZMG64rxA5WRfJjD+rRp0p13Hvg5Zi4CIUUJA4AI4Pdbvbl2l/48r0TrdwbKV05Gsr77tcG67OQBSkvq4OE8MzMwQ5GZi4BzlDAACGM+v9Xrq3fq4XnF2lhRL0nKz0rWd782RJeOG6CUxCO4+TUzF4GwQAkDgDDk9fn1z1Xl+vO8Em2uapAk9emWolvOKNTUMf2OrHztj5mLgHOUMAAII20+v15ZsUOPvF+i0ppGSVK/7qn6/hmF+s6J/ZSUEOc4IYCuQgkDgDDQ6vXrpeVleuSDEm3f3SRJGtgzTd8/o1DfHt1XifGULyDaUMIAwKEWr08vLC3Tox9s0o69gfI1OCddt55ZqAuO76MEyhcQtShhAOBIXXObrntyiZa2L7JamJehH5xZqG8c10fxnblFEICIRAkDAAdqG9t01ZOL9dn2verdLUU/+/qxOn9kr87dnxFARKOEAUCI7Wlo1RUzF2lteZ36dU/V8zeOV/8eaa5jAQgxShgAhFB1fYuumLFIn+/yaGDPND1/43j1yU51HQuAA5QwAAiRyrpmXT5jkYor6zU4N13P3zhe+VkprmMBcIQSBgAhsKu2WZc9/qk2VzdoaH6Gnr1hvHIzk13HAuAQJQwAgqxsT6Mue3yRtu1u1DG9s/S368epZwYFDIh1lDAACKJtNY269PFPtWNvk0b17aZnrh+n7LQk17EAhAFKGAAEyZbqBl32+KfaWdus0QOy9dS149QtNdF1LABhghIGAEFQUunRZY8vUqWnRScVdNcT15ykzBQKGIB/c3I/DGPMD40xa40xa4wxzxtjmB4EIGps2OXR9Mc+VaWnRacM7qmnrh1HAQPwFSEvYcaYvpJukzTWWjtSUryk6aHOAQDBsLa8VtMfW6jq+lZNLMrRE9ecpPRkTjoA+CpXR4YESanGmDZJaZLKHeUAgC6zqmyvrpy5WLVNbTpzeJ7+cvmJSkmMdx0LQJgK+UiYtXaHpPskbZO0U1KttfbtL7/OGHOTMWapMWZpVVVVqGMCQKcs27pHlz++SLVNbTrn2Hz99YoxFDAAh+TidGR3SRdKGiSpj6R0Y8wVX36dtfYxa+1Ya+3Y3NzcUMcEgA5bvGW3rpq5SJ4Wr74+qrceufxEJSU4ueQWQARxcZSYLGmLtbbKWtsm6SVJpzrIAQBHbUFJta5+YrEaWn361gl99OD0E5QYTwEDcHgujhTbJI03xqQZY4yksyStd5ADAI7KRxurdO1TS9TU5tPFY/rp/ktOUAIFDEAHubgmbJGkFyUtl7S6PcNjoc4BAEdj3ucVumHWUrV4/bp03AD9/jvHKT7OuI4FIII4mR1prf2lpF+6+GwAOFrryuv03WeWq9Xn19WnDNSvLhihwMA+AHQc4+YA0AmtXr9+9MJnavX5NW1sfwoYgCNGCQOATnjk/RKt21mnAT3S9ItvHksBA3DEKGEA0EFrdtTqkfdLJEm/v/g4VsIHcFQoYQDQAftOQ3r9VtecWqDxg3u6jgQgwlHCAKAD/jyvWJ/v8qigZ5p+fN4w13EARAFKGAAcxqqyvfrLB5tkjPSHqccrLYnTkACOHiUMAA6hxevTXXM/k89vdd1pg3RSQQ/XkQBECUoYABzCn94tVnFlvQbnpOtH53AaEkDXoYQBwEGs3L5X//fhJsW1n4ZMTYp3HQlAFKGEAcABNLf5dNfclfJb6caJgzVmYHfXkQBEGUoYABzAA+9s1KaqBg3JTdcPzx7qOg6AKEQJA4AvWbZ1tx77eLPijHTf1OOVkshpSABdjxIGAPtpbvPp7hdWyVrp5q8N0egBnIYEEBwsdgMAHo80Z45UXKz7up+ozXszNDQ/Q3dMLnKdDEAUo4QBiG3z50tTpkh+v5ZkD9TMyyco3vp0X5FVcgKnIQEED6cjAcQujydQwDweNbZ6dfeUO2RNnG759AUdd9kFUn2964QAohglDEDsmjNH8vslSb+fdLVKe/TR8Mot+sGC2YHH58xxHBBANON0JIDYVVwsNTTo0/4j9dTYC5Tg8+q+N/6kZJ9XavBKJSWuEwKIYoyEAYhdRUVqyO6hH59/uyTpe5/O1ciKTYHn0tOlwkKH4QBEO0bCAMSuadP0u5dXaVv33jqmYrNuXTD338/FxUnTprnLBiDqUcIAxKwFFS16etS5SvB5df+8vyrJ7w2MgMXFSW+8IWVkuI4IIIpRwgDEpPoWr+5+cZUk6bYzC3Xs8DsD14AVFgZGwChgAIKMEgYgJt37xnrt2NukkX2zdMs5x0jxI1xHAhBjuDAfQMz5uLhKzy3apsR4o/umHq/EeA6FAEKPIw+AmOJpbtNP2k9D3jF5qIb3ynKcCECsooQBiCm/eX29ymubdVy/brp50mDXcQDEMEoYgJjx0cYqzV6yXUnxcbp/6vFK4DQkAIc4AgGICX6/1b1vrJck3T65SEX5mY4TAYh1lDAAMeH11Tv1+S6P+nRL0Q0TB7mOAwCUMADRz+vz64F3N0qSbjurSMkJ8Y4TAQAlDEAMeGVluTZXNWhgzzR9Z0w/13EAQBIlDECUa/X69af2UbA7JhexJhiAsMHRCEBUm7t0u8r2NKkwL0MXHN/XdRwA+AIlDEDUam7z6c/ziiVJd549VPFxxnEiAPg3ShiAqPXsom2qqGvRsb2zdN6IXq7jAMB/oIQBiEoNLV49+kGJJOlH5w5VHKNgAMIMJQxAVJq1sFTV9a0aPSBbZwzLcx0HAL6CEgYg6tQ2ten/PtwsSfrROcNkDKNgAMIPJQxA1Jk5f4tqm9o0fnAPnTqkp+s4AHBAlDAAUWV3Q6uemL9FknQXo2AAwhglDEBU+b+PNqm+xauvDc3VSQU9XMcBgIOihAGIGpWeZs1aUCopcC0YAIQzShiAqPGX9zepuc2vc0fka1S/bq7jAMAhUcIARIUde5v03KJtMkb64dlDXccBgMOihAGICg/PK1arz69vHtdHw3tluY4DAIdFCQMQ8UqrGzR3aZnijHTH5CLXcQCgQyhhACLeQ+8Vy+e3+s6J/TQ4N8N1HADoECclzBiTbYx50RjzuTFmvTHmFBc5AES+4gqPXl65Q4nxRredxSgYgMiR4OhzH5T0prX2YmNMkqQ0RzkARLg/vVssa6XpJw1Q/x4cSgBEjpCXMGNMlqRJkq6RJGttq6TWUOcAEPnWltfq9dU7lZwQp1vPLHQdBwA6xcXpyMGSqiQ9aYxZYYyZYYxJd5ADQIT749sbJUlXjh+o/KwUx2kAoHNclLAESSdKetRaO1pSg6R7vvwiY8xNxpilxpilVVVVoc4IIMwt37ZH731eqbSkeH339CGu4wBAp7koYWWSyqy1i9p/flGBUvYfrLWPWWvHWmvH5ubmhjQggPC3bxTs2tMKlJOR7DgNAHReyEuYtXaXpO3GmH03djtL0rpQ5wAQuRZuqtH8kmplpiTopomMggGITK5mR/5A0rPtMyM3S7rWUQ4AEcZaqz++s0GSdOPEweqWlug4EQAcGSclzFq7UtJYF58NILJ9VFytJaV71D0tUddNGOQ6DgAcMVbMBxAxrLW6/+3AKNgtpw9RRrKrwXwAOHqUMAAR4+11FVpVVqvczGRdOb7AdRwAOCqUMAARwe+3X8yIvPWMQqUmxTtOBABHhxIGICK8tnqnNlR41KdbiqaP6+86DgAcNUoYgLDn9fn1p3cCo2C3nVWk5ARGwQBEPkoYgLD38ood2lzdoIKeafrOmH6u4wBAl6CEAQhrXp9fD75XLEm6Y/JQJcZz2AIQHTiaAQhr/1qzS2V7mjQ4J13fPL6P6zgA0GUoYQDClrVWMz7eLEm6fuIgxccZx4kAoOtQwgCEraVb9+izslp1T0vURaO5FgxAdKGEAQhb+0bBrhw/kHXBAEQdShiAsFRa3aC311UoKT5OV5wy0HUcAOhylDAAYenJT7bIWunCE/ooLzPFdRwA6HKUMABhp7axTXOXlkmSbpg42HEaAAgOShiAsPPc4m1qavNpYlGOhvXKdB0HAIKCEgYgrLR6/XpqwRZJjIIBiG6UMABh5fXV5aqoa9HQ/AxNKspxHQcAgoYSBiBsBBZnbR8FmzBYxrA4K4DoRQkDEDYWbq7R2vI65WQk6YITuEURgOhGCQMQNma2j4JdOb5AKYkszgogulHCAISFTVX1eu/zSiUnxOmK8QNcxwGAoKOEAQgLM+cHRsEuOrGfemYkO04DAMFHCQPg3O6GVv19WWBx1usnFLgNAwAhQgkD4Nyzn25Vi9evM4blqjCPxVkBxAZKGACnmtt8mrVwqyTpRhZnBRBDElwHABBjPB5pzhypuFgqKtKrw7+m6voWHdM7S6cM6ek6HQCEDCUMQOjMny9NmSL5/VJDg2x6umZe+gep5wDdMGEQi7MCiCmUMACh4fEECpjH88VDH+cO1YaeA5TXsEffLMxyGA4AQo9rwgCExpw5gRGw/cw46VuSpKtXvamkF19wkQoAnKGEAQiN4mKpoeGLHzfkDNRHg8cotbVZly/+h1RS4jAcAIQeJQxAaBQVSenpX/z4xNgLJElTV7+r7HgrFRa6SgYATlDCAITGtGlSXOCQU5WWrZdHnClj/bp22auBx6dNcxwQAEKLEgYgNDIzpTfekDIz9czJ31JrQqImb1mmQV5P4PGMDNcJASCkKGEAQmfCBDVv3a6/nXKRJOmGSYOl8nJpwgTHwQAg9FiiAkBIvbSxVrt9cTquXzeN+/4UibXBAMQoRsIAhIzfbzVz/mZJ0vUszgogxh22hBljbjXGdA9FGADR7cONVdpU1aDe3VI0ZVRv13EAwKmOjIT1krTEGDPXGHOe4Z+uAI7Q4x8HRsGuPa1AifEMxAOIbYc9ClprfyapSNJMSddIKjbG3GuMGRLkbACiyNryWi3YVKP0pHhNO2mA6zgA4FyH/ilqrbWSdrX/55XUXdKLxpjfBzEbgCgyc/4WSdIlJ/VXt9REx2kAwL3Dzo40xtwm6WpJ1ZJmSLrbWttmjImTVCzpx8GNCCDSVdQ165+flSvOSNedNsh1HAAICx1ZoiJH0kXW2q37P2it9RtjvhGcWACiyawFpWrzWU0Z1Uv9e6S5jgMAYeGwJcxa+4tDPLe+a+MAiDaNrV49u2ibJOn6CYMdpwGA8MH0JABB9eKyMtU2tWn0gGyNGchqNwCwDyUMQND4/FZPtF+Qf+NERsEAYH+UMABB8976CpXWNKpf91Sdc2y+6zgAEFYoYQCCZkb7KNi1pw1SAouzAsB/4KgIIChWle3V4i27lZmcoEvG9nMdBwDCjrMSZoyJN8asMMa85ioDgOB58pNSSdL0cf2VmcLirADwZS5Hwm6XxBIXQBSq8rTotVXlMka66pQC13EAICw5KWHGmH6Svq7ACvwAoszzi7epzWd11vB8FmcFgINwNRL2JwVud+Q/2AuMMTcZY5YaY5ZWVVWFLhmAo9Lm8+vZRYEbbFxzaoHbMAAQxkJewtpvdVRprV12qNdZax+z1o611o7Nzc0NUToAR+vNNbtUUdeiwrwMnVbY03UcAAhbLkbCTpN0gTGmVNJsSWcaY/7mIAeAIJi1oFSSdPWpBTLGuA0DAGEs5CXMWvtf1tp+1toCSdMlzbPWXhHqHAC63podtVq6dY8yUxJ00ei+ruMAQFhjnTAAXWbfKNjUMf2VnpzgNgwAhDmnR0lr7QeSPnCZAUDXqKlv0T8+27csxUDXcQAg7DESBqBLzF6yXa1ev04fmquCnHTXcQAg7FHCABw1r8+vZz8NLEtxNctSAECHUMIAHLV31lWovLZZg3LSNamIJWUAoCMoYQCO2lPtF+RfdcpAxcWxLAUAdAQlDMBRWb+zTou27FZ6UrwuHtPPdRwAiBiUMABH5emFpZKk74zpp8yURKdZACCSUMIAHLG9ja16ecUOSdJVpxS4DQMAEYYSBuCIzV26Xc1tfk0sylFhXobrOAAQUShhAI6Iz2/19ML2ZSkYBQOATqOEATgi8z6vVNmeJg3okaYzhue5jgMAEYcSBuCIzNpvWYp4lqUAgE6jhAHotJJKj+aXVCs1MV5Tx/Z3HQcAIhIlDECnzVoQuBbs2yf2VbdUlqUAgCNBCQPQKXXNbfr78jJJXJAPAEeDEgagU15YWqbGVp9OGdxTw3pluo4DABGLEgagw/x+q2cWlkqSrj61wGUUAIh4lDAAHfbhxiqV1jSqb3aqJh/DshQAcDQoYQA67Kn2ZSmuGD9QCfEcPgDgaHAUBdAhm6vq9eHGKiUnxGn6SSxLAQBHixIGoEP23aLowhP6qHt6kuM0ABD5KGEADqu+xasXl7UvS8EF+QDQJShhAA7rpeVlqm/xalxBD43o0811HACICpQwAIdkrf3iPpGMggFA16GEATik+SXV2lTVoF5ZKTpnRL7rOAAQNShhAA5p1hfLUgxQIstSAECX4YgK4KC21TTqvc8rlRQfp+njBriOAwBRhRIG4KCe+bRU1krfOL63cjKSXccBgKhCCQNwQI2tXs1Zsl2SdA0X5ANAl6OEATigV1aUq67Zq9EDsnVcv2zXcQAg6lDCAHyFtVZPLdgiiVEwAAgWShiAr1i4uUYbK+qVm5ms80f2dh0HAKISJQzAV+xbluKycQOUlMBhAgCCgaMrgP9QtqdR76yrUEKc0eUnsywFAAQLJQzAf/jbp9vkt9KUUb2Vl5XiOg4ARC1KGIAvNLZ6NXvJNkncJxIAgo0SBuALf1++Q3sb23RC/2ydOIBlKQAgmChhACRJfr/VE/MDy1LcOHGwjDGOEwFAdKOEAZAkvbu+QluqG9Q3O1Xnjsh3HQcAoh4lDIAkaUb7KNh1EwYpIZ5DAwAEG0daAFpVtleLt+xWZnKCLhnbz3UcAIgJlDAAmvFxYBTs0pMHKDMl0XEaAIgNlDAgxu3Y26TXV+9UfJzhPpEAEEKUMCDGzVpQKp/f6uujeqtPdqrrOAAQMyhhQAzzNLfp+UWBxVlvmDjIcRoAiC2UMCCGzVmyXZ4Wr8YN6qHj+rE4KwCEEiUMiFFen19PflIqKbA4KwAgtChhQIx6c+0u7djbpEE56TpreJ7rOAAQc0Jewowx/Y0x7xtj1htj1hpjbg91BiDWWWv1+Mf/Xpw1Lo5bFAFAqCU4+EyvpLustcuNMZmSlhlj3rHWrnOQBYhJy7bu0Wfb9yo7LVEXn8jirADgQshHwqy1O621y9u/90haL6lvqHMAsWzf4qxXnDxQqUnxjtMAQGxyek2YMaZA0mhJi1zmAGLJ1poGvbVul5Li43TVqQNdxwGAmOWshBljMiT9XdId1tq6Azx/kzFmqTFmaVVVVegDAlHqiflbZK10wQl9lJeZ4joOAMQsJyXMGJOoQAF71lr70oFeY619zFo71lo7Njc3N7QBgShV29imuUvLJLE4KwC4FvIL840xRtJMSeuttX8M9ecDsezZxVvV1ObTxEHZGv7aXKm4WCoqkqZNkzIzXccDgJjiYnbkaZKulLTaGLOy/bGfWmvfcJAFiBmtXr9mLSiVJN3w0E+krSulhgYpPV26807pjTekCRPchgSAGBLyEmatnS+JRYmAEHttVbkq6lo0tGa7Jq375N9PNDQEvk6ZIpWXSxkZbgICQIxhxXwgBuy/OOsNK1878L+C/H5pzpyQ5gKAWEYJA2LAwk01Wr+zTjm2VReueOvAL2pokEpKQhsMAGIYJQyIAY9/vFmSdFXPFiWnJB/4RenpUmFhCFMBQGyjhAFRrqTSo/c3VCk5IU5XXH2OFHeQP/ZxcYFZkgCAkKCEAVFu5vzAtWDfGdNPPfJ7BGZBZmYGRr6kwNfMzMDjXJQPACHjYokKACFSU9+ivy/fIUm6fkL74qwTJgRmQc6ZE7gGrLAwMAJGAQOAkKKEAVHsmU+3qtXr11nD8zQkd7+SlZEhXX+9u2AAAE5HAtGquc2nZxZulSTdMHGw4zQAgC+jhAFR6pUVO1TT0KoRfbI0fnAP13EAAF/C6cgjUFHXrPoWr7JTE9UtNVEJ8XRZhBe/32pG+wX5N04crMAtWwEA4YQS1kkbdnn0zYfnq9Xr/+KxzJQEdU9LUnZaorLTkpSdmqjuaYnqlpak7mmJX3o88LqslETFxfEXI4Ljw+IqlVTWq1dWir5+XG/XcQAAB0AJ66Q/vLVBrV6/eqYnyW+tapva5Gn2ytPs1bbdHX8fY6RuqYkampep+y85Xv17pAUvNGLOjPbFWa85rUCJjNQCQFiihHXCsq279e76CqUlxevNOyYpNzNZfr+Vp9mrPY2t2tvUpj2NraptDHzd09im2vave5vatLexVXvbn/M0e7W3sU2LS3frypmL9OItpyon4yArmQOdsK68Tp+U1CgtKV6XjhvgOg4A4CAoYR1krdXv3twgKbDeUm5moDDFxRl1S0tUt7TETr2f1+dXVX2Lbpi1VGvL63TNk4v1/I3jlZnSufcBvmzG/MAo2CVj+6tbKvsTAIQrzlN00Icbq7R4y25lpyXqxklHP90/IT5Ovbul6qlrx2lgzzSt2VGnm55epuY2XxekRayqqGvWPz8rV5yRrjttkOs4AIBDoIR1gN9v9Ye3AqNg3zt9iNPQR9kAABTlSURBVLK6cLQqNzNZz1x3snIzk7Vwc43umL1SPr/tsvdHbJm1oFRtPqtzR/TSgJ5cZwgA4YwS1gGvr96pteV16pWVoqtOKejy9x/QM01PXzdOmSkJenPtLv3slTWyliKGzmls9erZRdsksTgrAEQCSthhtPn8+uM7GyVJt08uUkpifFA+55jeWZp59UlKTojT84u36f63NwblcxC9XlxWptqmNo0ekK0xA7u7jgMAOAxK2GG8sLRMW6obNDgnXVPH9AvqZ40b1EOPXHai4uOMHn6/RE+0L7YJHI7PbzVzv8VZAQDhjxJ2CM1tPj34XmBE6s5zhoZkZfzJx+brtxeNkiT9+rV1emXFjqB/JiLfv9bs1NaaRvXvkapzR/RyHQcA0AGUsEOYtaBUFXUtGtk3S1NGhm7V8alj++unU4ZLkn70wmf6YENlyD4bkce73ynzmycNUTx3YgCAiEAJO4japjb95YNNkqS7zx0e8lsM3TRpiG6eNFhev9Utf1uuZVv3hPTzETleWr5Dm6saNLBnmqad1N91HABAB1HCDuLxjzartqlN4wf30KSiHCcZ7jl/uC4e009NbT5d99QSbazwOMmB8NXc5tOf3m0/ZX72UG5RBAARhCP2AVR6mr+4yPnH5w2XMW5O7xhj9NuLRmnyMfmqbWrTVTMXa8feJidZEJ6eW7RN5bXNGt4rU988ro/rOACATqCEHcAj80rU1ObT2cfm68QBbqf6J8TH6eHLRuukgu7aVdesK2cuUk19i9NMCA/1LV498n6JJOlH5wwL+SlzAMDRoYR9yfbdjXpu8TYZE/iLLRykJMZrxtUnaXivTG2uatC1Ty1RfYvXdSw49uT8LappaNXoAdk665g813EAAJ1ECfuSB97ZqDaf1bdH99WwXpmu43yhW2qinr5unPr3SNWqslp995llavFyn8lYtaehVY99FLhR993nDnN2yhwAcOQoYfv5fFedXl65Q4nxRj+cPNR1nK/Iy0rRM9edrJyMJM0vqdZdcz/jPpMx6q8fbZKnxauJRTk6dYibiSMAgKNDCdvPfW9tkLXS5ScPVP8e4Xnz44KcdD117ThlJCfotVU79atX13KfyRhTUdesWQtKJYXPKXMAQOdRwtot27pb766vVFpSvL5/RqHrOIc0sm83PX7VWCUlxOmZT7fqwfeKXUdCCP15XrGa2/w6b0QvHd8/23UcAMARooRJstbqd29ukCRdP2GQcjOTHSc6vFOG9NRD00crzkh/erdYzywsdR0JIbCtplGzF29XnJHuOif8TpkDADqOEibpw41VWrxlt7LTEnXjpMi5+fF5I3vp3m8H7jP5i1fX6l+rdzpOhGB74N2N8vqtvj26n4ryw2fiCACg82K+hPn9Vr9vHwX73ulDlJWS6DhR50wfN0B3nztM1ko/fnGVtu9udB0JQbJhl0evtE8cuWNykes4AICjFPMl7PXVO7VuZ516ZaXoqlMKXMc5It87fYjOHZEvT4tXd8xZKa/P7zoSguC+twMTRy4bNyBsJ44AADoupktYm8+v+98OjILdPrlIKYnxjhMdmcDtjY5Tflaylm3do0fe3+Q6ErrYim179M66CqUmxuv7Z4b3xBEAQMfEdAl7YWmZSmsaNTgnXVPH9HMd56h0T0/SHy85QcZID80r1rKte1xHQhf6w1uBfyxce1qB8jJTHKcBAHSFmC1hTa0+PfjeRknSnecMVUJ85G+K0wpzdNPEwfL5re6Ys0Ke5jbXkdAFPimp1oJNNcpKSdDNk4a4jgMA6CKR3zyO0KyFpaqoa9HIvlmaMrK36zhd5s5zhmpEnyxt392kX7661nUcHCVrrX7fPgp289eGqFtaZE0cAQAcXEyWsNqmNj36QeC6qR+fO1xxcdFz373khHg9OH20UhLj9NLyHXr1s3LXkXAU3l5Xoc+271VORrKuPa3AdRwAQBeKyRL22EebVNvUpvGDe2hiUfTdd68wL0M//8axkqT/fnm1yvawbEUk8vntFxNHfnBmodKSEhwnAgB0pZgrYZWeZj0xv1SS9OPzhsuY6BkF299l4wbo7GPz5Wn26s453Og7Ev1j5Q5trKhX3+xUTR/X33UcAEAXi7kS9vC8EjW1+XT2sfk6cUB313GCxhij333nOOVmJmtx6W49+kGJ60johFavXw+8G5g4csfkIiUnRObyKQCAg4upEratplHPL94mY6S7zx3mOk7Q9UhP0v1Tj5ckPfBusVZu3+s4ETpqzpJt2r67SYV5GbroxMhePgUAcGAxVcIeeHej2nxW3x7dV0Nj5L57k4bm6voJg+TzW90+e4UaWryuI+Ewmlp9emheYOTyrrOHKj6KJo4AAP4tZkrY57vqvrjv3g8nD3UdJ6R+fN4wDe+Vqa01jfoVy1aEvacWlKrK06JRfbvpvJG9XMcBAARJzJSwWQtKZa10+ckDY+6+e8kJ8frzpaOVnBCnF5aV6fVVO11HwkHUNrXprx8Glk+5+9xhUTtxBAAQQyVsxbbA9VDfPD56FmbtjKL8TP3s68dIkv7rpVUq39vkOBEO5PGPNkf18ikAgH+LiRLW3OZTcWW94ox0bO9uruM4c8X4gTpreJ7qmr364ZyVLFsRZqo8LXriky2SpLvPjd7lUwAAAU5KmDHmPGPMBmNMiTHmnmB/3vqddfL5rQrzMpSaFLtT/Y0x+t3FxyknI1mLtuzWYx9tdh0ptnk80owZ0k9+Is2YoUfeXqfGVp8mH5OnMQOjd/kUAEBAyJfgNsbES3pE0tmSyiQtMca8aq1dF6zPXFNeJ0ka2Td2R8H2yclI1n1Tj9M1Ty7R/W9v0GmFPXVcv2zXsWLP/PnSlCmS3y81NKis10A9d8WfZOITddc50b98CgDAzUjYOEkl1trN1tpWSbMlXRjMD1xTVitJGtmHEiZJpw/L0zWnFsjrt7p99ko1trJsRUh5PIEC5vFIDQ2SpAdHf0ut8Ym6YON8HZMZE1cJAEDMc3G07ytp+34/l7U/FjSrdwRK2Kh+lLB97jl/uIblZ2pLdYN+/c+gDULiQObMCYyAtSvp0U9/H3mmEnxe/XDR3MDzAICo56KEHehq469cIW6MuckYs9QYs7SqquqIP6y5zaeNFR4ZIx3bO+uI3yfapCTG66FLRyspIU6zl2zXm2tYtiJkiou/GAGTpAcmXi5/XLwuWfWOCso3SyXcYgoAYoGLElYmaf+7EfeTVP7lF1lrH7PWjrXWjs3NzT3iD9tY4ZHXbzUkN0PpySG/BC6sDeuVqZ+eP1ySdM9Lq7WrttlxohhRVCSlp0uS1uQP0evDJyrJ26rbFswOPF5Y6DggACAUXJSwJZKKjDGDjDFJkqZLejVYH7bvVOTIPoyCHcjVpxboa0NztbexTXe9sFJ+lq0IvmnTpLg4+Uyc/uesmyRJVy9/Tb3qa6S4uMDzAICoF/ISZq31SrpV0luS1kuaa60N2r101uwrYcyMPCBjjO6berx6pifpk5IazZjPshVBl5kpvfGGHj79Si3pP0J5nhp9b9XrXzyujAzXCQEAIeBkGpa19g1r7VBr7RBr7W+C+VlfXJRPCTuo3Mxk/WHqcZKkP7y14YviiuBZ2u9YPXjyVBlZPZBZru6/+41UXi5NmOA6GgAgRKJ6Lnyr168NuzySpBGUsEM6c3i+rjploNp8VrfNXiFPc5vrSFGrtqlNt89eKb+Vbv5aoU777T3S9dczAgYAMSaqS9jGCo/afFaDc9KVwUX5h/XTKcdoaH6GNlc1cFujILHW6qcvrdaOvU06vn+27jpnqOtIAABHorqEreZ6sE5JSYzXY1eOVbfURL27vlL3vb3BdaSoM3fpdr2+eqcykhP00PQTlBgf1X8EAQCHENV/A6zherBOK8hJ16OXn6j4OKNHP9ikV1bscB0papRU1utXrwYWxv3fb43UwJ7pjhMBAFyKiRLGSFjnnFqYo19981hJ0o//vkortu1xnCjyNbf59IPnV6ipzaeLRvfVt0YH9SYRAIAIELUlrM3n1/ovLspnjbDOuvKUAl1+8gC1ev266Zll2lnb5DpSRPvdm59r/c46DeyZpl9/a6TrOACAMBC1JWxjhUetXr8KeqYpKyXRdZyI9KsLRmj84B6q8rTopqeXqanV5zpSRJr3eYWe/KRUCXFGD00fzSQRAICkKC5ha3fUSeJU5NFIjI/To5eP0YAeaVq9o1Z3v/iZrGXGZGdU1jXrRy+skiT96NxhOr5/tuNEAIBwEbUljJmRXaN7epJmXD1WGckJem3VTj08j5tLd5Tfb3Xn3M+0u6FVEwpzdNPEwa4jAQDCSNSXMGZGHr2h+Zl6cPoJMka6/52NenPNLteRIsLjH2/W/JJq9UhP0h8vOV5xccZ1JABAGInKEub1+bV+Z/vpyD6UsK5w1jH5+sl5wyVJd85dqXXldY4ThbfPtu/VH94KrLN2/9TjlZeV4jgRACDcRGUJK6mqV4vXrwE90tQtjYvyu8rNkwbrotF91djq041PL1V1fYvrSGGpvsWr22avkNdvde1pBTpjeJ7rSACAMBSVJWx12b7rwViaoisZY3TvRaN0Qv9s7djbpFv+tkwtXmZMftkvXlmjrTWNOqZ3lu45f7jrOACAMBWVJYxFWoMncGujMeqVlaIlpXv081fWMGNyPy+vKNNLK3YoNTFef750tJIT4l1HAgCEqagsYVyUH1x5WSl6/KqxSkmM09ylZXrik1LXkcLC1poG/ezlNZKkX37zWBXmZThOBAAIZ1FXwnx+q3VclB90o/p1031Tj5ck/eb1dfpwY5XjRG61+fy6bfZKNbT69PVRvTXtpP6uIwEAwlzUlbBNVfVqbvOrb3aquqcnuY4T1b5xXB/ddmah/Fa69bnl2lRV7zqSM398Z6M+275XfbNTde9Fo2QMy1EAAA4t6krYvovyORUZGndMHqrzRvSSp9mrG2YtVW1jm+tIIfdJSbX++uEmxRnpweknqFsqM3IBAIcXfSVs3/Vg/ShhoRAXZ/THacfrmN5Z2lLdoO8/t1xen991rJCpqW/RD+eslLXS7WcN1diCHq4jAQAiRNSVsLXlzIwMtbSkBD1+1Rj1TE/S/JJq/e/r611HCglrre5+cZUqPS0aV9BDt55Z6DoSACCCRFUJ8/mt1pbvuyifNcJCqV/3NP3flWOUGG/01IJSPb94m+tIQTdrQanmfV6pbqmJemD6CYrntkQAgE6IqhK2pbpeja0+9emWop4Zya7jxJyxBT30m2+PkiT9/JU1+nRzjeNEwbOuvE73vvG5JOl33xmlvtmpjhMBACJNVJWwNTvaR8E4FenMJWP764YJg+T1W93yt2VROWNyx94m3fr8crX6/Lp03ACdN7K360gAgAhkImG1c2OMR9IG1zliSI6katchYgjbO7TY3qHHNg8ttndoDbPWZh7JLyZ0dZIg2WCtHes6RKwwxixle4cO2zu02N6hxzYPLbZ3aBljlh7p70bV6UgAAIBIQQkDAABwIFJK2GOuA8QYtndosb1Di+0demzz0GJ7h9YRb++IuDAfAAAg2kTKSBgAAEBUCcsSZoyZaoxZa4zxG2MOOsPDGHOeMWaDMabEGHNPKDNGE2NMD2PMO8aY4vav3Q/yulJjzGpjzMqjmQ0Sqw63v5qAh9qfX2WMOdFFzmjRge19ujGmtn1/XmmM+YWLnNHCGPOEMabSGLPmIM+zf3ehDmxv9u8uZIzpb4x53xizvr2f3H6A13R6Hw/LEiZpjaSLJH10sBcYY+IlPSLpfEnHSrrUGHNsaOJFnXskvWetLZL0XvvPB3OGtfYEpj93Tgf31/MlFbX/d5OkR0MaMop04vjwcfv+fIK19tchDRl9npJ03iGeZ//uWk/p0NtbYv/uSl5Jd1lrj5E0XtL3u+IYHpYlzFq73lp7uMVZx0kqsdZutta2Spot6cLgp4tKF0qa1f79LEnfcpglWnVkf71Q0tM24FNJ2cYYluM/MhwfQsxa+5Gk3Yd4Cft3F+rA9kYXstbutNYub//eI2m9pL5felmn9/GwLGEd1FfS9v1+LtNXNwg6Jt9au1MK7GiS8g7yOivpbWPMMmPMTSFLFx06sr+yT3edjm7LU4wxnxlj/mWMGRGaaDGL/Tv02L+DwBhTIGm0pEVfeqrT+7izFfONMe9K6nWAp/7bWvuPjrzFAR5jqudBHGp7d+JtTrPWlhtj8iS9Y4z5vP1fYzi8juyv7NNdpyPbcrmkgdbaemPMFEmvKHAaAcHB/h1a7N9BYIzJkPR3SXdYa+u+/PQBfuWQ+7izEmatnXyUb1Emqf9+P/eTVH6U7xm1DrW9jTEVxpje1tqd7UOnlQd5j/L2r5XGmJcVOOVDCeuYjuyv7NNd57Dbcv8DqLX2DWPMX4wxOdZa7rkXHOzfIcT+3fWMMYkKFLBnrbUvHeAlnd7HI/l05BJJRcaYQcaYJEnTJb3qOFOkelXS1e3fXy3pKyORxph0Y0zmvu8lnaPABAp0TEf211clXdU+w2a8pNp9p4nRaYfd3saYXsYY0/79OAWOhzUhTxo72L9DiP27a7Vvy5mS1ltr/3iQl3V6Hw/LG3gbY74t6c+SciW9boxZaa091xjTR9IMa+0Ua63XGHOrpLckxUt6wlq71mHsSPZbSXONMddL2iZpqiTtv70l5Ut6uf3PdIKk56y1bzrKG3EOtr8aY77b/vxfJb0haYqkEkmNkq51lTfSdXB7XyzpFmOMV1KTpOmW1auPmDHmeUmnS8oxxpRJ+qWkRIn9Oxg6sL3Zv7vWaZKulLTaGLOy/bGfShogHfk+zor5AAAADkTy6UgAAICIRQkDAABwgBIGAADgACUMAADAAUoYAACAA5QwAAAAByhhAAAADlDCAMQEY8xJxphVxpiU9jtArDXGjHSdC0DsYrFWADHDGPO/klIkpUoqs9b+P8eRAMQwShiAmNF+H8klkpolnWqt9TmOBCCGcToSQCzpISlDUqYCI2IA4AwjYQBihjHmVUmzJQ2S1Ntae6vjSABiWILrAAAQCsaYqyR5rbXPGWPiJS0wxpxprZ3nOhuA2MRIGAAAgANcEwYAAOAAJQwAAMABShgAAIADlDAAAAAHKGEAAAAOUMIAAAAcoIQBAAA4QAkDAABw4P8H00tIfoKKIg8AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "fig = plot_data(X5, y, xlabel='x', ylabel='y')\n", + "theta_start = np.matrix([0, 0, 0, 0, 0, 0]).reshape(6, 1)\n", + "theta, _ = gradient_descent(cost, gradient, theta_start, X5, y, alpha=0.5, eps=10**-7)\n", + "plot_fun(fig, polynomial_regression(theta), X1)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "Ten model ma dużą **wariancję** (*variance*) – zachodzi **nadmierne dopasowanie** (*overfitting*)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "(Zwróć uwagę na dziwny kształt krzywej w lewej części wykresu – to m.in. efekt nadmiernego dopasowania)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "Nadmierne dopasowanie występuje, gdy model ma zbyt dużo stopni swobody w stosunku do ilości danych wejściowych.\n", + "\n", + "Jest to zjawisko niepożądane.\n", + "\n", + "Możemy obrazowo powiedzieć, że nadmierne dopasowanie występuje, gdy model zaczyna modelować szum/zakłócenia w danych zamiast ich „głównego nurtu”. " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "Zobacz też: https://pl.wikipedia.org/wiki/Nadmierne_dopasowanie" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Obciążenie (błąd systematyczny, *bias*)\n", + "\n", + "* Wynika z błędnych założeń co do algorytmu uczącego się.\n", + "* Duże obciążenie powoduje niedostateczne dopasowanie." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Wariancja (*variance*)\n", + "\n", + "* Wynika z nadwrażliwości na niewielkie fluktuacje w zbiorze uczącym.\n", + "* Wysoka wariancja może spowodować nadmierne dopasowanie (modelując szum zamiast sygnału)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## 3.5. Regularyzacja" + ] + }, + { + "cell_type": "code", + "execution_count": 56, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "def SGD(h, fJ, fdJ, theta, X, Y, \n", + " alpha=0.001, maxEpochs=1.0, batchSize=100, \n", + " adaGrad=False, logError=False, validate=0.0, valStep=100, lamb=0, trainsetsize=1.0):\n", + " \"\"\"Stochastic Gradient Descent - stochastyczna wersja metody gradientu prostego\n", + " (więcej na ten temat na wykładzie 11)\n", + " \"\"\"\n", + " errorsX, errorsY = [], []\n", + " errorsVX, errorsVY = [], []\n", + " \n", + " XT, YT = X, Y\n", + " \n", + " m_end=int(trainsetsize*len(X))\n", + " \n", + " if validate > 0:\n", + " mv = int(X.shape[0] * validate)\n", + " XV, YV = X[:mv], Y[:mv] \n", + " XT, YT = X[mv:m_end], Y[mv:m_end] \n", + " m, n = XT.shape\n", + "\n", + " start, end = 0, batchSize\n", + " maxSteps = (m * float(maxEpochs)) / batchSize\n", + " \n", + " if adaGrad:\n", + " hgrad = np.matrix(np.zeros(n)).reshape(n,1)\n", + " \n", + " for i in range(int(maxSteps)):\n", + " XBatch, YBatch = XT[start:end,:], YT[start:end,:]\n", + "\n", + " grad = fdJ(h, theta, XBatch, YBatch, lamb=lamb)\n", + " if adaGrad:\n", + " hgrad += np.multiply(grad, grad)\n", + " Gt = 1.0 / (10**-7 + np.sqrt(hgrad))\n", + " theta = theta - np.multiply(alpha * Gt, grad)\n", + " else:\n", + " theta = theta - alpha * grad\n", + " \n", + " if logError:\n", + " errorsX.append(float(i*batchSize)/m)\n", + " errorsY.append(fJ(h, theta, XBatch, YBatch).item())\n", + " if validate > 0 and i % valStep == 0:\n", + " errorsVX.append(float(i*batchSize)/m)\n", + " errorsVY.append(fJ(h, theta, XV, YV).item())\n", + " \n", + " if start + batchSize < m:\n", + " start += batchSize\n", + " else:\n", + " start = 0\n", + " end = min(start + batchSize, m)\n", + " return theta, (errorsX, errorsY, errorsVX, errorsVY)" + ] + }, + { + "cell_type": "code", + "execution_count": 57, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "# Przygotowanie danych do przykładu regularyzacji\n", + "\n", + "n = 6\n", + "\n", + "data = np.matrix(np.loadtxt(\"ex2data2.txt\", delimiter=\",\"))\n", + "np.random.shuffle(data)\n", + "\n", + "X = powerme(data[:,0], data[:,1], n)\n", + "Y = data[:,2]" + ] + }, + { + "cell_type": "code", + "execution_count": 58, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "def draw_regularization_example(X, Y, lamb=0, alpha=1, adaGrad=True, maxEpochs=2500, validate=0.25):\n", + " \"\"\"Rusuje przykład regularyzacji\"\"\"\n", + " plt.figure(figsize=(16,8))\n", + " plt.subplot(121)\n", + " plt.scatter(X[:, 2].tolist(), X[:, 1].tolist(),\n", + " c=Y.tolist(),\n", + " s=100, cmap=plt.cm.get_cmap('prism'));\n", + "\n", + " theta = np.matrix(np.zeros(X.shape[1])).reshape(X.shape[1],1)\n", + " thetaBest, err = SGD(h, J, dJ, theta, X, Y, alpha=alpha, adaGrad=adaGrad, maxEpochs=maxEpochs, batchSize=100, \n", + " logError=True, validate=validate, valStep=1, lamb=lamb)\n", + "\n", + " xx, yy = np.meshgrid(np.arange(-1.5, 1.5, 0.02),\n", + " np.arange(-1.5, 1.5, 0.02))\n", + " l = len(xx.ravel())\n", + " C = powerme(xx.reshape(l, 1),yy.reshape(l, 1), n)\n", + " z = classifyBi(thetaBest, C).reshape(int(np.sqrt(l)), int(np.sqrt(l)))\n", + "\n", + " plt.contour(xx, yy, z, levels=[0.5], lw=3);\n", + " plt.ylim(-1,1.2);\n", + " plt.xlim(-1,1.2);\n", + " plt.legend();\n", + " plt.subplot(122)\n", + " plt.plot(err[0],err[1], lw=3, label=\"Training error\")\n", + " if validate > 0:\n", + " plt.plot(err[2],err[3], lw=3, label=\"Validation error\");\n", + " plt.legend()\n", + " plt.ylim(0.2,0.8);" + ] + }, + { + "cell_type": "code", + "execution_count": 59, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + ":5: RuntimeWarning: overflow encountered in exp\n", + " y = 1.0/(1.0 + np.exp(-x))\n", + ":19: UserWarning: The following kwargs were not used by contour: 'lw'\n", + " plt.contour(xx, yy, z, levels=[0.5], lw=3);\n", + "No handles with labels found to put in legend.\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7AAAAHWCAYAAABOj2WsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3SURRfA4d+b3iD0GkroAqEmICC9S5EqTaWI9CIISFMQpUixAIKAFEUE2ycg0hGkKoQivdfQi0B6ne+PCZCQsptkk03Ifc7Zk+zuvPPe3ZTduzNzx1BKIYQQQgghhBBCpHc21g5ACCGEEEIIIYQwhySwQgghhBBCCCEyBElghRBCCCGEEEJkCJLACiGEEEIIIYTIECSBFUIIIYQQQgiRIUgCK4QQQgghhBAiQ5AEVgghhMhkDMNoZhjGGcMwzhuGMTqe+90Nw/jdMIx/DcM4YRhGT2vEKYQQQjzPkH1ghRBCiMzDMAxb4CzQGPADDgBdlFInY7QZC7grpd43DCM3cAbIp5QKs0bMQgghxBMyAiuEEEJkLtWA80qpi9EJ6SrgtefaKCCLYRgG4AY8ACLSNkwhhBAiLklghRBCiMylIHAtxnW/6Ntimgu8BNwAjgFDlVJRaROeEEIIkTA7aweQHLly5VJFixa1dhgihfwfBHD7yl1sbW0oUCIfji6OFu0/4GEgNy7cJnuerOQulMuifQshXiwHDx68p5TKbe040ogRz23PrydqChwBGgDFgS2GYexSSj2O1ZFh9AH6ALi6ulYtU6ZMyiK7dQyiogd683mBTYZ8myKEEMICEnptzpCvDEWLFsXX19faYQgLOH/4Eh++9in+VwMYtrAv9bu8gp6xljJ3rt2jX+WR1K+Smy/3TsbB0d4C0QohXlSGYVyxdgxpyA8oFOO6B3qkNaaewDSlC2WcNwzjElAG2B+zkVJqIbAQwNvbW6X4tXlaEQh5qL8f9Se45EhZf0IIITKshF6bZQqxsKoSlT2Zu38qxSoWYeobsxnXcio3L95OUZ+REZFMe2M2EWERjFs1TJJXIYSI7QBQ0jAMT8MwHIDOwNrn2lwFGgIYhpEXKA1cTNMohRBCiHhIAiusLke+7Hz21yT6f9aD47tO0bv8MNYt2EJyK2Qvn/Qzx3adYuj8PniUzG/haIUQImNTSkUAg4BNwCngJ6XUCcMw+hmG0S+62cdATcMwjgHbgPeVUvesE7EQQgjxTIacQixePLZ2trR7twV1Or7MZ+98zZf9F7Lr1310n9SZsi+XMquP8LBwfpqxlh8m/48mPerRsFvtVI5aCCEyJqXUemD9c7d9HeP7G0CTtI5LCCGEMEUSWJGu5CqYk49/H82aORv5YcqvDK05jmqvVubNDztS2qdEgutjT+47w+d9F3D5+DXqdKzBoDlvp3HkQoj0Ijw8HD8/P0JCQuLc5+TkhIeHB/b2srQgXbJADQQhROaT2P99kf4l9bVZEliR7tja6tHY5r0bsGbuRn6asYbBL48le153KtQtS4U65chTOBcPbj3kwc3/uHrajx2r9pKrYA4mrXmfGq28rf0QhBBW5OfnR5YsWShatGisD72UUty/fx8/Pz88PT2tGKEQQghLSuj/vkj/kvPaLAmsSLec3ZzpPLotrQY0ZefP+/j3rxP8u/0Ef/20L1a7LDnceG1QM3p+0gWXLM5WilYIkV6EhITE+ybGMAxy5szJ3bt3rRSZEEKI1JDQ/32R/iXntVkSWJHuuWZ1ofnbDWn+dkOUUty8eJtH9/zJmT8b2fJmkyrDQog4EnoTI29uhBDixST/3zOupP7spAqxyFAMw6BA8Xy8VL0keQrnluRVCCFeVMmsRC+EEGnt/v37VKpUiUqVKpEvXz4KFiz49HpYWFiix/r6+jJkyBCT56hZs6alws3wZARWCCGEEOmEjKAIITKenDlzcuTIEQAmTpyIm5sbI0aMeHp/REQEdnbxp13e3t54e5uu37J3717LBGuGyMhIbG1tE7yekMQepyXJCKwQQogXTkL7SCd3f2khhBAiKXr06MHw4cOpX78+77//Pvv376dmzZpUrlyZmjVrcubMGQB27NhBy5YtAZ389urVi3r16lGsWDFmz579tD83N7en7evVq0eHDh0oU6YM3bp1e/ratn79esqUKcMrr7zCkCFDnvYbU2RkJCNHjsTHx4cKFSqwYMGCp/3Wr1+frl274uXlFed6SEgIPXv2xMvLi8qVK7N9+3YAli1bRseOHWnVqhVNmqTN7msyAiuEJZ08CT/+CPfuQaFC0LUrFC5s7aiEyFScnJy4f/8+OXPmjLcKsZOTkxWjE0IIkZqKjv4j1fq+PK1FktqfPXuWrVu3Ymtry+PHj9m5cyd2dnZs3bqVsWPH8uuvv8Y55vTp02zfvh1/f39Kly5N//7942wvc/jwYU6cOEGBAgWoVasWe/bswdvbm759+7Jz5048PT3p0qVLvDEtXrwYd3d3Dhw4QGhoKLVq1XqaeO7fv5/jx4/j6enJjh07Yl2fNWsWAMeOHeP06dM0adKEs2fPArBv3z6OHj1Kjhw5kvT8JJcksEJYwr170K4d+PpCeDhERICDA0ycCK1awXffgbNUSBYiLXh4eODn5xdvRcMne80JIYQQqa1jx45Pp94+evSI7t27c+7cOQzDIDw8PN5jWrRogaOjI46OjuTJk4fbt2/Hed2qVq3a09sqVarE5cuXcXNzo1ixYk+3ounSpQsLFy6M0//mzZs5evQov/zyy9O4zp07h4ODA9WqVYu1lU3M67t372bw4MEAlClThiJFijxNYBs3bpxmyStIAitEygUEQI0acOWKTl6feLJof906aN4ctm0DM9YPCCFSxt7eXvZ5fSHIdG8hRMbm6ur69PsPPviA+vXr89tvv3H58mXq1asX7zGOjo5Pv7e1tSUiIsKsNuYukVFKMWfOHJo2bRrr9h07dsSK9/n4E+v/+eNSmySwQqTU/Plw/Xrs5DWmkBA4eFAnsq+9lraxCSFERiLbYAghUiip03zTyqNHjyhYsCCg141aWpkyZbh48SKXL1+maNGi/Pjjj/G2a9q0KfPnz6dBgwbY29tz9uzZp3Elpk6dOqxYsYIGDRpw9uxZrl69SunSpTl06JClH4pJUsRJiJRQCj77DIKDE28XEADTp6dNTOmJUnpd8J49cOGCtaMRQgghhLCKUaNGMWbMGGrVqkVkZKTF+3d2dmbevHk0a9aMV155hbx58+Lu7h6nXe/evSlbtixVqlShfPny9O3bN95R3ucNGDCAyMhIvLy86NSpE8uWLYs1EpyWjIxYkdHb21v5+vpaOwwhIDAQsmXTa15NyZoVHj1K/ZjSA6Vg6VKYNEmvD7az01OqixeHyZOhdWtrRyhELIZhHFRKmd7HQCTIIq/N04tB0H39/cgL4Jor5YEJIV54p06d4qWXXrJ2GFYXEBCAm5sbSikGDhxIyZIlGTZsmLXDMkt8P8OEXptlBFaIlLCx0cmauW0zA6VgwAAYMkSvCw4M1Il7cDAcPw5duuhRayGESEwG/IBdCCGsadGiRVSqVIly5crx6NEj+vbta+2QUoWsgRUiJZydwdMTzp9PvJ1hQM2aaROTta1bB8uX68Q1PkFBMH48NGoEFSqkbWxCiHRO1sAKIURyDRs2LMOMuKZEJhkSEiIVjRoFLi6Jt3FxgZEj0yYea5s6NeHk9YmwMPj887SJRwghhBBCvDAkgRUipXr0gKpVE97n1cUF2reHunXTNCyrCA+Hf/4x3S4yEtauTf14hBBCCCHEC0USWCFSyt4eNm+GN98EJydwc9Nfs2TR348YoQsaZYbtIUJDzV/r+2SfXCGEEEIIIcwka2CFsAQnJ1iwQG+Vs349PHwIefNC8+YJj8y+iFxd9Yjz48em2xYqlPrxCCEyMCniJIQQIi4ZgRXCktzddZXd/v2hXbvMlbyCHmXu0wccHBJv5+oKw4enTUxCiIwjM8xUEUK8cOrVq8emTZti3fbFF18wYMCARI95svXYq6++ysOHD+O0mThxIjNnzkz03KtXr+bkyZNPr3/44Yds3bo1KeFnOJLACiEsa/hwnaAm9EbUzg5y54auXdM2LiGEEEKIVNClSxdWrVoV67ZVq1bRpUsXs45fv3492bJlS9a5n09gJ02aRKNGjZLVV1JFRkYmej0hERERKTqvJLBCCMvKnx927YI8efQ64JiyZIFixWD3btOVm4UQQgghMoAOHTqwbt06QkNDAbh8+TI3btzglVdeoX///nh7e1OuXDkmTJgQ7/FFixbl3r17AEyePJnSpUvTqFEjzpw587TNokWL8PHxoWLFirRv356goCD27t3L2rVrGTlyJJUqVeLChQv06NGDX375BYBt27ZRuXJlvLy86NWr19P4ihYtyoQJE6hSpQpeXl6cPn06TkyRkZGMHDkSHx8fKlSowIIFCwDYsWMH9evXp2vXrnh5ecW5HhISQs+ePfHy8qJy5cps374dgGXLltGxY0datWpFkyZNUvR8yxpYIYTllSsHV6/Cb7/BN9/A/fvg4QEDBkCTJuYXehJCCCGESIqJ7qnY96N4b86ZMyfVqlVj48aNvPbaa6xatYpOnTphGAaTJ08mR44cREZG0rBhQ44ePUqFChXi7efgwYOsWrWKw4cPExERQZUqVahatSoA7dq145133gFg/PjxLF68mMGDB9O6dWtatmxJhw4dYvUVEhJCjx492LZtG6VKleKtt95i/vz5vPvuuwDkypWLQ4cOMW/ePGbOnMk333wT6/jFixfj7u7OgQMHCA0NpVatWk8Tz/3793P8+HE8PT3ZsWNHrOuzZs0C4NixY5w+fZomTZpw9uxZAPbt28fRo0fJkSNHcp79p+RdpBAidTg4QKdOsGULHDqkt81p1kySVyGEeZQUcRJCZBwxpxHHnD78008/UaVKFSpXrsyJEydiTfd93q5du2jbti0uLi5kzZqV1q1bP73v+PHj1K5dGy8vL1asWMGJEycSjefMmTN4enpSqlQpALp3787OnTuf3t+uXTsAqlatyuXLl+Mcv3nzZr777jsqVapE9erVuX//PufOnQOgWrVqeHp6Pm0b8/ru3bt58803AShTpgxFihR5msA2btw4xckryAisEEIIIdINKeIkhMiY2rRpw/Dhwzl06BDBwcFUqVKFS5cuMXPmTA4cOED27Nnp0aMHISEhifZjJFBDpEePHqxevZqKFSuybNkyduzYkWg/ysSHgI6OjgDY2trGuyZVKcWcOXNo2rRprNt37NiBq6trrNtiXk/svM8fl1ySwAohhBBCCCFeDAlM801tbm5u1KtXj169ej0dfX38+DGurq64u7tz+/ZtNmzYQL169RLso06dOvTo0YPRo0cTERHB77//Tt++fQHw9/cnf/78hIeHs2LFCgoWLAhAlixZ8Pf3j9NXmTJluHz5MufPn6dEiRIsX76cunXrmv14mjZtyvz582nQoAH29vacPXv26TkTU6dOHVasWEGDBg04e/YsV69epXTp0hw6dMjsc5siCawQQgghhBBCpFCXLl1o167d06nEFStWpHLlypQrV45ixYpRq1atRI+vUqUKnTp1olKlShQpUoTatWs/ve/jjz+mevXqFClSBC8vr6dJa+fOnXnnnXeYPXv20+JNAE5OTixdupSOHTsSERGBj48P/fr1M/ux9O7dm8uXL1OlShWUUuTOnZvVq1ebPG7AgAH069cPLy8v7OzsWLZs2dPRXksxTA0vp0fe3t7qyb5JQgghREoZhnFQKeVt7TgyMou8Ns8oCYF39PfvnYEs+VIemBDihXfq1Cleeukla4chUiC+n2FCr81STUUIIYQQ6UNC+0cLIYQQ0SSBFYlTCvbsgTZt9L6euXJB48awebNUiLSU06ehXz+9f2rOnFC1KvzwA0Tv1SWEEEIIIYTQZA2sSFhUFPTuDT/+CMHBzxLWrVth3z6oW1fv8+ngYN04renMGdi2DcLCoEwZndzb2pp//Ny5MGoUhIfDkwpwDx5A374waRLs3Kk/OBBCiBfco6BwbEMjcLN2IEIIIdI1i4zAGoaxxDCMO4ZhHE/gfsMwjNmGYZw3DOOoYRhVYtzXzDCMM9H3jbZEPMJCJk7UyWtQUNzR1sBA2L5djxxmRpcuQY0aULkyjBgBo0fD669DvnywcqV5faxbp5PX4OBnyesTAQFw8aJOiGWkWwiRCTwOCScoLNLaYQghMqiMWNdHaEn92VlqCvEyoFki9zcHSkZf+gDzAQzDsAW+ir6/LNDFMIyyFopJpERQEHz2mf6akOBgnazdvp12caUHV6+Cjw/s36+fg+BgPd3X3x/u3dOj1kuWmO5nzBh9bELCw3USu3275WIXQoiMQt6MCiHM5OTkxP379yWJzYCUUty/fx8nJyezj7HIFGKl1E7DMIom0uQ14Dulf6v+Ngwjm2EY+YGiwHml1EUAwzBWRbc9aYm4RAr8/jvYmPn5xqpVMHRo6saTngwaBA8f6inW8QkK0m3atwd39/jbnDsHFy6YPldAAMybBw0aJD9eIYTIIOStpxAiOTw8PPDz8+Pu3bvWDkUkg5OTEx4eHma3T6s1sAWBazGu+0XfFt/t1ePrwDCMPujRWwoXLpw6UYpnbt7U6zpNCQnRI5KZxe3buoBVpIlpboYB334LQ4bEf//Nm3rtcGIjsE9kpudXCJFpSQFiIURy2dvb4+npae0wRBpJqyrE8b0sqURuj3ujUguVUt5KKe/cuXNbNDgRD3d3sDPj8w07O8iRI/XjSS8OHQJzpjgEBeliVwlxdzedBD+RmZ5fIYQQQgghEpFWCawfUCjGdQ/gRiK3C2tr2dK8BMveHjp0SP14EqFQHGUrc3iTyTRnAX05z4HUOVlC04aT2tbLC7JkMd1HlizQs6f55xRCCCGEEOIFllYJ7FrgrehqxC8Dj5RSN4EDQEnDMDwNw3AAOke3FdaWOze0a5f4aKODA1SrBqVLp11cz7nNJYZQihm0ZSffc4SNbOMbJlKPcdTCnweWPWGFCubtz+rsDK+8kvD9NjYwdiy4uJjup23bpMUohBAZkBFnDrGsiBVCCBGXpbbRWQnsA0obhuFnGMbbhmH0MwzjyR4r64GLwHlgETAAQCkVAQwCNgGngJ+UUicsEZOwgIULoWzZ+JMsJyfw8IBffkn7uKI95h7jqMFtLhJCwNPbFVGEEsRFfJlIPcIxYy2vuQoVglq1TC/WUkpXI07MwIG60JOra9z77O31NOMtWzL3PrtCiExFxbuySAghhHjGIgmsUqqLUiq/UspeKeWhlFqslPpaKfV19P1KKTVQKVVcKeWllPKNcex6pVSp6PsmWyIeYSGurrBnD0yZopNVBwd9yZ0bxo+Hw4chVy6rhbeeLwniIYr4p+pGEMZtLvIPv1r2xHPngptbwve7uOg9dE09N08KPS1eDBUr6vXEjo76ee/XD44d0yO+QgghhBBCCCDtqhCLjMrJSW+RM2QIPHqk13Vmy2b+FjupRKHYyFeEk/h03pDQYFYdm43N9RLcu/6A+zcekCWHG55ehSlavjA58mWLZ9qaCWXKwO7deor1rVu6YJNSOqlVCj75BN5917y+DAM6ddKXgABd1TlbNvMKaAkhxAtExl6FEEKYQ94lC/MYhk6s0okQAgjBP8H7o+47EPZ1MULnluDRbScmMB0AGxuDqKhn66qy5c5KnY41aNK9HqW8i5ufzFaooPdy3bNHT/MNCYFy5XRBK1PrWhPi5pb4yK4QQmQmStbACiGEiEsSWJEh2WJPVDxTh6PuOhAyvjxhy4tAiC12zW6RteddJnn+Qa6COciWJyv+DwK4fPwal49f48Te02xY/Cdr520in2ceGnatzWuDmpE9rxnJumHoQk2JFWvKaC5dgtmz9drm4GA9dXzoUOjcWReUEkKIVCRrYIUQQpgiCazIkBxwohDlucrRp7dF+GYnsH0N1G1HHN66guOQ89iVD+BlOlCa4k/bZcvtTqX67lSqX542g5vj/18Ae1Yf4K+f9vDDlP/x86y1NH+7IR1HtCZvkUy05/D330OfPnr7pLDowlf37+vp4x99BLt26SJWQgiRCpK6mkMIIUTmZN2FjCJzS+H0sHaMwRFdwTd0aRECatcDG4Xb3u24LDyEbfnH2ONEa0Ym2k+W7G4061mfqRvGs+TUFzToWps/Fm6he8nBfNFvIf/deZSiOFONUpabYrdrl05eg4OfJa9PBASAnx/UrQvh4ZY5nxBCCCGEEMkgCaxIO0rBxo1Qv76uZmxnB0WKwJdfgn/C61kTUoPXqRjVjNBB3gS/7YNd7XtkObANuyoPAXDEhRa8S3G8ze7To1QB3vumP9+en0vLvo3ZuORPepQazM8z1xIWYsHteJIrNFRXLS5TRj9/Dg7g46On/EbFX43ZLOPH6+Q1IZGRcO8erJVtmoUQQgghhPVIAivShlJ6hK9DB9ixQ4/kRUXB1aswdix4ecHNm0nq0lAGtv0bETyvKG7DrpBr/VHccjnhhBvZKUBP5tCV5O3MlKdQLgbNeZuFR2dR/pUyLBy1nO4lB7P1+50oaxUWefwYqlfXU3rPnNHPX0QE+PpCjx7QqlXyRkjv3IF//jHdzt9fbyEkhBCpwIiz/lWKOAkhhIhLElgRv/BwnVA+eGCZaapffgk//ACBgXHvCwqC69ehaVOzz6WU4qshS9iwaBudR7flp5m7GW23hkF8x4dsZQF+NKRXisMuXKYgk9eNZca2CeTIn51P35rDe/UmcPHolRT3nWSdOsHp0/r5el5goP5gYNSopPd7+7bef9Yc168nvX8hhDCTpKxCCCFMkQRWxHb9OgweDNmzQ/HikD+/nq66dGnyp6hGRsLkyfEnXk9EROgKuHv2mOxOKcWC975lzVcb6fheK3pN7oK94UA56uLDa5Skejyf5KdMpfrlmfP3FIYt7MeVk370rzqKr4YuIcg/kWm3lnThgk5QQxPZ9zYoCBYs0GtWk8Ld3fyRW3f3pPUthBBmkiJOQgghzCEJrHjm5Ek9lXfBAj2i96Sgz9mzOqlt21Yno0m1b1/iidcTgYGwZInJZss/+plfv/iDNoOb8870N83fuzWFbGxseLV3Q5ae+ZIW7zRizdyNvOM1nP0bDqf+yVetMu+5t7ODP/5IWt+FC+uLKa6u0LNn0voWQgghhBDCgiSBFVpkJDRuDA8fxj8aFxgIW7fCtGlJ7/v+ffM+WlfK5DrYrd/vZPmkn2naoz4DvuiZZslrTFlzZGHIvHf4fNfHOLo4Mq7FFD7qMJM71+6l3klv3jRvlDQ8XBdbSqoPPwQXl8Tb2NnBm28mve/MLjISjh2D/fv1emMhhBBCCJFsksAKbd06XaQnsTWoQUEwa5ae7psUuXKZt7bVMKBAgQTvPrrzJLPenkel+uUY+vU7VkleYypXszRfH55Bz0+6cGDDYXqXG8bqORuITM4otSn58+uKw6bY2+vnO6m6dIFevfQo6/NsbcHNDTZsgCxZkt53ZhUeDlOmQL58ULMmNGmiq243aQKHDlk7OiHSnTj/0a1VME8IIUS6Jgms0JYvN28rm4gI8yrWxvTyy+DkZLqdi4tOouJx7cx1JradTv7i+fjwlxHYO9gnLYZU4uBoT9ex7Vh0/DPK1SrNV0OX8O4rH3Dp+FXLnqhzZ7Ax4881IgJatEh6/4YBc+bA99/rbXns7PTPzMkJuneHw4ehRo2k95tZhYfromSTJ+sR8YAAePQIQkJgyxaoXRu2bbN2lEKkO8rC9QuEEEK8eCSBFdqDB+a1s7HR27kkha2t3mc0sSmqdnZQooQeqXrOw7uPGNdiKrZ2tkxeN4Ys2d2Sdv40kN8zL1PWj2P08iHcvHCL/lVGsfyjnwkPS8a2NvEpXhwaNEi8WrCLC/Trp0dLk6tNGz3V9b//4MoV/bNevFj/bIT5pk6Fv/9OuHBZUJB+rpNacEsIIYQQIpOTBFZonp7mj/AVLJj0/gcPhrfein+KqosLFCoEGzfGWSsbHhbOpA6zuHf9AR+teZ/8xfIm/dxpxDAMGnarzeKTX1D39Rp899FPDPQZzen95yxzglWroFy5+J9DV1do1AimT7fMudzcIE8ePSVZJE1EhN42KthEhWql9Ii3EEKTwVchhBBmkARWaH37grOz6Xb58+tKxUllGDBvHvz2GzRsqEcSbW114jx9Ohw9qtcKPmf+u8s4tusU733Tn7Ivl0r6ea3APVdWxnw/lElr3sf/QQBDa47j6+HLCA4MSVnHWbLA3r3w1VdQtqwetba3h+rV9RTw337TtwnrOnrUvIJbgYH6QwkhRAJkDawQQoi45N2u0Hx8oFIlOHBAb50THxcXPTUyucWTDENXOm7c2Kzm6xZs4fevN/P6iNY07FY7eee0ohqtvKlQ5yW+Gb2CX7/4g71rfRm2sC+VGyTjA4AnHB31mtTu3S0XqLCs4GDzZjOATmKFEE/JGlghhBCmyAis0AxDVyIuVy7uGko7Oz06+/HH0KFDmoRzbNcp5g5ejE+zSvSa2jVNzpkaXN1dGTq/DzO3T8SwMRjVaBKf9/magIeSuLywihQxb99jGxsoUyb14xEigzAkeRVCCGEGSWDFM9my6QI+y5dDrVp6DaSHB/TuravQDh+eJmFcP3+TCW0+JX+xPIxZMRRbW9s0OW9qqli3HAuOzKTje63YuORP3i43jD2r91s7LJEaPDzA29t0O2dnvTZcCCGEEEKYTRJYEZudna6Ouns33L4N167B/PlQunSanD44IJiJbWeAYTBlw7h0WXE4uZxcHOkz4y3m/DOV7HndmdhuBpNen8WDW/9ZOzRhadOnJ76m3MlJr1328Um7mIQQQgghXgCSwIp0QynFjF7zuHrKj/GrhpHfM/1WHE6JUlWLM/efqfSa3JW/fz9I73LD2PztDpSSgiUvjBo14Oef9brxmNtHGYauGP3yy7BmTfLXkwvxAorz5yD/E4UQQsRDEliRbqyatppdv/xN72lvUKVRBWuHk6rs7O3oMqYtC47MoHBZD2b0/IpxLaZw+8pda4cmLKVFC7h+HaZM0aOtFSrA66/Dli3w558p269XCKbdGnQAACAASURBVCGEECKTkgRWpAt/rtzNknE/UL9LLTq818ra4aSZQqUL8tlfkxg4uxfHdp2id/lh/O+LP4iMjLR2aMISsmWDoUPh77/h33/1tjk1asjIqxBCCCFEMkkCK6zujO8FZvaah1edlxixZCBGJntzb2NjQ5tBzfnm+Od41SnL/OHLGFpzHOePXLJ2aOmTn5/e7unMGZliKMQLJHP95xdCCJFcksAKq/rv9kM+ajeDHPmyMeGXETg42ls7JKvJWyQ3k9eNYewP73L7yj0G+oxm0ajlhASZsSVLZrB9u147WrIkNGoEVatC0aKwYIEkskIkkWEYzQzDOGMYxnnDMEbHc/9IwzCORF+OG4YRaRhGDmvEKoQQQsQkCaywmrDQcCZ1nMXj+/5M/G0k7rmyWjukVHOeA3xJN96jAqOpxm9M4zH34rQzDIP6nWux+OTnNO1Rn59mruUdr+Ec3PKvFaJOR777Tq8p/ecfCAmBx48hMBCuXtXbO/XqJUmsEGYyDMMW+ApoDpQFuhiGUTZmG6XUDKVUJaVUJWAM8JdS6kHaRip/00IIIeKSBFZYRVRUFDN6zOX47tOMWDKAEpU8rR1SqggjhKm0ZCL12cMqrnKMCxzgFybRn8LsZmW8x2XNkYXhi/oxc/tE7OxtGd30Ez7tPoeHdx+l8SNIB65cgX79IDg4/vuDgnTF31Wr0jYuITKuasB5pdRFpVQYsAp4LZH2XSCBf1YWZBgGSslEYiGEEImTBFZYxdLxq9jx4156T3uDep1qWTucVPMlXTnOn4QSiCLq6e1hBBNGMPN5m3/ZkuDxFeuWY8GRmXQb157tK/fwdtlMuOXOnDlgqqhVYCBMnZo28QiR8RUErsW47hd9WxyGYbgAzYBfE7i/j2EYvoZh+N69K1XUhRBCpD5JYEWa+/OHXaya9huv9m7I6yNbWzucWAJ5xHrmMJSX6EVuBlGC/zGFRyT9jdk1TnKEjYSRwMghOpH9jvcS7cfByYEeH3dm/qHpeJQuwIyeX/Fe/QlcOXkt0eNeGL/8AmFhptudOQMP0niGoxAZU3zDnAl9KtYK2JPQ9GGl1EKllLdSyjt37twWC1AIIYRIiCSwIk2d3n+OWb3n41XnJQbNfTtdVRy+wlEGUZwfGM0NTuPPPW5zgV/5hEEU5yQ7k9TfZuYTQbjJdre4gB+nTLbzLF+Yz3dOYtjCflw+fo2+lUayeMyKF7/IU0JTh59nZ6enEwshTPEDCsW47gHcSKBtZ9Jg+jDEk1VnppkmQgghzCYJrEgzfmdvML7lVHLkz86HP7+HvUP6qTjsz30mUI8A7hNK7CQojGBC8Gcqr3Kbi2b3eZ1TRBFhsp0d9tzBvC1zbGxseLV3Q5ac+oKGb9Rm1aer6V1uGH+vO2h2XBlO0aLmtVMKZARICHMcAEoahuFpGIYDOkld+3wjwzDcgbrAmrQKTFJWIYQQpkgCK9LEf3ceMab5ZACmbhxPttzuVo4otm18QzghibYJJ5Q1zDC7TyfczGqnUDjiYna/ANlyuzNyyUBm7fgIJ1dHPmg9jYntpnPn6gu4Bm3YMHAz8Vza2kLnzuDomDYxCZGBKaUigEHAJuAU8JNS6oRhGP0Mw+gXo2lbYLNSKtAacQohhBDxMTJiMRhvb2/l6+tr7TCEmcJCwhjZaBIXDl9i5vaJlKlWMtl9RRHFMbZxgu1EEkFRKlKd9jjglKIY+1OEe1w12c4BZ74nECPeJWSx7WEVX9OHEPwTbeeEG4u5m+zHEB4Wzq+f/8H3k37GMAy6jmtP+9ZlcFi7Gu7eBQ8PndwVKJCs/mN59Ah+/FGvN3Vzg1dfhWrVIDWngoeFgZcXXLwIEQmMaLu5wZEjULx46sUhXmiGYRxUSnlbO46MLKWvzQ+Dwng8rSyFbaI/iBtyBHK8mBXqhRBCmJbQa7OdNYIRmUdUVBSzes/n5N4zfPDT8BQlr2fYx2d0JIhHhBAA6ORvIf3pwec05O1k9/3YzCJNEYQRRrBZI6bVaMciBiTaxh4nGvJOihJwewd7Or/fhvqdazF/0EKWjPuBzeMDGGQcpWrkDT0qOXYstG8PixeDUzLOpRRMnAjTp4ONjV5ramMDs2ZBkSKwejWUTP7PNlEODrBzJzRooPd9DQh4dp+bm177unGjJK9CCCGEENYScAdu/qsv2YpAhY6pdipJYEWqUUqxcMR3/PnDbt6e0pU6HWoku68L+PIxjeKsT32SyC5lCFFE0Ji+yerfEddEqwXHZG9msmmPA2P4g09oTAhBPL+6ywFnClGerkxJarjxypvLhYnnlnPANoC5EeUZbdSkNn70DT1KXoLgt9/g1i3YvFlPuU2KESPg668hJMY066govX3N6dNQvTocPqyT2dSQNy8cOwabNsFXX+m9Yd3doWdP6NIFXJI2BVsIkRFkvBliQgjxwlMKHvk9S1ZvHdVf/W8+a1O8oSSwImP6cfoafv3iD9oMbk6n99ukqK+veSdO8hpTKEF8y3BeoRvOZq49jakWndnCAiITrRpsUJnm2CRh6XhpajCFf/iBMfzLFuxwQKGwxY7mDKYtY1I8/fmpr7+Gq1fxiQhmETf5WZViJWXYTz46cYbXg8/guH8/rF8PrVqZ3++FCzBvXuzkNaaoKD21ePRoWJmKxUptbKB5c30RQrxwDAyUGcszhBBCpBGl4L9LcOPws4T15lEINrFt4c1/9bGptMRMEliRKv5cuZvFY1ZQv0st+n/eI0Xb5VzjBDc4Y0ZLg92sSNYo7KsM5U+WJJrAOuBMW8Ykue9ClON91vKYe9zlMnY44sFL2Fryz08p+Oyzp1vOOBBFN07TmCsspALfGeXYpIrSJ+Aotad9ipGUBHbuXIiMTLxNVJSeRvzff5A9ewoeiBU8eACXL+upymXK6CnJQgghhBCZydNk9QjcPPIsaQ15ZN7xds6QzwvyV4T8FUBFgZHEGX9mkndqwuL+/esEM3t+hVedlxixZCA2Nikrdn2Jw9hg+g8glEDOsC9ZCWx+SjCQpXxFT8IJRsWZ7utCV6ZQmppJ7vuJrOQiK7mSfXyigoLgzp04N+chmPH8Qyt1ka+oyMdGDSrsf8DAo1coVsHM6b579kC46f1scXTUxZ1efjmJwVvJqVN6bfCGDTr2qCiwt4fBg2HMmOStFRZCJJ8MvgohRNqIk6xGfzU3WXV010lq/orPLjlLgE3qJKzPkwRWWNS5Qxf5sPWn5C+el4n/G4mDY8r3ejWwwdx3NkmZ3vu8mrxOXorzKx9zmA3YYEskEbxEHTrwAeWom+y+U52Njf5nlICK3GU+21ivPFkaVZ7+VUbSqn9T3pr4OllzZkm8b3PXyyql48gI9u+Hhg31Gl6lIDT02X0zZuik9q+/wNnZejEKIYQQQqSUUvDf5WejqjeORI+sPjTveOccUKAS5K/0LFnNXjR1d6AwQRJYYTHXzlxnTLNPyJLDjWmbPiBrDhOJkZlKUt3E2lTNCTfK0yBF5ypOVUaxmlCCCeQ/XHDHCdcU9ZkmnJ11AaWLFxNsYouiFRepW68ky8q04vf5m9i2Yhddx7XntUHNEv6woVEjXaApZpIXn/BwKFs2BQ8ijYSF6e1/YlYzjik4WBeMGjsWPv88bWMTQjyTAbf5E0IIq4qVrMYYXTU7Wc2uE9UClZ8lrdkKWzVZjY9FEljDMJoBXwK2wDdKqWnP3T8S6BbjnC8BuZVSDwzDuAz4A5FAhOzDlzE9uPUfY5tPxrCxYdrmD8jtkdNifeejOMXx5gx74kztjcnAoAaWqXjmiDOOZLDRt1GjYPhwPZ04Ia6uZB07kiENG9KqfxMWjVrOwpHf8fv8Tbw9tRt1Orwcd71y//4wc2bi57a3h27d9LY26d3q1aaT8ZAQWLQIpkyRUVgh0ohhIEWchBAiKR7fgOsH9eXJ6Kq5yapTttiJaoHK6TJZjU+KE1jDMGyBr4DGgB9wwDCMtUqpk0/aKKVmADOi27cChimlYpavqq+UupfSWIR1BD4KZFyLqTy8+5iZ2z/Co2R+i5+jP4sZQzWCeRxvEuuAM4P4DnscLX7uDKNnT/j2Wzhy5Gkxp1hcXKB1a72fKuBZvjBT1o/Dd/O/LBjxLZ90+oyyNUrRd1Z3yr5c6tlxBQrAxx/DhAnxJ8d2dpAnD0ydmkoPzMJ+/DHh0deYbG1h376nz5cQQgghhNUEP9RJ6vWDcP2Q/hpwy7xjnbLFTlQLVNJ7tWaAZDU+lhiBrQacV0pdBDAMYxXwGnAygfZdgFTca0OkpZCgUMa3msbl41eZtOZ9SnsXT5XzFKAUU/iH2XTDL/pX68l2NC64049FVCaTb6/i4ADbtsGAAbBqlU4sw8P17ZGRMGQITJ4c55+Vd5OKVD48g83LdrDsg1UMrTmO+l1q0WtyV/IVzaMbjRgB2bLB++/rPsPCdIIXFQW1a8Py5ZArlQpUWZo5ySvo5ymx0eyEHDwIBw7o7318oGrVpPchhBBCiMwrPARuHYseWY1OVu+fN+9YJ/e404CtvGbV0iyRwBYErsW47gdUj6+hYRguQDNgUIybFbDZMAwFLFBKLbRATCINhIWG81H7GZzce4axK4fh06xyqp6vIKX5FF+ucYKT7CSKCArjRVnqYsi0M83ZGZYuhVmz4Pff4eFDyJcPWrYE14TX8tra2tL87YbUfb0mP366ml8++53dv/5Dy35N6DquHdlyu0Pv3tCjB2zcqNfaOjlB06Z67W1GUqYM/PknREQk3i48HDw9ze/3wAHo3h2uXtWJPegXiyJFYNkyqFYt2SELkRnIf3EhRKYUFQn3zj6bCnz9ENw+DlEm3qcA2LvqRLVgZShQRX//giWr8bFEAhvfM5TQQsVWwJ7npg/XUkrdMAwjD7DFMIzTSqmdcU5iGH2APgCFCxdOacwihaKiopjRYy6+m/7lvcUDqNuxRpqduxDlKES5NDtfhpQjh06mksglizM9P+lCy35NWP7Rz6yZu4FNS7fT4b1WtB/WEpcszjoZzsj69dPrW00lsEWLQjkzf8/++UdPNY5vxPbUKahfX4+OZ5QthoSwEinbJIR4oSkFj/xiJ6s3j0CYGbPDbOwgbzkoWFUnqwWrQu7SabZ1TXpiiQTWDygU47oHcCOBtp15bvqwUupG9Nc7hmH8hp6SHCeBjR6ZXQjg7e0tr3FWpJRi3tCl7PhxL72nvUGznvWtHZKwsNweORm+qB8d3mvF0vEr+W7iT6yZu4Fu4zvQsl9j7B1Svj2S1bz0EjRrprfKCQmJv42zM3z2mXn9KQWdOyc+3TgoCLp00SPXL/inokIIIYSIFuqvk1S/A+DnC9d9IfCuecfmKK6T1IJVoWAVyOcF9lJYEiyTwB4AShqG4QlcRyepXZ9vZBiGO1AXeCPGba6AjVLKP/r7JsAkC8QkzBEYqNcDZs+u10maacm4laz5aiPth7Xk9ZGtTR8QFQUPHug1k9myyRv4DKRwmYJM+GUEp/efY/GYFcx7dym/zV5Pj0mdqNupJrbm7hGb3vzwA3ToADt26IJXT6b8OjvrhHTBAj092hy7d8NdM16M7t7VbWvXTnbYQrzI4lRAF0KIjCQqSq9T9dv/LGG9cxJUlOlj3fI+S1QLVtVTgZ2zp37MGVSKE1ilVIRhGIOATehtdJYopU4YhtEv+v6vo5u2BTYrpQJjHJ4X+C36RcsO+EEptTGlMQkTNm6ETz7R0x7ton8FOnSAceP0+sBE/DDlf6ya9hst+jSm78y3En/D8egRfPklzJ6tE2WlIG9eGDkS+vQBx0xcMTiDKVOtJNO3TsB30xG+Gb2CqW/M5ocp/+PNCa9Tu311bGxsrB1i0jg5wbp1et3qF1/A0aP697FtW3jnHV1V2Vx795relgd0m717JYEVQgghXgTB/+lpwH6+cG2/Hl0NeWT6OIcses1qzKnAWQvIAE8SWGQfWKXUemD9c7d9/dz1ZcCy5267CFS0RAzCTB98oKdGPpnu+GQd4MqV8NtvsHZtgtuGrFuwhaXjV9KwW22GzOudePJ6+zZUr66/xpymee0ajB4N33+vR7/SyR6bCsVJdvIX3+HPPXJRmAa8jSeVrB1aLA+5zTa+4QK+2ONAFVpQg9dxwCnVz20YBj7NKlO1SUV2/fI33330E590+oxiFYrw5oSO1HzNJ+Mlsj4+sGJFyvqIitIfzpii1LORXiGEaeb8XQkhRFqIioQ7p56NrPodgHtnTB9n2ECesuDhDR4+UNAbcpWCjPZ+KZ2xSAIrMojff4+dvMYUGamnFL/2Gly6FGdLlL9+2svsAYuo3qIKI5YMMJ2otGkD16/HXygnKEiPeA0YoCvmpqFQgrjJOQDyUxJHXLjLVT6hKQ/wI4RAQGFgw58soQQ+jGI1rmRL0zifp1CsZDzr0Osyw9EfChxiPd8wiHdZSVVapEksNjY21H29Jq+0r85fP+7lu49+5qP2MylWsQg9JnXm5ZZVzZoK6M8D7nEFOxwoQGlsM+q/o0qV9B67/v6Jt3Nx0W2FEPEy0P99hRDC6gLvRSer0Zfrh8wrtOSSEzyqxUhYq4BjltSPN5MxVAb8hNPb21v5+vpaO4yMp1q1Z/tTJsTZGSZM0Pt9Rtv3uy8ftZ9JmeolmLbpA5xcTEz9PXZMV1s1tYemkxPcuKHX4KayR9zhRz5kJ8uxiU6UooigJp04xHr8uUcUkXGOs8MBD8oylf3YYb3CRav4gHV8RijxP6cOuDCWPyhHvbQNDIiMiOTPlbv5ftLP3LhwmzLVS/LGBx2o1rxyvInsdU6zgjEcYQN2OKKIwg4HXmUobXgfezLY1PLISChQAO7cSbxdnjz69z2jrht+wRmGcVAp5W3tODKylL42B4ZGcHtyOYrZ3NI3DDoIuUpYKDohhEhAVJTexuba33D1H/31wUXTx9nYQd7yUKiaTlY9vCG7p0wFtqCEXpsz6JDHC0YpvTZuzhw4fVrv19mpk94Gxd3dMud48AD+/dd0u+BgPSoancAe2nqUjzvOokTlokz+Y6zp5BXg55/NWxNob68rwXaNU/PLou7jx2h88Oc+kYTHum8H36JIeFpnBGHc4jz/8D9q0SneNsH4s5Pv2c0PBONPfkrSjEGUpY5F9qd9zD3WMoNwEn5OwwjiGwbyOSdSfL6ksrWzpfGbdanfuRabl+1gxeRfGd9yKiWreNJ1XPtYU4vPsZ9JNCSUQBQq1mNazTQOsZ6P2JEmU6ItxtYWFi+G11/Xfz/xcXHRbSR5FUIIIawrPFiPqD5JWP326/Wsprjlg0I+0SOsPpC/Iji4pH68Ig5JYK0tIEDvq+nrq0csn4yI//svjB2rk8HmzVN+nsePdcIYFmZeW+DMgfNMaDsdj9IFmLJhHK5ZzfwjvX9fj0qZEhGhCz2lss94ncfcI4q405kTS16fCCGANUyPN4E9znY+5TUUUYSi65Nd4SiH2UBhyjOOjSmefrydpRiYXitxl8tc4ojV1u3a2dvx6juNaNy9LttW7Gbl1P/xUfuZFCnrQadRbXiliw9T7F8lhPin4IQRzFWOspJxdGdWGkefQi1b6nXkPXvq3+sn04mzZNGF0pYuzfj75wqRymTQQgiRKgLuwNW/4do/+uvNfyEqPPFjbB0gfyWdqBby0V+zFpR/VOmEJLDWpJR+U/v333FHLAOjizU/2erDxydl58qVC8JN/LE+kT8/V05eY+yrU8iWOytTN44na44kzN8vXFhXdDU1CmtvD/nzm99vMvhxissciTd5TYpbnI9z2xWOMo2W8UzrVYQSyCUO8wlNmcLfKRqJvcQhwkhgZC8GA1uuc8rqhafsHexp1rM+jd+sw18/7WXVp6uZ3mMuCye4EDU6D0aP/zAc4//gIIxgtrKQznyCI+mjwJfZXntNFy1bu1b/TYOeSt+6tf5dF0IkUcZb4iSEsLKoKF1cKWbC+t8l08e55IRCL0Ph6vprgUpgl8GWNGUiksBa0759euQ1sUQvKEhX7d22LWXncnPT+1quW5d4ZUc3N2506sWoxh9jZ2/LtM0fkDN/EteodusGEyeabqeUZUaXE3GIP+Jd25pU8RUYWsk4QhNJLCMIw4+THOdPvGiY7HPbmzmd1kCv2U0vbO1sadC1NvW7vML+9YeYMXkqj/uXw/i4GI5Dz+HY5yKGe9wPFgxsOMfflKd+ks95haOs43OOsoVIIiiMF614j4o0wcaMUewUs7eH9u31RQiRZFLESQiRJOEheiuba39HJ637IeSh6eNylYJC1aHwyzphzVlcRlczEElgrWn2bNOFjkCvj715M+WjlRMm6EQ4oXMaBnddcjJq3nEiwiKYteMjCpZIxjk9PKBdO70tT2JrAkePTvW9YEMJIgIzR54TYGBDeWJvLeTPA46yBVMjBCEEsI4vUpTAVuFV/uZXQki8ym04YZSmVrLPk1oMw6B6i6pUePUxh7b+S+j00oS8X4GQyS/h2O8CjoMuYOPx7PfEwEiwWFVinlRpjiDs6YcWx7jNOf6mOD6M4Y+MN6prSX5+sGWL/pssXhwaNZI1uSJdsUTNACHECy7UXyepV/bqy3VfiDSxPM7WEQpUfja6Wqg6uOZMm3hFqpAE1prOnDFvnztHR7h6NeUJbNWqev/VN97Qa1Rjjvy6uvLYLQdjsrbA/7Y/M/6cQNFyhZJ/riVL4N49nXwHxFjzaGOjqw+/9RaMGZP8/s2Uh6I44Zrguktz2ONEa0bGuu0BftjhmGhhpSduRW/bk1w+tMGWvom2scWOSjQhO/lSdK7UVNAozYnG27FvvIuIg9kInVGa0JmlCf2sFPYd/XAceg67av8RQTh5KJqkvjfzNX/webxTrUMI4Cz7mE03RvI/Cz2aDOTOHejRA/78U6/HjYzUI8WOjjBjhr5PCCGESI+CHuiR1St7dMJ6819QJmbWyXTgF54ksNbkYmZRpMhIvb2NJbRtqysdz5sHK1botbb58xPcZwDjV1zlxr9XmLJhHKWqFk/ZeRwddYXhrVv1m+QjR3TyWqcOvPeeXhuYBqrTnkX0T/bxjrjQlEGUpFqs2x1wMXtdrUMKR/3ssGcEvzKVloTFMzJpix1ZyU0fFqboPKmtCf35k8WEEYFd1YfYrfqHyEvHCJtbgtDFnoSvLIxtrXsUHB5CgdZlwMzBwUgiWcUHiY7ahhPCYTZwiwvkI4W/2xnJ/fvg7Q23buk18E8+tAoJ0YWmBg6Ehw/h3XetG6cQ8cmA2/wJIVLI/9az0dUre+GOGbsr5CyhpwIXriHTgTMJSWCtqUsXXW34ScGmhDg5QblyljtvoUIwdaq+AGEhYUxoNY0zBy/ywc/vUal+ecucx8YGmjTRFytxwpU2jGE1UxPZQ9WZ0tTkHPuJJAyFAgxcyEonJtE4ntHPfBTHlewmp7o64EwtOqf4cZSnPhP5k28YiB8nsY3ekzaCMKrwKr2ZRzbypvg8qakQZalIE46wiXBCALD1DMJ51lGcJp4kbElRQr8oxdX2uehVfChth7agaY96OLsl/gHASf4iwoyRcEUk21lCFyZb5PFkCOPGPUte4xMUpGdCdOqU6gXVhDBF3m8KkckoBQ+vRCer0SOsJvdfNSBvOShSU18K14Qs6fv9j7A8SWCt6a239DrQxDg7w7BhqbZWLTIikindvuTwtmOMXDqQV9pWT5XzWFN7xhHEQzbyFVFEPt0L1hZ7bLGjEX24xonotZMRRBGJLQ4E8Ygz7KM+PbEn9tQTA4PWjGIlY0wksQYNeNsij6Mk1fkUX65zBj9OYIMdpaiBO7kt0n9aeJdVzKQdJ9lJGMFPtzFyyGKH49DrDBs4HpvfSvHLZ7/z1ZAlLB23ksZv1aVV/yYUKRv/lPYHXI/+0CFxEYRzGzM2Jk9LYWF6KylXV/NnZJgrMBCWLzev+vj8+TBpkmXPL4QQQsSklE5QL++Cy7t1wvr4euLHGLZ6/WqRmlCklp4W7JzE4qLihSMJrDVlzQq//qoLHsVXWMnZGWrUgJEj495nAUop5gxazJ7f9jPgi5406V4vVc5jbQYGbzGTRvRhPbM5xU7AoBx1aUhvPqcTt7lIBM+KAERGf/83P/OQm4xjY5wqts0YyFE2c4Lt8SaxDrjwLj+QlVwWfTwFKU1BSlu0z7TigBNj+IPz7Gcdn3OV49jjSHXa0Yh3cLfLAx2hbscanNx3hrXzN7F+0VbWfLWRCnXL0qJPY15pVx0Hx2fb0riQ1ax9cg0M3EgnRRtOnYIpU/Q+z6CXCdSoAR98AI0bW+YcJ07oNa+mhITo4k6SwAohhLC0/67ohPXSLv3VVMJq66j3XH0ywurhA45uaROryDAkgbW2pk31Pq+jR+uCR46O+s2skxMMHw4jRqTaHpLff/wLfyzcQuf329B2yKuJtg0lmH38xGHWE0kEJahOA3pZPDlLTQUoRW/mxrptLbO4y5VYyWtMYQRzln0cZgNVaRHrPltsGcUa/uBz1jKTUAKxwZZwQimBD92YRmlqptrjyagMDEpSnWGsSrRd2RqlKVujNP1mdWfT0h2sW7CZqd2+JFvurLz6TiNa9W9CroI5KU9Ds9YjO+DCK3Sx1MNIvm3b9N6woaH6b/2JXbugTRsYNUpXDE+pqCjz52RGxb8vrxBCCJEkj67HSFh3wsOribd3yKJHVZ9MBy5YRQouCZMMlQGLJHh7eytfX19rh2F5N2/qasPOznrNaypucfH7/E3MHvgNjbvXZeSSgRiJvNH15Xe+pBugnlbzdcAZhaItY+jABxly+wOFoi8e/McNk21fojaT2Jng/VFEcY0ThBFETgqRgwKWDFUAUVFRHNp6jLXzNvL37wcxbAxeblmVV99pxNGmS9hhuyTeKsQANtiSn1J8zgnr/q7evQuenomve3d11SOzKd0j+b//oEABPcKaGHt76N8fvvwyZefL4AzDOKiU8rZ2HBlZSl+bQ8Ijuf5xOYrb54i2XQAAIABJREFU3NQ3DNwPuTPmbBMhMg3/W8+S1cu7Ta9hdciik1XP2npKcL4KYCvjaSJ+Cb02y29MepI/f5oUUtn5yz7mDFrMyy2r8t6i/okmr0fZyud0ipMYPLm+hk+xwYb2jE/VmFNDCIE84o5Zba/wb6L322BDEbwsEZZIgI2NDd5NKuLdpCI3L97mj4Vb2LRsB3vXHCB3oZw49KhLZPcDRBa7H+s4OxxwJRtjWW/9D1oWLIg96hqfwED4+OOUJ7DZs+uR3l9+SXyE1dYWBg9O2bmEsBBl7b9RIUTiAu5Gr2GNHmW9b2KrQHtXXSHYszYUrQP5K0rCKlJMfoMymeN7TjPtzTmUrVmKcauGYWuX8CivQrGI/gmOagGEEsSvTKYpA3EjYy2q12tazZ2BYHqNpUhAZCRs3KgLBV2/DjlzQu/eeu23g0OyusxfLC+9p71B90md+Pv3g/yxaCuHPnmA+rg+TnUeY/fWRZw63MY+KzSiL68xEnfyWPiBJcO335oeEQXw9dXFndzdU3a+Tz+FzZt1X/HNtnFxgbffhhIlUnYeISxAqhALkQ6FBepiSxd3wIXtpre1sXOCQtWfJawFq4Bt6iyFE5mXJLCZiN/ZG0xoM508hXMxafX7OP2fvfsOj6LqHjj+vbvZ9AQCIaH3jhSlSO9IUUBBBcGGKCKCr/qz9/oqiu1VFLHRVMRGUaR36VVAWuihl/Syu9md3x8TIGWT3SSb3U1yPs+TBzJzd+ZESbJn7r3nBOe/x+AI24jjjNPrKgysZiY385i7QvWIAIKJpAbnOeZ0bENKX3Vmj4iNhR494Nw5ve/oFZs3w3/+AytWFKlFlMnfRJeh7ekytD3nT15k+ay1LJm+ktgHw7E+5k/XOzvQ+sE+hHeohE9M7GT9b5AfkwkSE4uewNauDRs36jOxp07pxeI0TU9c7Xb9/8FbbxXtHkIIIUoPWwac3qEnrEdWwsnNYM+nmr3RH6q3y0xYu0D1NrKHVRQ7SWDLiLhz8Tzf/20MBsV/F75AeMUwp6+J5V+XllxaSOUo290RpscN4hlm8lS+rXACCGEQxVMJulRLSYFOnfTEKeey2aQkSE6GLl30arluWDofVSOSu56/jeHP3cq+jQdZ9O1KVs5ex9Jpq6nZpBp97ulGz5FdiKrhxcJj0dF6Mu+MzQYVKrjnno0awf79sGED/PGH/v+lUSO9D3VEyVo1IYQQws00DS7FXJthPbYWzIl5jzf4QbU21xLWGu3AlH+/diHcTRLYMiAtJZ2XBr5L3Nl4Jq18jar1Krv0OiMmXJ22ytkntaToyWhWMZ0T7MJC7qWdAQTTmoFcR08vRFfCzZoFly7lvedT0/Rk6uOP9aWubqKUulbB+MP7WPXTehZPW8k3L/zAty/+SMvuTek1siudh9xIaPkQt93XJRMm6H2dk5PzHqMUDBigF3NyF6WgY0f9QwgfleuBaQksMilEiZB8Ho6szpxlXQWJsfmPj2oKdXtA3e56ASZpayO8TKoQlyI2bOxmGec4gj9BtKAPEfYqvHnnh6yfu5nXfn+GDgNdL7J5iVgm0ACrg8Quq0DCmMAM2nFrUb8ErzCTypc8zEZ+xoAfNqz4EYCdDPoxnhG8g5HiqwhdajVuDAcOOB8XHg7x8cW+Ae704bMsn7WWZbNWc/rwOUz+frTp14oewzvTfmBrgkICC39xTYO//4Y9e/Tlv126QMOGucelpEC9enD+fN5vzoODYd06uP76wscjCkyqEBddUX83WzLsHH/jOhoYMvtEjtsEUY3dFJ0QZZg1DY79rS8JdmUfa1hVqJeZsNbpBmHRnohSiFykCnEpt4LvmMUzZGDGRgYKI3ashLzaj2O/mRj7wX0FSl4BKlKdZnTnH5ZiJ+/KqSYCaM0tRf0SvCaAYB5jJvfzEdtYQArxlCOaNgwiCHnKWGixTp7oXpGaCmlpeuJWjKrWq8w9r97B3a/czsGth1n54zpWzVnPhvlbCQoNpOvtHehzXzead2mCwVCAol2LF8OYMXD5sj7bbDDof15/PcyYkb1AUkgIrFmjJ7gpKdnb6QQE6BWBf/hBkldRJkkRJyHcRNPg4kGIWaZ/HF8PGflMRgSE68uB63bXPyIbyDek8GmSwJYCf/ARs3kp1z5Oyw81uPC2ifAHLtDn8Q6FuvZYvuZZbiCJSw6T2ACCeYpfMZaCf0rhRNKDUd4Oo/Tw98+/3+kVdnuhqxEXhlKKRm3r06htfcZMupfda/axdMZq1vyygcXTVlK5diV63NWZniO6ULtZjfwvNn8+DB+uJ+A5bdoEbdvqFYXr1bt2vGFDOHwYZs6ETz+Fs2f1xHbkSBg/HqpXd+8XLIQQovRLT9CXBR9eDjHLIeFk3mMNJn3vat3u+kfVG6S1jShRZAlxCXeJU0ygfq5lvhkbK5DcoxvG9pcpv3gjvfwf4CE+L9Q9LnOar3iEXSzBDz3RsGGlOs14iM+pT9sifx2iFLr3Xn020Unf0wuda3J57Wwa0t6rfVrTUtL5+/fNLJu1hh3L/sFu16jbohY9hnei27COVKmTYwlVejpEReVfWdhggK5dYeXK4g1eFJksIS66ov5uttrsHHs96xLijRDVxE3RCVHK2O1wdpeerMYsh5ObQMvn921kI6jfS9/LKvtYRQkhS4hLqSV8Qc5epvZTgaTc1hFD9TRCftmAzd/CamZwL5MIoODLNCtQlWeZRzznOMRGbGRQg2ZUQ/YmlUQ2bKSSgB/+xbtE+qmn4JdfHM9OZkoPga+fO8le+lCBqjzN79Sg8G11iiIoJJDed3el991diTsXz+o5G1jx41q+eeEHvnnhBxq3q0/3YXoyG1m1gv61OXsAaLfrbWyOH4datTzzhQhRQilc78wtRJmUfEHfxxqzTE9aUy/mPTYgHOp2g/q9oV4vKO9kRZEQJYgksCXcLpZgxXz1c82iSLmzA1qyH6HL1mCoaAHAgJFY/qUehZ9gKE80bRlc5JiFd8RxlgV8wDK+xIoFOzaq0ojbeI7OjMBAAfZ9uqJFC73H6Msv6/tcc0gPhhUPwPabNSCFM8TwIh15ly1UxUEBJA+KiC7PrRP6c+uE/pw5eo7Vczawes56pvzfdKY+PYO2/a+nb/Je2ien4LQ9u8mkJ7GSwArhlOy6EyILux3O7ICDS+DQYr0/a36qtNIT1vq9oHpbMDr9DSVEiSQJbAmXc19q2v+1xLahIsE/bcDYLDHHWLsnQxM+5DQHeYmOpJFEBparx2PZy1TGspnfeYI57q+2/OST0KABvPgiWswhUv3TMWZAQhT8/Aqsvi/rYI10kviWx3iJRe6Nowiq1Ilm+LO3MvzZWzl54BRLZ6xmyfRVbDptphy30IsT3MQx6pGQ90Xs8r0nhCuuLh8GiDsmS4hF2ZOeqM+yHlwCh5ZAyvm8xwZH6slqvV5QryeEVvJcnEJ4kSSwJVxD2nOC3diwYplVE8vk+gQ8cRD/O05lG5eBxeuzWsI7bNh4kz4kcxnNwQI9MynsZBHzeI8hPO/+AAYOhIEDWXzkKTac+5T4CAunG+FwqkVD419Wc5nTVKCq+2MpohqNqvHA2yO47/VhbHvsbRZ9tYQFGXX5TTWgnhZHX47TixOEZ3lIQEaGPhsthMiXyln1dOEz0Ki/d4IRwpMuHYaDi+HgIr1isN3qeJwyQo0boX5Pfaa1cku91oIQZYwksCVcfx5jJd9h+SeY1IdvwNj1AoHv7s42RmGgHbcRQnkvRSm8aReLSSbOYfJ6hZlUFjCJwTxdbBWlt9bdw791LU7HmQjgOP/4ZAJ7hdHPSLt3H6fdtHdJzLCxUqvBYmrxuWrFV1pz2nOG/hylNecwNG4Mzbyzr1eIEs3qQhVzIUqiDAuc2KAnrYcWw6WYvMcGR0KDPtCwr16AKUjeywkhCWwJV41GdEm9n7nDjqLKWwmZvRFlupaoKBTBhDOSd7wYZemWTgpr+Z61fE86SVSmAf2ZQGM6ebWq7hWrmUE6+VTKzWQjg4NspAmdiyUOQwGWJ7t9P25xCA+HiRMJf+45BqceZjCHOaKVYzG1WUZN1qrq1FDJ3D5gKL3TLfgHeq5VkBBCCB+TcvFawhqzAiz5/F6u3EJPWBv2g6rXg8HN23uEKOEkgS0F0p5sie3gBcot3UBAZWPm4kVFAMFUoBrPsYBKSAGZojjDIU5zED/8aUh7gggDYC+rmcgg7Ngwo88WHGMn2/mT2rTkeRYSQjlvhk4S+VQpzEKhSM1vH2cRNacXe1mFhdwFnbKykk5tWhVbHG41YYK+fOuZZ8BgoG5yAo+wiwdDj7LWWIufozvz0X8XM+2bjQwe359B4/oSFiGtC4RwJPfjPu8/ABSiSC4dhgMLYf9COLkRtDzqIZiC9X6sDftCg5sg3HdXIAnhCySBLeHWz9vCwqnLuPOpQQzv+SXr+J5THCCIUNowiIZ08IlZwJLqAOv5lv8Qy97MHrgaGVjpzAh68ADvMABzjoRMQ8NMCofZxtv0423We/X/QSS1UBjQnBTxsmMjgirFFkcPRvEjL+Y7xoCRVvSjHFHFFofbPfoo3H8/zJkDO3ZAQACmnj3p2bcvPZRi58o9/DxpPtNens2c9+Zxy9ibGPrEzVSoHOHtyIUQQriT3a5XCj7wp560XtiX99jyNfUZ1gZ9oXZnMAV6Lk4hSjilOetj6IOK2izd6+LjYfp0+PJLuHwZIiNh7Fi49159WaKLLp2JY0yL/yO6ViSfrH8bk7+US3enHSxiEkMdzhj6EYARv6uzrnkJJJRnmEtzehVXmE4dYjOv09NprAaMlCOKmrRgEE9xHT3dvpR3KVOZxhMO/5sqDIQQwfvsIJLS16/u8K5jzJ44lzVz1mM0+dF/dE9GvDiUilUkkfUFeTVLF64r6u9mu13D8EaW/X0hleDpfPYGCuELMsxwbC3s/xMO/AVJZ/IYqKBmez1pbdgPKjWCnIXLhBDZ5PW7WRJYT9uyBW66CSyW7L0xQ0LA3x9WrIBWzpdP2u12XhjwX/as3cfn296jZuNqxRh02WMmlQeJJp3kIl+rNbfwHAvcEFXhaGi8RCeOsC1bC538BBJKA9rzHAvwx71Phdcwi+94nAwsZGBGYQQ0atOKx/mRKGq79X6+5lTMGX6aOI8l01cREOzP/W8MZ9C4vhj9ZI+TN0kCW3RF/d2saRrqdUlgRQmQFg8xy2D/H3BoWd77Wf0C9cJLjW/Wk1ZpcyNEgeT1u1mWEHvSqVPQuzckJuY+l5Kif/ToAQcOQFT+Syjnf76YbUt28djnD0nyWgz+ZrbbrnWGQ267VmEoFM/xB6/RnfMcdSkpTyeZ/azjU+7h//jZrfF05W46MZxdLOE0B/DDn+b0phqN3HofX1WtfhWe/Gosdz4zmMmPfcPnj3/H4mkreXzKGBq3a+Dt8ITwHSXwAbsoxVIu6gnrv/Pg6BqwZzgeF1RBT1Yb3wz1eoB/iGfjFKIMkATWk/73P0hPz39Mejp88QW8+mqeQ84dv8A3z39Pm74tueXhPm4OUgBs5ne3zL4CmNw8g1kYYVRgItvYzO8sYBKnOUQaCfm21rGSzjb+4BxHiaYOqSSyku/4k4+4zGn88KcVfRnMMzTgxgLFY8SPGxjADQwo6pdWYlVvUIX/LnyRtb9u5IsnpvGfji9yx1ODuPe1O6VisRBC+IKks7BvgZ60Hv877yJMEbWh0c160lrjRjDK22shipN8h3nS1Kn60uH8OElgNU3j47FfAvD4lIdzN34XbmHByYMGF/kTRCeGu+VaReWHiY7cSUfuZDcreJ9bSXPSXseOnZV8Ry9G8yIdSSX+atEqG1Y2M5cdLOJWnuUOXvHEl1GqKKXoensHWvdpwZdPzeSn9+axfv5Wnp0+nkZt63s7PCE8KtfvM/n9JrwhIfZa0npiI+T1oLdKK2h8CzQeAFFN5d+rEB4kCayn2O168SZXXLqU56nl369l6+JdPPrJA0TXkr0UxaU2rdjHajKwFvFKil486JaY3OkysfnOvl5hw8JZDvEaPYjnbK5Kxhp2LKQyj4lUpwkduKO4Qi7VQsqF8ORXY+lye3s+emgKT3R5mWdnTKDbnR29HZoQQpR+l4/Cvvnw73w4ldc+7swiTE0GQZOBUL70FRsUoqSQBNZTDAYICACz2fnYQMdLThMvJTHlyWk0vrEBA8fd5OYARVY3MZZFfAZOEthgymHDmquVDuizr48xi3AiiynKwgsiHOVChWG9N2wiiVzItw2PmVRm87IksEXUtm8rpux8n1dvfY+37/qYxMvJDBwr3+uibNKQTrCiGF0+Cnt/02daz+xyPEYZoFYnaDpYn20NL75Wc0II10kC60kDB8Jvv+mzsXkxGmHIEIenpj49k+T4VJ748mGMxpJbsdRGBtv4g83MxUwKNbiOXoymItW9HdpVlalHF0ayjh8dtnwB8CeY5/mD/fzNAj7AQhoGjFgxU5fWjORdmtDZw5G7pjm9sLkwu+xPMCnEubQf+CInOc1BqtLQHSGWWeEVwnhn0Uu8NexD/jfuKxIvJTHihSGOtwscOABffw1HjkDFinDXXdC9uyxlE0IIRxJPw97fYc+vcGqb4zHKCHW76TOtjW+RysFC+CBJYD3p2Wdh4cLs7XNyCgiAp57KdXjPun0snraSYc8Mpm6LWs7vlZQEx46ByQT164Ofb/yvjmEL73ALFtJIz9x/uY0FzOVdunEvDzIZo4/8sxzDFBSKNczCjh1bZguaQEJRKP6PX2lMZxrTmYE8xUn2YCaFSGr6VDLuSBBhdOd+VjENC2kOxygMRFITKy6sGkDfY5vAeUlg3SAwOIDXfnuaSaM/Z9rLs0m6lMTDH9x3LYlNSYFhw2D5crDZwGrVk9Yff4TKlWHRIqhXz7tfhBBFJg9ihBukXIR/58Ke3+D4ehzuaTWYoF5PaDoIGg2A4AoeD1MI4TrfyBTKijZt4P334emnHSexwcF6peLmzbMd1jSNL56cTqXqFRn58u353+PoUXj5Zfj1Vz151TT9zwkT4LnnICjIjV9QwZxkL6/TM9ds3pUEaQ2zsGFlHN96I7xcjPgxlq+4jRdYwhccZxf+BNGO2+jAnQQQlGWskdq09GK0BXcfH3KMnRxjV65ZZj/8CaE8L7CQr3jEpevZsBJGxeIItUzyM/nxzLTxhJUP5deP/6RG42rcPKaPvoJjwADYvDl7VXNNg+RkfTa2fXvYvVtPZoUQoqxJi4f9f+ozrUdWgWbLPcbgB/V6wXVDoFF/CCzn8TCFEIXjlgRWKdUP+AQwAl9rmvZujvPdgXnA0cxDv2ma9oYrry11xo2DZs3gjTdg3To9ubRaoVs3eOUV6Jx7yemaXzZycOthnvp2HEEh+bRk2bNHf31Skv4mN+ub2/feg/nz9XuGeKcn2QyeIp2UPM9bSGUdsxnMsz7VEzSaOtzDe94Ow+38CeQ1VrGEL5jP+yRxCQMGDBjpxRhu5RnKEUVvHmI/65wuIy5PZarR2EPRF79UElnNTDbwE2ZSqU5T+jOB+rT1WAwGg4FHPr6fEwdO8fnj39GsU2Nqn9wL27fn3ZLLboeEBJg4ET76yGOxCiGEV1lS4eBfsPtXiFkKNgddH5QBaneB64bqhZhkplWIEklpRWwUrpQyAgeBPkAssAW4S9O0f7OM6Q48pWnaLQV9rSNt2rTRtm7Nq0pcCZKYCHFxUKEChIU5HJJhzWB0syfwDzAxZef7ee99tduhdm2Ijc27+XtgINxzj97Ox8PiOMuj1MHqpD2NERN9eJjRfOqhyASAhkY857CTQTmi8cN09ZyNDMZRmzhO51m5OIAQHuILunGPp0IuVjtYxAfoqx3MmQ9dFAb8CaQxnXma3wkg2GPxXD4bx8OtnqZ8VDifRe4mYOUy5y8KC9MrmptMzscKlFLbNE1r4+04SjK3/G5+7dosmBZSCfV0TBGjEqWa3QbH1sKu2XrrG0seD1prtNeT1qaDISzaszEKIQotr9/NzsuQOtcOiNE07YimaRZgNjDYA68t+cLDoVatPJNXgL++Xs7pmLOMfmdk/oWbli7V2/Tk90AiPR1mzdJnaD3sLIcwEeB0nA0rR8ijsIIoNgpFBJWpSPVsySvoS6lfYyVhRGIi9wqAAILpyyOlJnk9xGYmMRQzKVeTV9BbBplJ5V/W8B63eTSmCpUjeGb6eI7tOcmXmxzvWc7FZoPz54s3MCGKleyBFXk4txeWvAwfXQczBsOuH3Mnr1VaQp834PE9MHox3DhGklchSgl3LCGuBpzM8nkscKODcR2UUruA0+izsXsL8FqUUmOAMQA1a9Z0Q9i+Lz3VzKw3f6F5lybcePMN+Q/++WfXElOTCVauhEGD3BOkiwz4udR3FPRZWGfOc4y/+IzN/IYVM9HUYyBP0ppbfKYIVGlShQZ8zD4W8wV/8SmJXMCAgevoya08R3N6ejtEt5nFM3lWngawks4B/iaGLR5dTty2byuGPnELv370B/0pTwOc9JW22XymeJsQQhRZ0lnY/TPs+gnO7XY8pmIDaHEnNBsCkfU9G58QwmPc8e7G0SPSnJnKdqCWpmnJSqkBwFyggYuv1Q9q2lRgKujLlAofbskx77NFXD4bz8tznnTcQiOrxETXLqpp+VZBtmFjDys4z1H8CaIFvYmg6H3PatMKOw6KKOTgTzBtGJjvmOV8wzeMz1YZOI7THGMH0dTjVVYQhuxrcbcwKnI7L3E7L2HDhgEDqpTNkFziFIfY5HSchTT+4lMmMMMDUV1zzyu3s+TzP5lhbsab2t/5D65cGaKiPBOYEEIUB0uKXoxp14+ZxZgctCEMjoTmt0OLYVD1emkjJkQZ4I4ENhaokeXz6uizrFdpmpaY5e8LlVKfK6UiXXltWZWWks6c9+fRtl8rruvcxPkLGjYEf3+wOChakJWm6cuWHVjFDGbyFBbSsZOBASM2rLSkL+P4tkgVZgMIogejWMZXZOTblkWjJ6PzPLuTxXzLBId7adNJJpZ9vE0/3mFTqUuufImRktuHOD8XOIaJAKd7tTXsnGKfh6K6JqRcCLc+0JWZX6zmBGHUJI9VF8HBerVzeSMnhChpNA1ObIQds/SerVYHxR+NAdB4ALQYDvV7gVH2+gtRlrhjD+wWoIFSqo5Syh8YDszPOkApVVllTiEqpdpl3veSK68tq/76ajmJl5IY+ZKTtjlXjB4NBhf+d0ZE6C02cviTT/iKR0jkAukkZfZpTcaKmZ0s4jnakkxcAb+K7O7ibSpREz/8HZ73J4ixfEUoEXleYxbP5tm3FMCGhVPsYz/rihSrKJtMBKLh4Am/A/4eLOKU1cBX78FkVPzql8eDraAgaNsWxozxbGCiRFFK9VNKHVBKxSilnstjTHel1E6l1F6l1GpPxyjKmKSzsO4j+KwNfNcPds7KnbzW6gyDPoWnD8Ed06BRP0lehSiDijwDq2lahlJqPLAYvRXOt5qm7VVKjc08PwW4HXhEKZUBpAHDNb38scPXFjWmks5ms/HbJ3/SvGsTmnV0sZ1MnTowdCj89huk5ZHgBQfDpEm5ZmXiOMP3PJfnrFMGFi5zip94pUjVgYMJ5x228C3j2cAv+OGPhoaGnfJU5gH+xw0MyPP1ZznMGQ46vU86KSxiMk3oUuhYRdlUixYYXJhdDiCETgzzQES5RUSVo+c93Vn54xrGVzyJKTWzcIlS+r7Xhx7SW+hI9WGRh8wOAJPJ0gFAKTU/R/eA8sDnQD9N004opTy/Hl1WEJR+NiscWgLbZ+p/OurXGtlQXx7c4k4oXzZqoAgh8ueWCh+api0EFuY4NiXL3z8DPnP1tWXdpj+2c+74BR7+4L6CvfDbbyElBZYs0SsO2zNnkgIC9DcCEyfCnXfmetkSvnB66QwsrOQ77mZikdqHhFCOCcxkFP9jH2uwkE5l6lOXG5wu+b1ELH745zsDq9M4f7XlsBCu88NEfx5jPu87/XfW1YNVl82kkoGFIMIxYKDj4LYsnraSvd8vopV/Apw6pVc1795df1AlRP6udgAAUEpd6QCQtYXdCPSe7ScANE2TktbCfS4cgB0z9fY3KRdyn/cPg+ZD4fp7oFpreZghhMhGSlT6oHmT/6JSjYp0HFTAloT+/vD777BlC3z4Iezcqc/CDBgAjz4KNWo4fNlOFjvd8wdgwEgs+6hH64LF5UAoEbQtYMekYMJdKgQFEEL5woQlBEN4kT2s4AjbHCax/gTzFL8SRN7tr9zBjp31/MTvvEssezFgxJ8g+jCW3j3H4WcysmXxLlq9VzraFwmPcqUDQEPApJRaBYQBn2ia5tmqZUXsUy98jCVV39O6bRrEbnY8plYnPWltOgj8QzwanhCi5JAE1scc3xfL9mW7eeDtERj9Clkop21b+PFHl4fbyHBpnEK5nEAWh5q0IIBg0smjUXmmQMLoRgFnr4XIZMKfV1jO7/yXhfwPGxkYMGDFTAPaczcTaUC7Yo3Bjp2PGMYO/rrai9aOjQws/MlHrAj7hoZdR7P5rx08JAmsKDhXOgD4Aa2BXkAQsEEptVHTtGz7OMpiiztRQBcOwNbvYNcPkJ6Q+3xYFWg1AlqNhIr1PB+fEKLEkQTWx/z55VJM/n70f9BzfTUb0p4T/OM0kbVipgoNPBRVbkaMDOYZfuIVzPn06fTDn/YM9WBkorQx4c+dvMYQXuQEu7GSTiS1qEg1j9x/Hu+xg4UO/51nYCGZy6T2WM2ll6qTkpBCSDmZqRAF4koHgFjgoqZpKUCKUmoN0BKyFyIo1hZ3smy05Moww74FeuJ63EFRRYMJGvXXZ1vr9QSjvB0VQrhOfmL4EHOamWUzV9NpyI2Ur1TOY/ftz2Os5Lt8E1iFgbYMyrdCsCfczBMcYD07WXx1ZuoKA0YCCOElFmMiwEsRitLEDxN1ucGj97SRwXzez/chjYYdW5MLQHViD52lURuZtRAFcrUDAHDAPH2IAAAgAElEQVQKvQPAiBxj5gGfKaX8AH/0JcYfeTRKUfJcPgrbp+tFmVIv5j4fUQfajIKWIyC0kufjE0KUCpLA+pC1v24iKS6Fmx/q7dH7VqMR3biPNcxw+KZZoQginBG869G4HDFg4El+ZhXT+J13uMhJjPihYacTwxnKS0RT19thClFoB9no0rJ+W0O98EnsgdOSwIoCcaV7gKZp+5RSi4B/ADvwtaZpezwbqczAlgh2m15BeMvXELOcXKvRlVHv2drmAajT3bWWf0IIkQ9JYH3I4mkrqVI3mpbdm3n83g8ymUBCWMRkFCqzeI0ikBDKU5nnWEA0dTwelyMGDPTkAXowikQuYCGdckThT6C3QxOiyFKJd1qRG8BQPxmURuzBnCs/hXDOWfeAzM/fB973ZFyiBElPgB3fw+YvIe5Y7vPh1eCG++CGeyC8qsfDE0KUXpLA+oj4CwnsWrmXES8OQXlh348BA/cyidt4njXM5BQHCCSUtgyiMZ1dekPtaQpFOTzfmlCI4lSeKi4VSzMEQGBlAxdPXfZAVEIIkeniIdj0Jez8AawpOU4qqN9bn21tcJPsbRVCFAv5yeIjNv6xHU3T6Hxbzk4GnhVGRW7mca/GIERZVpcbCKWC02rb/gThF1IOc5rZQ5EJIcosux0OL4dNUyBmWe7zgeX02da2oyGitsfDE0KULZLA+ogN87dQqUZF6rWq7e1QhBfEcYbdLMdCGtHUoxndMSD7hMoihWI4bzGVsVjyKORkxJ+aNOdScDnMqRYPRyiEKDMsKfpM66YpcCkm9/lKjeHGh6HFMOnbKoTwGElgfYA5zcy2JbvoO6qHV5YPC+9J4AJf8AD/sBQjJuzYMWAgkFDu40M6c5e3QxRe0I17OM9R5jKRDMzZlhQHEkolavM8f/JSyAekp8oMrBDCzZIv6Htbt3wNaXE5Tiq9Bc6ND0OdbtLuSAjhcZLA+oDda/djTrPQ/pbW3g5FeFASl3mONsRxBhtWrFxLRNJJ5gseJIV4+vKIF6MU3nIHr9CGQfzBh+zkLzKwEk09BvM07RiCCX+UQWG32b0dqhCitLgYAxs+hZ0/gi3Hw7GAcL1va7sHoYJU+xdCeI8ksD5g54rd+JmMXNelibdDER70A89fTV4dsZDKdJ6kPUOlWFUZVYdWTGBGnufTU8yEVQj1YERCiFLpxCZY/z/Y/ye52uCUrwUdHoVWIyFAft4IIbxPElgfsGPFHpq0b0hQiLSBKSvSSGYNs/JMXq9QKJYyldt5yUORiZIkPcVMoPzcEEIUht0OBxfB3x/DyU25z1e9Hjo+Bk0GSTVhIYRPkZ9IXpYcn0LM9iOMfOl2b4ciPOgkezC68O1nIY2dLJIEVjiUnpJOYHCAt8MQopjI3spiYbfBv/Ng7Qdwbk/u8w1u0hPX2p1lf6sQwidJAutle9cfwG7XaNm9mbdDER6kF+Vx7Y2B5kJPUFH2aJpG4qVkwiKk8qcQwgU2K+z+RU9cLx3Kfs5gghZ3QscJECXbmYQQvk0SWC/bt+EgBqOBhm3reTsU4UFVaUQG6U7H+WGiPt7tDSx8U9LlZKxmK5HVK3ongBMnICEBqlSByEjvxCBKOc35EOFchllvhbPuI4g/nv2cKQTajNL3uIZX9U58QghRQJLAetm/Gw9St0Ut2f9axoQTSSv6s5X5aORdRVZhpD8TPBiZKCkuxF4CILJaBc/dVNPghx/gzTf1BNZkArMZOnaEt97S/xTCTTQ0WURcFNZ02D4d1n0MSaeznwsI19vg3PgIhHjpIZgQQhSSJLBeZLPZOLA5hl53d/V2KGVGHGdZypf8wxLs2GlMZ/rxKFHU9ngs9/A+e1hBGkk4mmkIIJgePEBlZHZe5Hbx1GXAgwmspsGjj8KMGZCSoh9LS9P/XLkSeveG776DYcM8E48oAyR9LRSbFXbMgjXvQ+Kp7OeCIqD9o9DuIQgq7534hBCiiCSB9aIzh8+RmpRGwzaSoHjCH3zMDzwPgDVz+e5RtrOIz+jPBO5mIsqDb5iqUJ+3+Jt3GUgSF0knBdDwJwgNjb48ykje9Vg8omQ5e/Q8AFG1Knnmhr/9lj15zSktDUaN0mdha9TwTEyiVNNkBXHB2G3wzxxY/S7EHct+LiRK39/a5gFphSOEKPEkgfWiE/v0J6O1m1X3ciSl3wq+YzYvXk1cr8jAAsBiJhNACHfyqkfjqsl1TOYI/7KarSzATArVaEJX7iYMWdYl8hZ78DRBoYFUrBLhmRu+/XbeyesVdjtMngzvyoMXUXSSv7rIbod/58Kqd+DiweznQipB5yf1fa6mIO/EJ4QQbiYJrBed2K8nsDUaSeGE4mQjg5k8jZnUPMeYSWUeE7mFJwgm3IPR6b1em9GdZnT36H1FyXbq0BmqN6yC8kSbi/h42OOg3UZOZjPMni0JrHALTaZg86dpcGgJLH8Tzu3Ofi4oAjr9B9qNAX+pVC6EKF0kgfWiE/tjqVg1gpByhfvlYiaVnSwiiUuEU4mW9CUAecKa004WY8ucac2PwsDfzKYPYzwQlRBFc/LAaRrf2MAzN0tNBT8/sFqdj72yL1aIIpISTvk4vQOWvAzH1mY/HhCuVxRuPw4CPfswVgghPEUSWC86d+wCVepGF/h1NjL4gedZzOcYMGLHhgE/NOz05zGG8QZGjMUQccl0jsNXlwrnx0wKpznggYiEKJq0lHTOHbvATfd198wNK1Z0fUOi7H8VovjEHYcVb8Lun7MfNwXrVYU7PgbBHqxMLoQQXiAJrBddPHWZhq3rFug1duxMYgj/sByLgyWxC/mYsxziCX7yaEEiX+ZPEAYXEnqFIgBZaiV83/G9J9E0jTrNa3rmhgEBcMcdegsdmy3vcaGh8PjjnolJiLIkLQ7WTILNU8GW5YGsMur7W7s9C6FR3otPCCE8yODtAMoqTdO4fDquwAVYNvEbu1nhMHkFfVnxdhaynYXuCLNUaEEf7Pn0Wr3Cn2Bac7MHIhKiaI78cwKAui1qee6mL70Egfn0qzYaoVIluP12z8UkSrX4OvLzGJsVNkyGT1rBhs+yJ6+Nb4FHN8HNH0jyKoQoUySBLQiLBRIT9Yp/RZSSkEp6qpmK1QpWaXYuEzGTfyVQMynMZWJRwvOoDKykkOBSklkYUdSmMZ0w5rPgQKGIpAb1aVcsMQjhTkd3HycwJIDKdTz4prVhQ1i0CMLDISTHSoWwMKhdG9auzT/JFcKJRYH9r/7dGljGl8LGLIcvOsLiFyA9/trx6m1h1CIY/j1EemgfvBBC+BBJYF2xcCF07gxBQRAZqb+Be+wxOHmy0JdMvJQEQPlKrhdZ0NA4xg6Xxh5mc6Hi8qQ9rOQNejOSIEZTiXsIZQpjOEOM2+81gVmEE4URU65zCgPBlOcZ5smya1EiHN51jDrNa2IwePhHeOfO+s+9iRPhhhugfn3o0QOmT4f9+6FaNc/GI0qdZBV29e+qrFYhvnwUfhwBs4Zkb4sTUQfumA6jl0KtDt6LTwghvEz2wDrz5JMwdeq1/od2u16Jc8oU/U3bypX6G7kCSk3UK3UGhxesarDmYme84prNdJdfeZvf+W+W1jY2bFhZyXf8zQ88z0Ka0tVt94ugMu+zk+95lr+ZjRETCoUVM20YyN28RxS13XY/IYqL3W7n8I5j9L7Hfd8fBRIeDo8+qn8I4WZalrZQZS59taTA2g9h/adgM1877h8G3Z+Fdg+Dn7/34hNCCB8hCWx+fvwRvvxSbyGRk9Wqf/Tpo89IBAcX6NIpifo1C5LAKhRR1OYcR5yOrYLvLivawaIcyes1djJIJ4N3uIXPOUoYBVtinZ9yVGIc3zKKTzjBHjTsVKMJYZTxZWqiRDl9+BypSWk0uKFgBeCEKAmyrYEpKzOwmgb/zoVFL0DS6eznWt0NvV6BsIJ3LBBCiNJKlhDn5/XXHSevWVks8NNPBb70tRnYgiW+g3iaAPJ/TQAhDOaZAsfkKb/whsPkNSs7Gazg22K5fxBhNKIDjekkyasocQ7vOApA/RvqeDkSIdxPy+ezUinuGHx/B/x8f/bktVpreHA53DpZklchhMhBEti8HD0KJ044H5ecDF995fjcxYv6XrGuXaFjR729RIy+v9NqtgLgH5h7T2Z+unEfUdTBD8fLiPwIoAoN6MTwAl3XU1KI5wjbnI6zkFZsCawQJdnBrYcx+ftRq2l1b4ciRDErxQmszQrrPobJ7SFm6bXjIVEw+HMYvQyqt/FefEII4cNkCXFe4uPBZIK0NOdj4+JyH/vuOxg3DpS6do2tW/Ulyfffj73bSAAMhoIVDQogiDdYyySGcoiNZGDFTgZG/DBiojFdeJI5mAgo0HU9JZUEjJjIwOJ0bBoJHohIiJLlwNbD1GtVG5N/wR5+CVEylIFCeic3w4LH4fzeLAcVtH0Qer0MgeW8FpoQQpQEksDmJTpaXx7siqpVs3/+228wfjykp2c/fmXf7IwZ2I/rRZYMxoJPgocSwWusIJZ9rOV74jlLBFXpyt1UpWGBr+dJoVTAhtWlseFIXzshsrJl2DiwJYa+9/fwdihCFI8sRZxK3QSsOQmWvgpbvyXbFxfdHAZ+AtVbey00IYQoSSSBzUvVqtCiBWx20o4mLCx7NU5Ng//8J/+9s6mp2JctB65HFaENRnWacBdvFfr13hBEGM3pzQ7+Ir93J4GE0p/xngtMiBLg6J4TpKeYadK+gA+qYmNh8mT45Rf9wVr9+vDEE3DzzWA0Fk+wQhSRIcOFFVAlxZFVMG8CJGTZmmQKhh4vwI2PgFHejgkhhKvkJ2Z+3n4bBg3KexmxwQAVK+pjrvj7b335sRN+Sk/ebNYMd0RaotzBq+xlJRbyenOiCCCYTtzl0biEcIUVM/+wlHjOEkIELelLEKEeufe+jYcAaNqhAAnsrFkwZozeAsyc2ZojNlbf0lCvHqxYARWkmJnwDRVtF6/+vcqeKXD7RC9G4wbmJFjyMmz7Lvvxhv1gwPtQvqZ34hJCiBJMEtj89O4NH32kz1RYrZCRJdkMCYGICFi1Cvyy/GfMLNLkjL8lDRSY01xcplyKNKAdjzKNyYzCjpWMLEuKAwgmgBBeZzWBhHgxSiGy09D4jf8yj/cADTt2DBiwYaMXo7mHSZjyKK7mLvs2HqR8VDkq13Fxef2KFXry6ughXHIy7NsHffvqK01UGdh7KHxez/SlzgeVFIdXwvwJkHDy2rGgCBgwCa4bKt9zQghRSFKF2JmHH4bt22H0aKhUSV8y3LgxfPAB7N8PtWplHx8U5NIvpQBsAFjKYAIL0JE7+ZDd3MQ4yhFNEGFEU4+7eIf/cYhqNPZ2iEJcpaExhYf4nf+SRiJpJGEmhTSSsJDKcr7hvwzARvGuqNi38SBNOzREufrG97nn8i9EZ7HoP8fWrHFPgEIIsKTAH0/AzFuzJ6+Nb4FHN0Pz2yV5FUKIIpAZWFc0bgxTpugfzvTooc/WOuEfFADpkJ5aNhNYgGjqMoqPGcXH3g5FiHz9yxr+Znae/YstpHKIjazle7pzX7HEkHgpidiDZ1wv4HT8OOze7XxcSgp89hl061a0AIUQcHon/PogXDp07ZjMugohhFvJDKy7RUXphVH8819KGBoeCEBKfIonohJCFMF83s8zeb3CTApzebfYYjiwRd+e4HIBp9hYCHChnZamwZEjRYhMCIHdrvd1/bp39uRVZl2FEMLtJIEtDlOnQrVqjpNYpSAsjHKzvgUg4WKSh4MTQhTUftbhSk+P0xzEirl4YtgUg1KKBq3ruvaCsDCw2VwbW076TgpRaAmnYOZgWPYq2DNXYJlCYPDnMGwWhEpLOCGEcCe3JLBKqX5KqQNKqRil1HMOzo9USv2T+bFeKdUyy7ljSqndSqmdSqmt7ojH6ypUgG3bYNQoCA6G8HD9DWJgoD47u2UL4d3aA5BwMdHLwQohnLFjd2mcQmHHxaSxgPZvOUStptUJDgty7QXXXacXm3MmNBTuvbdowQlRVu3/E77oCEez7COv1hrGroXrR8qsqxBCFIMiJ7BKKSMwGegPNAXuUko1zTHsKNBN07QWwJvA1Bzne2ia1krTtDZFjcdnREToe2bPn4dFi2DBAn1P2oIF0KgRfiY/QsuHEH8+wduRCiGcqE4Tl8aFEYk/LiaYBaBpGgc2x9CobX3XX2QwwDPP6A/R8uPnB8OGFS1AIcoamxUWvwizR0B6Zus8ZYCuT8MDi6FiPe/GJ4QQpZg7iji1A2I0TTsCoJSaDQwG/r0yQNO09VnGbwSqu+G+JUNICHTo4PBUZPUKXDx12cMBCSEKahBP8zn3k07ee9b9CeIWnkTh/hmXCycvknAxiYZtCvim+PHH9d7UixfrxZqyMhr1qul//aX/KYRwTeJp+HkUnNx47Vi5GjBkKtTq6L24hBCijHDHEuJqQJY68cRmHsvLaOCvLJ9rwBKl1Dal1Bg3xFNiVK4dxdlj570dhhDCiXbcSk2aYyLQ4XkjJspTmZsYWyz3P7T9KIDr+1+vMBjg55/h44+hXj19G0NoqP7n8OH6Vof27YshYiFKqcMrYUqX7Mlrw37w8BpJXoUQwkPcMQPraLrBYbUTpVQP9AS2c5bDnTRNO62UigKWKqX2a5qWqylhZnI7BqBmzZpFj9oHRNeqxO61+7wdhltoaBxjFwmcI5QK1KU1BqkRJkoJI368zDI+4S52sRQ7NmxYMWDEjwBq0YJnmUcw4cVy/0PbjmAwGqjbohA/+wwGePBBvZf1mTOQng6VKztfWiyEuEbTYO0kWPE2V9/iKAP0fBk6Pa5/nwkhhPAIdySwsUCNLJ9XB07nHKSUagF8DfTXNO3SleOapp3O/PO8Uup39CXJuRJYTdOmkrl3tk2bNs7LgZYA0bWjSElIJfFyEuEVwrwdTqGtYRazeYlELmLEDzt2AghmCC/QnwnFsqRSCE8LJIRnmc9ZDrOaGVziJOWIojMjqEWLYr33oR1HqNmkGgFBLrTFyYtSULWq+4ISoqywpMDccfDv3GvHQqPh9m+hdue8XyeEEKJYuCOB3QI0UErVAU4Bw4ERWQcopWoCvwH3aJp2MMvxEMCgaVpS5t9vAt5wQ0wlQs0m+krrE//Gcl1n14rEFAcrZjbyCwv5H3GcJohwunM/PRlNGBXyfe1PvMp8JmHJ0SMznSR+4HmOsZNH+EaSWFFqVKYew3jdo/c8sus4rXpe59F7CiGA+BN6oaazu68dq90Fhn4DYdHei0sIIcqwIq950TQtAxgPLAb2AXM0TdurlBqrlLqyIewVoCLweY52OdHAOqXULmAz8KemaYuKGlNJUbdFLQAO7zrutRjOc4wJNGAqY4lhM5eIJZZ/mcNrjKMWe1iZ52tj2OIweb3CTCrrmcNW5hdX+EKUeomXkrh46jJ1m9fydihClC3HN8DUHtmT13YPwz2/S/IqhBBe5I4ZWDRNWwgszHFsSpa/Pwg86OB1R4CWOY+XFZHVKhBeMYzDO4955f5m0niZzsRxBi1Hn8srSem73MJEtlGNxrleP5/3sZLu5B4p/M67tGWw+wIXogw58o/+gKtuS0lghfCYHd/Dgv+A3ap/bjDBzR9A6/u8G5cQQgipsuNNSinqtapNzI4jXrn/3/xICvG5ktesrJj5jbcdntvFknxfe0UMm7FhK3ScQpRlVxLYOs1LR/E6IXyapsGqd2HeuGvJa3Ak3LdAklchhPARksB6WaM29TjyzwnMaWaP33shn2DOp68lgB0bG/gFK7njy8Dq0n0UCpuLY4UQ2R3bc5LwimFERJf3dihClG42K8wfD6veuXYsujmMWQW1HPdzF0II4XmSwHpZkw4NsWXYOLTN87Owl3MXi3ZIoUgmLtfxSrg2IxRMOfzz6J8phMjf0T0nqNuiJkpJITQhio05CX4YBjtmXTtWryc88BeUr5H364QQQnicJLBe1qR9QwD2bTzk8XsH4FofSBsZBBKS6/hAniLAwfGsTATSj/GFik+Iss5ut3NszwlqXyfLh4UoNimXYNotcHj5tWOtRsKIORBQclvcCSFEaSUJrJdFRJWjSt1o9m444PF7d+Iu/HDeV7IuNxBE7l/inRlBBFUw5lELTGEgmHD6M6HIsbqLhXRWM5PJjOJT7mMJU0gjydthCeHQuWMXSE8xU0cqEAtRPJLOwrQBcGbntWPdnoXBk8Fo8l5cQggh8iQJrA9o3rUJe9buw253XhDJnfozHgPGfMcEEMIQXszjXBBvso5qNCWQUMjS6zWQUCKpwVusJ5xId4ZdaJv4ndFE8TXjWMU01jCDGTzFg0SzhCnOLyCEh508oC/zr9m4qpcjEaIUij8B3/aDC/szDyi45WPo8QLIkn0hhPBZksD6gJbdmpFwMYnj/8Z69L4Vqc5jzMSfYLImn1cEEEJfHqENA/O8RnmimcROnudPOnMXTehCe27nSebwGUeoTL1i/Apct52F/I+RpJNEOslXj5tJwUIa0/k/lvG1FyMUIrfYg3oCW72RJLBCuNWlw/Btf4g7qn+ujDD0a2gzyrtxCSGEcMotfWBF0bTo1hSAXav2UsfDe91uZAivU4M5vMZuluOHiQysVKcpt/MyN3Kb02soFE3pSlO6eiDigtPQ+JIxWEjLc4yFVKbzBF25WwpOiSJLJo4VfMs2/sCGlXq0oR/jqUL9Al0n9uAZQsuHUC4yvJgiFaIMunwEpt0MSWf0z43+cMc0aHyzV8MSQgjhGklgfUDl2lFUrl2JnSt2c+v4/h6/f33a8gJ/kkICiVwgmHDKEeXxOIrLXlaRSoJLYzfxK10YWcwRidJsNTOZysOAwkIqoPdCXsqXdOc+RjMZo5Ol+1ecOnSa6g2rSAViUWb8WO5B7kooxtUw8Sdh+qBryaspGIZ/r1ccFkIIUSLIEmIf0aZvK7Yv243F7L1+qSGUowr1S1XyCnCSPdjIcDounWSOscsDEYnSaht/MJWHsZB2NXkFsGHFSjprmMl0nnT5emePXaByndL1/ShEfvYGtc5+wJ21IRLPwPSBkHBS/9wvEEb8JMmrEEKUMJLA+ogbb25NWnI6e9bu83YopY4RE8rBHt+8xgpRGBoa3zAh36XqZlJZxpfEcdbp9ex2OxdOXCSqZiV3himET9Nyvi1Jj3fPhVMuwYxB1/a8Gv31mdc6vrn1RQghRN4kgfURLXs0wxRgYtOf270dSqnTxMW9uYGE0ozuxRuMKLVi2EwiF1wYqVjJd05HxZ1LwGrJILqWJLCi7NByPmx0x/J5Syr8OAwuHtQ/N/jBHdOhfu+iX1sIIYTHSQLrI4JCAmnV8zrWz9+CpmneDqdUqUFTqtPM6SxsMOVoTi8PRSVKm7PEuDTTbyWdk+xxOu78iYsARNfyjTZUQpRIdhv8+iDEbsk8oGDIV9B4gFfDEkIIUXiSwPqQbnd04OzR8+zbeNDboZQ6jzGLIMLzTDACCOH/+AWDfEuIQjIR6PJS9QBCnI6JO6svnaxQJaJIcQlRkhhUjge4RXmgq2nw17Nw4M9rx/pPhOuGFP6aQgghvE7erfuQzkNuxD/QxLKZa7wdSqlTjUa8w2Ya0RkTgQQSRiBh+BNEHW7gdVbTkPbeDlOUYE3pRgbOi7AFEkZbBjsddzkzgY2oXL7IsQlRUhjIUbTJbiv8xTZ9CVu+uvZ5h/Fw48OFv54QQgifIG10fEhIeDAdb23HqjnreeTj+zH5S0Ehd6pKQ95kDec4QgybsWOnNq2oQVNvhyZKgXAiacMgtjCXDCx5jgsgmFb0c3q9uLPxKKUoX0l6wIqyw6DlSGC1Qiawx9bB4heufd7sNujzZuEDE0II4TNkBtbH9L67K0mXk6WYUzGKpi6dGE4XRkjyKtzqIb6gAtXww9/BWUUgoTzPHy71gY07F09YhVD8TPKcUZQduWZgcya0rkiIhTn3XUt+q7WGW6eAQd7yCCFEaSA/zX1Mm5taUqFyeZZMX+XtUIQQBRRGBd5jOz0YRQDBBBNOMOUwEcj19OO/bKIebVy6VlJ8CmEVQos5YiF8i5Ece14LuoTYmg4/3Q2pehE0QirBnTPBFOieAIUQQnidPNr3MUY/I73v7sovH/1B3Ll4IqJl/5sQJUkI5RnDFO5hEsfZhY0MqtKICCoX6DrJccmERTgv9lTqmc1w4oQ+e1azJphka0VpZiBHwlrQJcTLXoPTOzIvltkup1w1t8QmhBDCN8gMrA+66f7u2G12VvywztuhCCEKKYhQGtOJZnQrcPIKkByfSkj5MpzAXrgA//kPREbCDTdAy5YQFQUvvghJSd6OThQTQ84Z2IIsIT64BDZ9ce3zm96G2p3cE5gQQgifIQmsD6rVtAaN29Vn4dfLpCesEGVUSkIqweFB3g7DO06d0hPWKVMgOVn/SEmB+Hj48ENo3Rri4rwdpSgGZhWQ/YDdxQQ2+TzMG3ft80YDpOKwEEKUUpLA+qiBj/TlxL5T7Fixx9uhCCG8wJxqJjA4wPlAX3P0KCxfDuvXQ3p64a5x2236DKzFQTXn9HQ4fhzuv79IYZZ1Sql+SqkDSqkYpdRzDs53V0olKKV2Zn684om4TgQ0zH4g7bLzF2kazHsUUi7on4dWhkGfgXKtL7MQQoiSRRJYH9V9WEfKRYYx99OF3g5FCOEFljQL/oGOqhn7qL//hvbtoWlTGDoU+vfXl/w+/TSkpbl+nd27Ye9eyMjIe4zFAosXw+nTRY+7DFJKGYHJQH+gKXCXUspRSfa1mqa1yvx4w0PBZf98xVvOX7NrNhxacuUCMORLCKno9tCEEEL4BklgfZR/oD8DH+nLhvlbObH/lLfDEUJ4mCXdSkBQCUlg//gD+vSBTZv0GdKEBEhM1PeqfvYZdO3q+mzs3Ll64SZn/PxgoTzgK6R2QIymaUc0TbMAs4HBXo4pU44E9sKB/IcnX4DFz1/7/MaxULe7u4MSQgjhQySB9WGDx/cjIMifOQdtsMAAACAASURBVO/N83YoQggPs5qtmAJKQKH4xEQYPjzvWdb0dNizB15/3fXr2VyoPJuRoe+NFYVRDTiZ5fPYzGM5dVBK7VJK/aWUauboQkqpMUqprUqprRcuXHB/pAYnPZMXPQtpmfuhy9eEXi+7PwYhhBA+RRJYH1a+Ujn6PdCT5d+v4ULsJW+HI4TwILtdw2AsAT+iZ8xwPiY9Hb74wvGe1pzq1IEgF4pX+fvrbXVEYTjaHJqzYuB2oJamaS2BT4G5ji6kadpU7f/Zu+/wKKqvgePfu5tkkw1JaKH3Kk1aBBGlgyIgUlRQQURA6k+KIKgIiIhUEQUR7FhBESzYUIFX6b1KJ3QILaRnk533jwmygW0hW1LO53n2STZ7Z+YEApmz995zNC1K07SoyMjI7Ad2c2TKSQJ78DfY8+2N5x3nQFA+rtwthBD5RC64O8rfuo/qhNWqsWzOT/4ORQjhQ5rVisoNRWi++UavEOyKpsHOna7H9eihj3VHhw7ujRM3OwWUtXleBsi0oVjTtGuapsVnfL4SCFRKFfVdiBmq3W//62mp8ItN7am6PaFKa9/EJIQQwq8kgc3hSlQoRsseTflp4e9cuyy9D4XILzSN3JHAulugSSn39sEWLgzPPgtms+MxoaEwfjyYcmGV5pxhM1BVKVVRKRUE9AC+tx2glCqhMn4AlVKN0O8XvL4USAFL05rd+EKog1ndTQvh8hH9c1MEtHOj2JMQQog8QRLYXOCxMZ1Jik/mu7ekYIkQ+YVS5I4+0LVrg9HFPkXQCzNVquTeOWfNgu7d9STWYPNrKiBAX148aBA8//ztxSvQNC0NGAr8CuwHlmiatlcpNVApNTBjWHdgj1JqJzAX6KH56AcyEZs3Jv6efeuAhIuwZvqN583HQKjvJ4eFEEL4hySwuUDFOuW5r1tjls35iWuXZBZWiPwgICiANIsbxYz8bdgw92ZCmzSB0vbqBNlhNMInn8DatfqS4sqVoWpVePpp2LIFZsyQHp/ZpGnaSk3TqmmaVlnTtCkZX1ugadqCjM/f0TStlqZpdTVNu1vTtHW+iEspCFU2VajT7Mza/zUFUmL1zwtXhkYDfBGaEEKIHEIS2Jzi3Dl44w146ikYMgRWrcq0D6zXhEdJik/mqze+82OQQghfCQgKIC3VSS/UnKJePWjTxnnhJbNZTzqzqmFD+PxzOHwYDh6EhQv1PrMiT/s9vcGNJzcvIT63B7Z+fOP5/VMgIJe0mxJCCOERksD6W3o6DB0KFSrobSY+/RTmz4cuXfSv7dkDQMXa5WjTuxnL3/mF89FeaFXgZRZSOccRznGENCz+DkeIHC8wKABLbkhgAb7+Gu6/X09ibZcTFygAYWHw/fd6MiqECwrFMa3kjS+EFL7xuabBry+CZtWfV2oJ1R7wbYBCCCH8ThJYf+vfHz76SN8fZlvgJD4eTp6Epk3hiF6o4unJPVEKPnr5Sz8Fm3XxXGExY3iGSEZTj+epyzNE8hljSSDW3+EJkWMFBQeRmuRG25mcIDgYvvsONm6Efv3gvvv0hPatt/TVJa2lOqxw3yUt/MaTqycgPeONnIO/wLE1+ufKAA9MlaXkQgiRDwX4O4B8bd8++Oorx1U8NU1PZMeOhaVLiSxThK7DO/LVG9/RdXgHqjWs7Nt4syiWC4zlLq5yjjQy34j/xBzWs4SpbCIc94pvxBLDHyxiF6uwks4d3Mv9DKIIZbwRvhB+ZQ4PITHezQq/OUWdOrBggb+jELmYUnCJCC5oBSmmrkJaEsSegPAy8NvLNwY2fBqK1fBfoEIIIfxGZmD96a23wOJiOa3VCj/8AJcvA9Djhc5EFA1j4ejFOb5C6Wwe5Qpnb0leAdJI4TKneIsn3DrXL8xjEOX4ltfYy1/sZy0/MothVOUrxqORs/8shMiqkLAQEq/lsgRWCA85aruM+OJh2PIBXDqsPzdFQMsX/ROYEEIIv5ME1p82b4Y0N/a4BQfrRUyA0IhQek14lJ2r97Lxp21eDvD2neUQh9hIupP9rmlY2M9aLnDc6blW8ymLGYOFZFK5cUNvIQULyfzIbL5jqqdCFyJHMIeHkBQnCazIX66vCD5otVlZs3sp/GnT57X5aGmbI4QQ+ZgksP4U4OYKbk3LNLbDgDaUrV6Kd0d+TGpyztwjt4Xv0bC6NXYrPzh8LZ10PmEkqSQ6HJNCIt/yGklIiyFxQywxfMsURlKbIVRkIi3ZxHLSyR2FkUIjzMRdSfB3GI4lJsIHH8Bdd0HFivrH99+HhBwcs8g11lttqk3vXgKp8frnRatJ2xwhhMjnJIH1pwcf1GdXXbFaM7WOCAgMYMjcvpw5fI4vp9pvq3OUbaxhMf/wFZc45amI3ZZMvN2lwzdLx0Iy8Q5f38Vvbp1HYWAdX2cpRpF3beVHBlOBZUzhJHu5wHH2spq36cXz3EksF/wdoksFi4YTG3PN32HYt2cPlC8Pw4frfVmPH9c/jhihf33XLn9HKHIphT4F+4+1NunGm34/KgM8vAAC3Og7LIQQIs+SBNafBg50PcZk0qt63pToNmxbl1aP38tXb3xH9P4bCeoe/uI57uAVmvE+g1nAAIZRhdd4wKeJbBHKEkyoy3FBhFCEsg5fP8sh0t1IYFNI4DQHshSjyJuOsIXZPEYqiZmWnIP+xspZDjOJVqST7qcI3RMRGU7c5XjS03NYnDEx0Lw5XLyoF5mzFR8Ply5BixZw/rxfwhN5wzVCOXDHUJuvKOgwG8pIOyYhhMjvJIH1pxIlYPZsMJvtvx4UBOXK6f1h7Rg4uw8hBYJ5a+BCrFYr21jJVDpyhgOkkEAy8SQTh4UUdvMHY2jgsyT2brpjdWMJsYZGI7o4fD2QYBRGh69fp1AEEZKlGEXe9CUvO11yno6FGE6wg599GFXWRUSGo2ka1y45XqHgF/Pn68uHnUlMhHnzfBOPyFtsuuIcqdoXenwBzV+A/n9A1NP+i0sIIUSO4ZEEVin1gFLqgFLqsFJqrJ3XlVJqbsbru5RSDdw9Ns8bNAg+/BBKlYICBfRHWJg+49qlC2zaBOHhdg8tVCyC/tN7sfv/9vPj+78yh54Ob9ytpBHPZRbixqyvB5gJ50Gew4SD5BwwYeYhRjudqb2TNm7tpQ3CTH3a31asIu+I4xJ7We1yXDJxrGSu9wPKhsIlCgJw+ewVP0dyk3ffzdyz2p6UFGmnI7JNUwru6KBXHC4tM69CCCF02U5glVJGYB7QHqgJ9FRK1bxpWHugasZjAPBuFo7N+x57DE6dgpUr4Z139EIop07pPWILFnR66AN9W1GvZS0WjvmUtDOBTsdaSWc3f3CFs56M3qGeTOE+niQIMwabWVQDRoIIoQV9eIRXnJ6jOJWoSmMMTlsWKwpTimrc7TImC6m5poiPyLrLnCEQ9/bHXeCYl6PJnqJligBw8fRlP0dyk5gY98ZdvKgXoBMiC5TrIUIIIfI5T8zANgIOa5p2VNO0VOAroPNNYzoDn2q6DUBBpVRJN4/NH5SC++6Dp56CRx+FIkXcPEwx/L1nsaSmcWVwdZf3iwEEcYB1HgjYNQMGnuU9XuNv7qUnxalEcSrRjF68zgb6Me+/gh3O/I/PCaMIRjtJrMKAmXDGsMLhuZJJ4Cfe4lnK8gQh9CSI4dRkNZ9IMpvHhBDmtHWTrWAKeDma7Im8nsCeuuTnSG7iTuE50PfvK0lHxO3L6b3OhRBC+IcnEtjSwEmb56cyvubOGHeOFS6UrlKSapMVad+XIvWjCi5Ga1h9XLymIvUZxmLe4QjvcIQhfER57nT7+CKUZgY7aMKjBBKMmQjMRBBIMA3pyDS2UoYado+N4zIv0JAvGMdlTqFhRUPjNPt5nyG8RjssbhSJErlDJOUpSAmX40yYaUYvH0R0+wqXKIjBoIg5mcMS2I4dweDiV4fBAB06+CYekacoedNDCCGEC242InXK3m+bm982dTTGnWP1Eyg1AH35MeXKlctKfPlCmxGNOPLTSpKG1yWg5QWMFe3vhU0njbLU8nF02VeIEjzH5yRwlePsRMNKWWoRQTGnx82iO+c5ZreScQoJHGQDHzOc/sz3VujChxSKrrzERzxHCo77kSoMtMQHBWEuXdL3saen662wKlVy+1BjgJEipQtzLjqHtfwZPRp++AGSkhyPCQ6GMWN8F5MQQggh8g1PzMCegkx9UMoAZ9wc486xAGiatlDTtChN06IiIyOzHXRe08rwNGEfbwcDJD7VCM3BJGspqufKBPa6UApSi+bUpqXL5PU0/3KIDU7b8KSSxGo+JpEc2m9TZFkr+tKIhzHZKQ6mV6s28zzLKEAh7wVx/ry+t710aejZE558EmrVgqZNYft2t09TqnIJzh7JYe1ooqJg6lTH1dPNZpg8GRo18m1cIk+Q+VchhBCueCKB3QxUVUpVVEoFAT2A728a8z3QO6Ma8d1ArKZpZ908VrghjCI8Wm4U4e/sJf3voqTMqH7LGBNm+uWjmcZ/+Nqt/ZAGAtjGTz6ISPiCQjGMxTzD25SgMoEEE0wYAZhoSCemsI66tPVeAOfOQf36sGyZXo03NlZ/JCfDunX6Xvf16906VanKJTiT0xJYgOeeg+XLoUkTfa9rWJj+8e679e975Eh/RyjyANkCK4QQwp5sLyHWNC1NKTUU+BUwAh9qmrZXKTUw4/UFwErgQeAwkAj62j1Hx2Y3pvyqKy/BE4qPflhJ8oRaBLQ7T0CDqwRTgACCGMU3VKeJv8P0mWtccKtIkxULCeSwViUiWxSKljxNC/pwmdMkk0BBShBKhPcv3q+fXqk3zcHPXkICdO4MZ8+C0XmP41KVi3P1QiyJcUmYw3JYn+O2bfVHTIy+VLpwYSjmfFWEEK7IFlghhBCueGIPLJqmrURPUm2/tsDmcw0Y4u6x4vYoFN3USzSZ35sh68aiPdaWZlsiuCviAaJ4iACct9nJawpThgCCSHNRpMlIIBEU91FUwpcUiiKU8d0Fz56FVascJ6/XJSfrbbM6dXI6rHS1UgCcOniGag0reypKz4qM1B9CeJhmvySGEEKIfM4TS4hFDlOqSFkmfzWepOOKS32r0Vjrmu+SV4D7eBzlxo+4hkZ9HvRBRCLPW7MGgoJcj4uLgx9/dDmsfE09+Y7eeyq7kQmRK8gErBBCCFckgc2jaje9g/7TnuSf7zaxbE4e3N957hwsWQKffQabN9vdLBVJeerTnkAc9600YeZBnsNEDlueKXKn1FT3N+4lJ7scUqpycQICjUTvO+lyrBB5jeyBFUIIYY8ksHlYtxEdadqlEYte+Iw9//zr73A84/x5eOghqFBB32s4aBC0bAnVqsHvv98y/H98RkUaEGynIq2JUO6iM4/xqg8CF/nCHXe4d9cdEgL16rkcFhAYQJnqpYjeJzOwIn+QPrBCCCFckQQ2L7h2Dd5+G6pW1auBRkbCwIGogwd5/oPBFC9flNcem82ls7m8UNGFC9CgAfz8s17dNS4O4uP1ojiHD+uFcb77LtMhJsy8yhoG8zGVicJEKMEUoA6tGc0y/sfnGOSfgfCUu+6C4m7sp9Y0eOopt05ZoXY5juw8nr24hMiFZAZWCCGEPXLnntsdPqzPPo4dq38eHw8XL8IHH0D9+hT4bgkTvh1NQmwiE7vOIDXZeUGjHG3YMD2JdVQgJykJevXSE1obRgJoQnfeYDOfEc9i4niFVdSlHUp2XAlPUgrefVefYXXEbIYxY/SqvW6oXLcCMScvEX81wfVgIXI5+R9ZCCGEK5LA+oum6XvgsvMWc3IyNG+uJ3WJiZlfS0vTE7qhQ6l0NZoXPh3GvxsP8eaz76Hlxre1L12C7793Xd1VKfjiC9/EJIQ97drB4sUQGqo/rgsO1nul/u9/MHGi26erWLssAMf3+mAfbEoKWK3ev44QbsiFv6mEEEL4gCSwvrZlC3TrplcqvX6D++yzcOhQ1s/1zTf68mFnCWliIkyYwL1dGvPUpMdYtXgtS2Z8f/vx+8vmzfrNvyvx8Xp7EiH8qVs3fb/27Nn60vYOHWDcODh+HKZOzVKzywq1ywFwbPcJ78R6+jSMHg0REfrscFCQ3t/1zz+9cz0hnJEpWCGEEC54pA+scNOiRTB8uD5zen2WIykJPvwQPv8cli+HNm3cP9+8eXrC5sr69XDpEk+83I3ofSf5YNznlL2jFPc8dNftfR/+YLF4Z6wQ3hIaCgMG6I9sKFauKKERZo56Yx/sjh3QooX+/1CqzfaCVatg3ToYORImT/b8dYVwQ65cLSSEEMLrZAbWVzZu1JPXxMRbl+ilpen7Nh9+GM6ccf+c5865Ny4oCGJiUEox6oPBVG1YiamPv8WBLUfcv5a/1aqlL290JTgYoqK8H48QPqKUonK9ChzecdyzJ05M1N8wi43NnLzavj57Nixb5tnrCuGE0WZ1glUSWCGEEHZIAusrU6bosxzOpKXpBWDcVbCge+NSU/8bG2w28eqKF4iIDOfljlM5e/S8+9fzp0qV9ArE7nj2We/GIoSPVa5bgWO7oklPT/fcSb/6ynUv2sREmDTJc9cUwgWj4UYCmy7bsYUQQtghCawvWCzwyy+uCzalpMBHH7l/3n79MheJcaRmTShR4r+nRUoWYsrKF0m3pPHig1OIvXjN/Wv609tv63v0HDGb4bnnoGRJ38UkhA9UqV+RlKRUTh3IwgoNVxYuvKVit10HD8Ip6UMrfMNgm8DKDKwQQgg7JIH1hfh494u2XMtCMtm7NwS42MZsNsOECbd8uXyNMry64gXOR1/klc7TSE50Y3muvzVoAL/9BkWL6v1urwsJ0ZcOjxihF8gRIo+p2rASAAe3HvXcSS9fdm9cUJD7Y4XIpkxLiK2SwAohhLiVJLC+YJtsuVKkSNbO+9tv+sfAwFtfN5v1IiydO9s9vPa9NRj3+XPs33CIKT3eJM3iokVNTtC0KZw9q7cp6dcPnnhCLzJz6hS89lqWqrsKkVuUq1Ga4FATBzYd9txJixd3b1xKChQr5rnrCuGE7RLiNElghRBC2CEJrC8EBEDXrmBw8ccdEgIDB2bt3I0awe7d+r7P0FD9WkYjtGoFK1a4rCB6X9fGDJvXjw0/bmV6n3c8u8fOWwIC9KR80SL47DMYNSprib8QuYzRaKRqg0qeLbw2eDAUKOB6XL16mbYgCOFNtgmszMAKIYSwRxJYX3nxRX2ZqzNBQfqsYlaVL6/vD42LgytX9MIsf/zhdkueTgPb8czUJ/jry394e/D70rpAiByo+l1VOLLjOJZUD7WJ6tZN7/3qbNWC2SxtdIRPGWUPrBBCCBckgfWVOnX0Xq9m863LfYOD9RvJVauyN5OolD6j4mpfrB09XniYHmO78NOiVSwas1iSWCFymDsaV8WSYuHozmjPnDAoCFav1pcH31wczWjUV4TMmAFt23rmekK4waBsqxDL7yEhhBC3kgTWlx5+WF/uO2iQXogoOBhKldJnZw8e9Hv/0r5TevLQ4PtZOusHPnnla/tJ7K5d8NRTNwop1a6tV0521SJICJEtNZtUA2Df+oOeO2mVKnDggN7mq2JF/f+kggWhVy+9d/XgwZ67lhBuMNrclcgSYiGEEPZkfapOZE+lSvDWW/ojh1FKMWRuX1KTLXw+5VsMRgO9Jz56Y8CMGTBxol7U5fpe2b17Ydgw/Qb4779lr5wQdpznGNv4kWQSKEo5GvEwJpy0hLIjskwRIssWYd/6A3T534OeCy4iAoYP1x9C+JnRplaEFHESQghhjySwIhODwcCIhc+CprH41aUopeg14RFYtkxPXhMTbz0oIQGio/U9t7t3SyVgITLEcoE5PM4B/gEgHQtBhPAeA+jCOLryIgr3/73UbFKNvf8c8Fa4QvhdpjY6spVFCCGEHZLAilsYDAZGLBqIpsGnk5YA8OQX41H2ktfr0tL0JPbPP6F1ax9FKkTOFc8VxtKIK5whnRuFl5KJB+A7phLHJfow2+1z1m5agzVL1nM+Oobi5SM9HrMQ/ma7hFj2wAohhLBH9sAKuwwGAyPfH0i7Pi34dNISPjxmxuWtRHw8LFjgi/CEyPG+5TWucjZT8morhQR+ZwEn2ef2Oes0qwHArrXuHyNEbmKQKsRCCCFckARWOGQwGBj1/iA6dqrFV2mVmU9drK4OOnnSF6EJIJ00YogmhmjSSfN3OMKGhRRWsYg0Up2OS8PCStzfD1+xTjkKFAxl99r92Q1RiBwp0xJimYEVQghhhywhFk4ZDAb+N7Ezpp9/5Nu0qiRrAQxnK0ZHBxQt6svw8qUEYlnBdH5lPukZCVIAQdzPEDozBjPhfo5QXOAYuF6zgJU09rHG7fMaDAZqNKnGvvWyD1bkTbZ9YKWIkxBCCHtkBla4pOrW5dmiZ3lS28cvqiJTaUyqvR+dsDDo29f3AeYjscQwhvr8yCwSuUoKiaSQSAJX+YGZjKE+cVzyd5gCIAvFmbKi5t3VOLH/NPFXE7xyfiH8yTaBlRlYIYQQ9kgCK1wzGFDjX+Yp83EGaLtYo8oynqYk2k7gKwUFCsBDD/kvznxgDj25xCkspNzymoUULnGSOTzuh8iErUgq4M4MrIEAqtM0S+eueU91NE1j/8ZDtxecEDmYUfbACiGEcEESWOGegQOhZ08eCT3NaG0zO4hkNM24ShAEBUGhQvDHHxAgq9K95RxHOMA/DosCgb6ncj9rucBx3wUmbhFEMC3pSwBBTscFEEhHRmTp3Hc0qoLBoNi//mB2QhQiRzLY7IFNd1l0QQghRH4kCaxwj1KwaBF89hntGpdgktpAtApnhKE1Z/sMgb17oUYNf0eZY6RhYQPfMoEWDKUKY2jASt4mgdjbPucWvsedWb0bY4UtCyn8zZe8zL0MpQovEMXvvEdSRlsbT3uEVwgnEqODUgMmzLTkacpR2+W54rjECmbwPHUZE1aHkDppbFj/N+mkezpsIfxKlhALIYRwRRJY4T6l4OGHYcMG7k4+xhu/judqRDH+t+Ia/56I8/rlz3KIf/iaf/iacxzx+vVu1yVOMZw7mEcf9rGG8xzhGNv5grEMohz7+b/bOm8ScVhcVLUFSCP1v16jQneOIwylMu8xgAP8w3mOcJStfMIoBlGOI2zx+DXDKMIbbKYKjQki5L/ZWBOhBBFCB0byDO+4PM92fmYQ5VnCBKLZxXmOYGlynMMbTvJ8ej1iifF47EL4ixRxEkII4Yqs9xS3JyiI2m0bMOef13i5w+s833IiL34xnHs63+XxS51gD+/Rn+PsxJDxI5uOhUo0ZCCLKEPOmflNJZnx3MclTmK9aXYshUQAptCe6WyjFNWydO6ilMVEKCkuklMTZopQJmuB52FJxDGe+4jlPNpNjaBSSCCFBCbRmtnsoShlPXrtQpTkNf7mNAfYwvekkEAk5bmb7oQQ5vL4o2xjJt1JzfjZuS7gnkukLqjMyb1nmXRnK2awA6Pj2uBC5BqZ2ujIHlghhBB2yAysyJbyNcowd/3rVKxTjoldZ7DsrZ88ev5j7OAl7uEgG0gliWTiSCYOC8kcYB0v0pgT7PHoNbNjPUuJ4+ItyastC8l8y2tZPndjuqG5sWTUSjqN6Jrl8+dVf/ExScTekrzaspDED8zyWgylqU5nRvMoE2nJ024lrwBf8TKpJN3ydeM9eqXp1HURxHCcHfzi0XiF8JdMRZxkBlYIIYQdksCKbCtUvCAz/pzIPZ2jeHfEx8wdvIg0S1q2z6uh8SaPkoyj5ckaScTzJo9l+1qe8hNzXC7ftZLOepbarSTsjJlw2jMME2aHY0yY6cAIQiiQpXPnZT8z97/Zb0fSsPAH76O5ucfYF+K4zB7+xN6+Z0PFBFSJJNLWFSGZeFbylneC0DTYuBFmzYIZM2DVKrBKZR3hPQapQiyEEMIFWUKcW1it+iOHVvkNNpsYv3QUH477giUzv+f04bO8/PVIwgrdfiJ1kA1c5oyLURoXOM4RtlCZKLsj0rBgJADlpb6cti5x0q1xCkUclyhMqSyd/3Gmco0Y/uFrLCT/N6uoMBBIMPfyBD2YnOW487IrnHVrXBopJJOQY5L/q5wlgCC7b3QopS8jTv+nCADnOer5ALZuhccfh9OnwWLRk1mTSe/3/OGH8MADnr+myPcyLSGWGVghhBB2yAxsTma1wtdfQ4MGEBiot6spVw7eegvic16RHqPRSP/pvRj1wWB2rdnH/5q8yKlD7iUP9uxjNRaSXY6zksY+1mT6WiwxfM0rPE1RHsdEDwKZRGt28vttx+OOIELcGpdOGiZCs3x+AwYG8yGvsoZ7eIxIyhNJBZrSg1dZy0AWYpB/1pkEEezWOA3N7bG+YCKUdByvZDDecwnrsQJYzwYTfBs/S05t2wbNm8PBg5CQAKmpehIbHw9nz0LXrvDjj569phBIESchhBCuyZ1uTpWWplf8feYZ2L5dT2Y1DU6ehHHjoH59iMmZ1UcfeLol01dN4NqleIY1HsfmX3fc1nnSScPqZN/idVasmW70T/MvI6jBCqYTzyU0NKyks4c/mUEXPuX524rHHU3p4bL3J0A56hBKxG1fpzJRDOcL5nOc+RzjOT6nMg1v+3x5WSO6/lf8y5maNHfY8sYfIilPQYo7fD2gqb4PlnUlaUYvz178qaf0xNWRpCTo3VtPaoXwIGmjI4QQwhVJYHOql16CP/6wfxOZlATR0dC5s+/jclOd+2rwzqapFCtXlJc7vM7SWT+gZXE/U1lqEezGck4TIZSlFqAvF55EK+K5bHfpZQoJ/MYC1rA4S7G4636GYHBRDdZEKF150SvXz6niucJ5jpLAVZ9fuyMjCCDQ6RgTZrowzkcRuUeh6MKLDvc8G+tfgeB00v4uTEv6eu7CO3bAUTeWJKelwQ8/eO66QgBGm7sS2QMrhBDCHklgc6KkJJg/HxKdFJ6xWGDnTti1y3dxZVHJisWZ8/dkmnZtzMLRnzLtqbdJSXK/cFFDOrnVGiQAOlcauAAAIABJREFUE/VoD8BmlpNEvNNiPCkksJRJXinYE0k5BvMRQQ6SDhOhtKAPjfNJleC9rOYVmtOfEjxPXfpRnAm0YC+rfRZDGWrwNG85+Tsx8yDDuZM2PovJXa3pRxQP2V1uroIgsNFViqy7izAKe+6iGzfqqz1ciYuDdes8d10hAIOSKsRCCCGckwQ2J/r9dzC48VeTkgKffeb9eLIhpEAI478eyVOTHuOPz/6PEfeN59zxC24dG0AgfXnbYeIBEISZZ3jnv0R3FYucVC2+4SpnOcMB976JLGrKY7zC79SmFQGYCCGMQIIpxR08y0Ke4W2fFJTyt5+Zx+t0YD9rSSOVZOJJI5V9rOF1OvAL830WSxv6M5YfuIOmBNr8nZTjToaxmMeZ4rNYskKheI4v6MObFKMiQYRkxG6iPu1p07QTp7ddJinB9V5xr5AZMuFhmZYQy8+XEEIIO3LOhi9xw+XLkO663yfp6XDunPfjySalFE+O707lehWY1vttBke9wEtfDqdh27ouj23Gk6Rj4QOGAooU9CXVwRRAw0p/3qWpTRuda1x0KyYjgcRz+ba+H3dU5x4m8AfXuEgs5wkhnKKU9dr1cpojbOUzxpDqoH1NKoks5nmq04SK1PdJTHVoRR1aEcsFrhGDmYIUobTXr5uGhWh2kkoSxahIEcpk6XiFog39aU0/YogmhQQKUpIwCrPp3u38PHUd/248RP1WdTwTcMOG7r2BFhYGjRp55ppCZMhUxCldElghhBC3khnYnCgyEoyul84SEAClvX8D7ilNOkXxzqY3KFKqEC+2n8JX05a7tS+2JU+ziPM8xSzu40ma0Ys+zOEDYmhO70xjC1HSrVjSSCWcYrf1fWSFnrbWylfJK8D3THdZQdpCCiuY4aOIboigGGWp5fXk1UIKXzKeZyjGJFrxBp0YRlXG04xDbMzy+RSKYlSgLLX+WzJc4+6qAOzfcMhzgUdFQRk3kmyDAbp08dx1heCmNjoyAyuEEMIOSWBzojZu7sULDNSrheYiZaqWZO66KdzX/W4+GPc5E7vOIP6qk2qnGUIoQFue5X8sZhif0ppn7Ba3acuzBBPm8nzFqEhJqtzW9yBc28IP//WodUTDyhZW+Cgi37KQwkRa8SOzSOQqScSRSCwWkvmX/2MirdjOz9m+TlihApSqUoJD2zzcB/bjj8HseOk+ISHw/vt6ay8hPMh2Blb2wAohhLBHElhfuXwZvvsOvvwSNm92vnfMZILRoyHUSW9HkwnuvRfuuMPzsXpLdDQsXUrIT9/z0sQHGTS7Dxt/2saghmM4uPWIRy7RgA6EU9RpJeAgzPTgNY9cT9jnTv9egFQ3x+U23zKF42wnlSS7r6eSyCweIZFr2b5WtYaVOLjFM/9+/nP33fpe/LJloUABfbZVKX3ZcJEi8Pnn0L27Z68pBGCwTWAlfxVCCGFHthJYpVRhpdTvSqlDGR8L2RlTVin1l1Jqv1Jqr1LqOZvXJiqlTiuldmQ8HsxOPDnS1avw+OP6Ut8+feDZZ6FlS6hSBX76yfFxL76o3yDaS2JDQ6FmTVi61Gthe9SRI9CqlZ5sP/MM9O+PatCArl9MYvaiJ0i3pDO86cv8sOC3LLfauZkRIxP5i4KUuGWGVmEgCDPdeZnGyNJHbyrgZlXcMIp4ORLfS8PCL7ztMHm1tdYD7ZyqNazMhRMXuRoTm+1zZXLPPfqbTt9/D5MmwcSJetG4c+dk6bDwmkxLiGUGVgghhB3ZnYEdC/yhaVpV4I+M5zdLA0ZpmlYDuBsYopSqafP6m5qm1ct4rMxmPDlLbKy+n+zbbyE5Ga5d01tPJCTofRYfecRxFWGDAT76CJYv15cUm80QHAx33gnvvQcbNkBEhG+/n9tx6JD+Z7Bmjf5nEBenP5KSYPNmag7pybvv96Beq9rMHbyI1x+fQ8I1J+2D3BBJeeawnyeYRjEqEYiJYMJowiO8ytoc1+8zL2rLQAIwOR0TSDDtGOijiHwnml1YcV2ELYUE1rEk29er2rASAIe2engZMeizri1bwssvwyuvwEMP6XvvhfCSTEWcJIEVQghhR3bvRDoDLTI+/wRYDbxgO0DTtLPA2YzP45RS+4HSwL5sXjvnGzcOTp6E1FT7ryclwYAB0KEDFLpl8lq/eWzTxv09sTmEhRS28iPnOUrzJ94hIjYW5WhmNSGBiH69ee3YMZbM+J6Pxn/Fgc1HeOmrEVSPqnzbMYQQRnuG0p6ht32O69JJYxsrOcMBAjFRhzaUpabrA/OxBxjKL8wjDcd9fwMJ5n6G+DAq37CQjHLzvUFHVZqzonK9CgAc2RnNXQ/4pqKzEN6SqY2OJLBCCCHsyO4MbPGMBPV6ouq0rKtSqgJQHzKV4ByqlNqllPrQ3hLkXCsxET791HHyep1S8MEHvonJB1byNs9QjHk8zZr94wjZc8Jx8nrdlSsYVq+mx9guzFo9iTRLGsObvsSyOT9le0lxdv3FJzxDMebyJF/yIp8xhrFEMZZGnOeYX2PLyQpRgon8SQEKYyLzMngToRSgcMZS7+J+itB7IqmAxUnifp3CQGlqZPt6YYUKULx8JEd2Hs/2uUT+oZR6QCl1QCl1WCllb/XU9XF3KaXSlVI+2fScqYiTVCEWQghhh8sEVim1Sim1x86jc1YupJQqAHwLDNc07XrlkneBykA99FnaWU6OH6CU2qKU2hITE5OVS/vHjh3utcJJTIQff/R+PD7wDZP5grEkcY1k4qix2o1etgDx8fDnnwDUbnoHC7bP4K729Xl35Me80nkasRezX+jmdvzKfN5nMAlcIZk40knDQgqpJHGUrYwlihhO+CW23KAi9ZnPcXoxnXLUoRClKMed9GIG8zlORer5O0SvKEJpqtLY5bgggnmQ/3nkmpXqlueoJLDCTUopIzAPaA/UBHretLXHdtw04FdfxWaQPbBCCCFccJnAaprWRtO02nYeK4DzSqmSABkfL9g7h1IqED15/VzTtGU25z6vaVq6pmlWYBHQyEkcCzVNi9I0LSoyMjJr36U/WCz67Kq7Y3O5GE6wjNdJsVkSaUgD5byTyg02fwbhhcOY9N0YhrzVl62/7WRg/dHsWuvbFedxXOITRjlc4qlhJZFYPuI5u68LXQhh3M9gZrGLhZxmFju5n0GEuNHqKDfrxQyC7LR5ui6QYGrQjMpEeeR6letW4NSBM6QkuZ75FQL9d+1hTdOOapqWCnyFviXoZsPQf3fb/d3uDTIDK4QQwpXsLiH+HrjeiPQpuLWpo1JKAR8A+zVNm33TayVtnnYB9mQznpyjenVIceNmMjBQL3KUy/3KvFv6fp6qBemBbhxcoIBenMqGUoqHh7Vn7obXCQ41MbrVRD6Z8DXpaW7O6mbTn3yIwvkbEFbS2cHPxPru3k7kElW4izF8h4nQTEuor1fCrkULnudbj12vQu1yWK0aJ/8947FzijytNHDS5vmpjK/9RylVGv338gIfxkWAbREn6aMjhBDCjuwmsG8AbZVSh4C2Gc9RSpVSSl2vKNwU6AW0stMuZ7pSardSahfQEhiRzXhyjhIloEUL17OwRiMMzX6hIX/bw5+kkXm/794WkBTu5gkc9JSsUq8i8zZPo03v5nw2+RtGNn+Fs8fOZy9YN+zmD7faoAQSzHF2eD0ekfvUpR2LOEdvZlKLFlTmLprRi1dZy0v8fEubp+yoULssAMf2yJJ24RZ7v5huzhbnAC9omub0XUNPb+8JNN64LbGku7uERwghRH6SrSrEmqZdAlrb+foZ4MGMz//G/i9LNE3rlZ3r53izZsHff+t7PO0xm6FnT6ha1bdxZUVMDLz/PqxerT9v3Vrv5VrEdf9OzQALF8CIx8DkKBc0m2HGDL1FkKMhYSGM/nAIUe3q8daghQysN5ph8/rR5slmWf9+hPChEArQjoFebxdUukoJAoMCiN570vVgIfQZ17I2z8sAN0/fRwFf6YuoKAo8qJRK0zRtue0gTdMWAgsBoqKisj1lGhRwI4FNlQRWCCGEHdmdgRXO1KwJf/0FxYpBmM2eP5NJf/Tpo/d0zalmzoRy5WDyZPjtN/0xcSKUKQNvv51p6B3cRwC3rhfe2gne+QSSQiGpgM0LZjOEhMC0aTDQvZv7lj2a8t6OmVSqW55pvd9myuNziLvi4M2BbKpJMwJxnFRfZyGZctTxSgxCuCsgMIAy1UsRve+Uv0MRucNmoKpSqqJSKgjogb4l6D+aplXUNK2CpmkVgG+AwTcnr94QaLzxfndqmiSwQgghbiUd6b0tKgrOnIGVK2H5ckhIgFq19FnMUqX8HZ1j8+fDhAmQnJz560kZU6ljx0JoKPTtC0B7hvIb7wK3FqTa8Ahs6whNvzbQ6o/y3GFtAvfcA716Qbi7a4x1xctHMvOviXw9bQWfTlzC3r//ZfTHQ6jfyrNJZGv68Q2TnY5RGKhNawpR0uk4IXyhTPVSHN0Z7e8wRC6gaVqaUmooenVhI/Chpml7lVIDM1736b5XW7YzsLKEWAghhD3K3302b0dUVJS2ZcsWf4eRdyUn67PGcXHOxxUsCBcu6IWogM94gV94J1Ml4usUBgpQmBnsoEjmWiG37cCWI0zrNZeTB87QbURH+k7pSVBwkEfODbCc6XzDJAffjyKEcN5gCyWp4rFrCnG7Pnr5S76atpyfEj8nIFDem8wqpdRWTdNyf0U9P/LE72ZN06g4buV/z4++/iAGg5sV/YUQQuQpjn43yxJicavlbq4SS0/P1MP2Cd6gCy9iwkww+nphIwEEYaYCdXmDzR5LXgGqR1Vm/tbpdBrYjm/f/JGhjcdxbLfnZqAeZgyP8wbBFPjv+zFgxISZUlRnCusleRU5RplqpbCmWzl71PtFzoTwFqUUQUbZByuEEMIxeZte3OrgQceFp2wlJeljMygU3XiJB3mODXzDBY4SRAj1eZAK1PVKqMFmE/+b35/GHRsy65n5DLlrLE9PeZxuIzpgMGT//ZkHGUZr+rGJZZzhAIEEcydtqcJdHoheCM8pXU1fyn760DnKVvfcG0VC+FqgUZGaUfs4Nd1KcKDRvwEJIYTIUSSBFbcKCtLb+6SlOR9nNOpjbxJCAVrSxzuxOdD4wQYs3DWLNwe8x8LRn7Lxp62M/mgIxctHZvvcJkK4jyc8EKUQ3lOyUnEAzh2TvsQidwsKMJCQkcFapJCTEEKIm8gSYnGrtm3tJqa3MBj0sTlEwcgIJi4bzaj3B3FwyxEG3DmKXz/+i9y4z1uIrCoYGU6w2cS545LAitxNWukIIYRwRhJYcauGDaFSJVBOCmcYDHqboNq1fReXG5RSPNC3Fe/tnEnl+hWY2Xc+k7rN4MqFWH+HJoRXKaUoXiFSEliR62VKYGUGVgghxE0kgRX2ffMNREToierNjEa9AvHXX/s+LjeVrFicGX9MYMD0XmxauZ0BdUby93cb/R2WEF5VrHwkF6Jj/B2GENkSaJQEVgghhGOSwAr7qleHrVuhfXswmfRkNiICgoOhQwfYtg0qV/Z3lE4ZjUYeef4h5m+dTmTZIkzqNpM3es8l7oobBaqEyIWKlCzE5XNX/R2GENkiVYiFEEI4I0WchGOVKultcs6dg1279K/Vq6f3iM1FKtQqy9z1r/PFlGV8PuVbdv61l1EfDCaqnXcqIwvhL4VLFOTK+VjS09MxGqVyq8idTLKEWAghhBMyAytcK1EC2rXTH7kseb0uIDCA3hMfZe761zGHhzDugdeYO3gRSQnJ/g5NCI8pVKIg1nQrcZdllYHIvWQPrBBCCGckgRX5SvWoyszfMo3uIzvx43u/M7D+aPb886/9wbt3w+DBeqXlbt3g22/BYvFtwEJkQUTRcABiY665HqxpsH499O2r/4z37Am//gpWSRiEfwXKEmIhhBBOyBJike+YQkw8O7M3d3dqyMyn5zGy2St0G9GRPpMfwxRigoQE6N4d1q6FlBRI1/sR8vvvEBICv/wC9ev795sQwg5zWDAASfEuVhZcuqTvb9+3DxIT9WQW9C0DkZGwapW+hUAIP5AZWCGEEM7IDKzIt+o2r8V7O2fRYUAbvpn9A4MavsD+DQehUydYvVq/sb+evALExcGFC9CiBRw54q+whXAoJCwEgMQ4JwlsSgrcdx/s2KG/WWPbJzk+HqKjoUkTuHjRy9EKYZ9tESeLzMAKIYS4iSSwIl8zh4Xw3LsDeOPXl0lOSGb4vS/z4T9XSU1OdXxQQgJMmOC7IIVwkzkjgU2KS3I8aMkSOHHC8XJ4qxViY+Gtt7wQoRCu2c7ApsgMrBBCiJtIAisE0LBtXRbtmkXb0ml8aanMUFpxmIL2B6en6/th46VQjshZTOYgAFISUxwPmjlTfxPGmZQUmDcv8+ysED4iCawQQghnJIEVIkNoRCjPB+5isvYPsZgYSisWU4M01K2DAwPh1CnfBymEE4aMpZdWq5PE8+hR904WF+c60RXCC0ICb7SASkpNdzJSCCFEfiQJrBC2goK4m7Ms4jeacYpPVS2eoyXRhGUalpqeyJigJvShMJNozXZ+xorMFAj/MhgyElhn+wYDA907mdUKQUEeiEqIrAk13agvmSgJrBBCiJtIAiuErc6dwWQiHAsvsonx2nrOE8og2rCUaly/lUoIT+d4haskcIU9/MlsHuV12mPBydLNLIjnCn/xMd8zk9V8QgJXPXJekbe5NQPbrh0Y3Pivv149SWCFX2SegU3zYyRCCCFyIklghfdZrblnL92QIaBuLBluxmkW8Rt3cY6F6k5G0YKjwQX4YRRoNv96kolnP//HAgZk6/IWUniPZxlAKT5kGF/wIh8wlP6UZCGDsOCkuJSPWbGikUv+XvMJqzs9XJ9/Hkwm52NCQ+GFFzwTlBBZZA66kcDKDKwQQoibSQIrvCM+Xq9iWr48BAToyxbvvhuWL8/ZyWyZMjB3LpjN/32pEClMZD1jtE0cV+EMtbThW1UF7aZcIZUk1vM1Vzh7W5dOJ53X6cBaFmMhmWTiScdCMvFYSGYNn/AGHUnHfzd0CcSyghkMoDQ9CKAHgbxCc7axUpLZHCA1SX+DwxTiZOY0KkpPYm1+xjMxm6FjR3jkES9EKIRrmRJYiySwQgghMgtwPUTkKrGx8OWXsGcPhIRA+/bQsmWmWUWvu3BB7yN57pzeSxX0yr0bN8KTT0KHDnqM7ixj9If+/aFkSf0m/9QpCAggXUvlXu0E53pf4MvjDUgcXQ/jijKYP9yMsYptoRvF33xJJ0Zm+bIbWMohNpCK/RYoqSRxgHVsYhlN8H1ycZGTvMQ9xHPpvxg10tnPWo6ylab0YCCLUPaKXvnTjh3wzTdw+TJUqKD/DJYq5e+ovCLFnQQW4NVXoXJlGD8erlwBo/HGntcxY/SffV/+nyGEjZCgG7cmUsRJCCHEzSSBzSs0DSZPhqlT9cTweuK4YAEULgwrVuh72nyhQwc4edJ+n8mEBPjxR3jtNXjlFd/Eczs6dtQfu3fDiRNsCf2F+U0WkmRKxqytw7K4HInP1SOuXltCpu8iaOBRlAEsJHOJ26tOvJxppOC86msKCSxnus8TWA2NybTlKmex2pkBTiGBf/iSctShA8/5NDaHzpyBhx6C/fshOVlP0Ewm/efuscdg0aI8t8fz+gxskKsEFuCpp6B3b9i6Fc6fh4IFoXFjfcWEEH6UeQmx7IEVQgiRWQ6dAhNZNn48TJ+u36hfT15BX8p74gQ0a6bfyHvb5s36dewlr9clJsLs2ZCac/ZzOlSnDnToQHKLKDSTnhQoBUG9TxC++3cC7rtI0tAGJNx/H9ZoM0YCCKNwli+joXGC3W6NjWZnls+fXXv4i8uctpu8XpdCIt/xul+XOP/n8mVo1Ah27tR/3q7vDU1J0R9Ll0K3bjl7OfttSLimz4ybw4LdO0ApfUlxhw7QtKkkryJHCJE9sEIIIZyQBDYvOH0aZs503rMxPh6e88HM2OLFkGR/CWwmmgZ//eX9eDykIR1JJ/NMgKFMEqEr/ybkva2kbSzMtTvbkvJ+RRpr3b0cje+Trr/4iGTiXY5LIYnDbPJBRC7MmAEXL0Kag9mbpCT952/1ap+G5W1XL8QCUKh4QT9HIsTtM0sfWCGEEE5IApsXvPee6zGaBmvX6smuN509e2O2y1U8ly55NxYPCqMITXiEQDLPbCkFpv7HCNv1O4FRV4kfUJf5HZdx8czlLJ1foShFdbfGlqZGls7tCVfdLExlwEA8fv57TUuDd9/VZ1qdSUzU3/jJQ66c1xPYgsUj/ByJELfPHCR9YIUQQjgmCWxesH6965t1gOBg2LvXu7GULu1ecSaloFgx78biYQNYQHnuJIhbq7cGV7BS8fdTPPPWo+xavZf+tUey6rO1aFlYotqZMZgIdTrGRCgP4/v2JoUpDW4UZ7KSTjh+/nu9cMG95emaBtu3ez8eH7p89gohBYIJCXVzCbEQOZDtEuIkqUIshBDiJpLA5gVZ2bdmNLoekx19+uiJsisGAzRv7t1YPGnLFkxPDuD1Wtd4t25BnhkXTuSpAAIIIoJiPMIEZhq202PYIyzYPoNyNUozrffbTOgynUtnr7h1iab0pCy1bpnlvS6QYMpThyY86snvzC2t6YfJTuJ+sxDCqEyUDyJywmh0f29rTq2EfZvOR8cQWbaIv8MQIlukiJMQQghn8tbdW37Vrp3jno62UlKgfn3vxlKvHjRo4Ly6a2gojB2r94bN6SwWvR9m8+bw5Zeoff8SvusM97+ZwryqRj5+ZzKLOEcXxhJCGABlqpVi9tpXGTCjN1t/20n/2iPcmo0NJIgJ/EkDOhCIiUBMGV83EUgwDenEK/xBAL7/c7uDeylOZQxOCpebMNOdCRj8/d9KZCQUKuR6nNEILVp4PRxfOrH/FOVqlPF3GEJki1mKOAkhhHBCEti8oE8f1zNOAQHw8MN6Sx1vW7ECqlTRE9WbhYbCo4/qvSZzg3794KefMleyBVRKCio5BdMLk1BfL7nlMKPRyCOjOrFg+wzK3qHPxk7qPpMrGUV2HAkmlOf5hrc5TE9epzMv8DhTeZvDjGKJW7Og3qBQvMyvRFLOzhJqhQkzbRhAWwb4Jb5MDAYYOVLvg+yMyaSPyyMsqRbOHDlPuTtK+zsUIbIl0xJiSWCFEELcRBLYvKBQIZg3z/EsrNEIRYrAm2/6Jp7ChfXeknPnQvXq+kxrSAi0bg3ffQcffKDvgc3poqNhyRLnVZUTE2H0aIdvIJStXlqfjZ3ei00/baN/7RGs/Wa9y0sXoQydGMmTvEFHRlAE/yclhSjBTHbRm5kUpzJGAgkihAY8yDhW0oc3UW7sk/WJoUOhRg3Hy9nNZujfX18tkEecPnQOa7pVZmBFrhdkNBBg0P8vSbNqpKRJEiuEEOIGSWDziqefhk8+gZIloUABPWEMDdVv4Fu2hG3boEQJ38UTHAx9+8K//+oFdRITYdUqaNs2dySvAB9+6F5F5StX9EJaDhiNRh55/iHmb51OsfKRTH50NlN6vknsxWseDNY3ggnlfgbxDof5ilQ+J5Fx/Egtcth+5uBgver2I4/on1//NxEWBhERMHGi797Q8ZHD248BULFOOT9HIkT2KKUoEHxju0J8suyDFUIIcYN0rc9LuneHrl1hzRo4eFDfh9q6NZSTG9rbcj35dkUpOH4c7rnH6bAKtcoyd90Uvp62gs8mL2XHX3v53/z+3Ne1sWfiFZmFhsKnn8KcOfDzz3Dtml4l+4EHnO/RzqX2rT9ISIFgyteSGViR+0WEBHI10QLAteQ0ihQw+TkiIYQQOYUksHmNwaDPuLZs6e9Icr8IN3tpKmV/v68dAYEBPPFyN5o8FMXMvvN4tftMWvRoytC5fYkoGp6NYIVDhQvDE0/4Owqv+3fjQao3qoLR25XGhfCB8OAbxepikyx+jEQIIUROI0uIhXCke3d96akrqalZrmZb6c7yzF3/On1e7cHf326gf52R/LN80+3FKfK9pIRkjuyMpubd1fwdihAeER5y4/31a5LACiGEsCEJrBCOtGmjF8hytmc3OBh69nR/ttbG9dnYeZunUbhkISZ2ncGUnm9yNcZ5pWIhbrZ77X6s6VbqNKvp71CE8AjbGdhryZLACiGEuEESWCEcMRhg5UoID9c/v1lwMFStqldbzoZKd5bnnY1T9dnYZRvpV2sEa5asy9Y5Rf6yaeU2gs0m7mxWw9+hCOERmRLYJCniJIQQ4gZJYIVwpnZt2L4devTQE9bwcH1ZccGCMGoUbNjg3jJjF67Pxr67bQbFKxTjtR5v8uqjs1z2jRVC0zQ2/bydeq1qExSc94pTifwpwix7YIUQQtgnRZyEcKViRfj8c72K7fHjEBCgz7wGBro8NKuuVypeMuN7Fk9awq7Vexn69jM0f/QeVG5pPyR86uSBM5w9ep5HRnXydyhCeEy4TRsdWUIshBDCVrZmYJVShZVSvyulDmV8LORg3HGl1G6l1A6l1JasHi9EjhAeDnfeCTVreiV5vc4YYKTnuC7M2zKN4hWKMaXnHF55eBoxpy557Zoi9/rry79RSnF3pyh/hyKEx4SH2C4hlgRWCCHEDdldQjwW+EPTtKrAHxnPHWmpaVo9TdNs77KycrwQ+UrF2uWYu24KA2b0Zvuq3fSrPYIfFvyG1WrN8rk0NPaymul0YQS1eIEoljOdOCQpzs2sViu/f7qGBm3vJLJMEX+HI4THZC7iJHtghRBC3JDdBLYz8EnG558AD/v4eCHyNGOAkUdGdWLhrllUi6rM3MGLeL7VRE4fPuv2OZKIYzz3MZWObGYFp9jHUbaylIkMohybWeHF7yD3ucwZNrGcjSzjHEf8HY5T21bt5nx0DO16N3fvgIQE+Pln+OYb2LQJNM27AQpxm2z3wF5JSPVjJEIIIXKa7CawxTVNOwuQ8bGYg3Ea8JtSaqtSasBtHC9Evlaqcgmm//4Ko94fxNGd0TxQMWtzAAAgAElEQVRb93mWzFhBelq60+M0NKbSgaNsIYUE9H+KulSSSCGROTzOAaTqcQzRTKE9Q6nEOzzFPJ5mJLV5iXs4zk5/h2fXink/UzAynHu73e18YFISDB0KxYrpBcmeeQZat9b3dy9d6ptghciCoqGm/z6/GJ/ix0iEEELkNC4TWKXUKqXUHjuPzlm4TlNN0xoA7YEhSqlmWQ1UKTVAKbVFKbUlJiYmq4cLkesppXigbyve3/smDdvVZdELnzGk0VgObD7s8JiDrOco27Dg+AYwlUQ+4wVvhJxrnOcYY2jALn7HQgpJXCOJa1hI5iDrGc+9HGazv8PM5OiuaDb8sJVOg+4nyORkT3ZKCjRvDh98AImJejGya9cgPh6io6FPH3jnHZ/F/f/t3Xd4HNW9//H30Rb1Zsu9V4wBY7ApxpTQwQ6YEkpIgXupl5BQAhcHkgAhlyQQShJKfkBIIFQTig2mBAwBjGnGYFzBDVdZsmX1trva8/tjVvLK2pVWlnZX5fN6nn12dubMzHePxzvz1TlzRiQWBdm7R9QuUQusiIiEaTOBtdaeYK3dP8JrLlBkjBkEEHovjrKNbaH3YuAl4NDQopjWD637sLV2qrV2ar9+/drzHUV6lILBfbj1xRv49b+up7SonJ9Nu4kHrn6MmsraFmVf53581LS5zXUspoQt8Qg36cooYgF/41XuZRFzqKdlPf2ZH1JNGUEit2jXUcUfOQtL53W5raeWRczhVe5lAX+jjKJ2rf/k7c+TkZPOmVfPaL3gH/8Iy5dDXV3k5TU1cMMNsGlTu/YvEk99MncnsLuqfQSD6u4uIiKOjnYhngdcGJq+EFreTGeMyTTGZDdOAycBy2NdX0RaMsZw1FmH8djKe5l5+UnMvf8NLtnvWhbNbd5KuI2vY0q6PKSyg43xCjcpaqjgbs7hSkbwd67mKWbzEJdwCf15ntua6qWQNWxgCZbWB8eqpoxlLOhwXBbL89zGJfTnIS7hKWbzd67mSkZwN+dQQ0Wb29iwbCMfvPAJZ/50Btn5rTyHuKEB7rvP6ULcmmBQrbDSpaS6XU2P0mkIWkpr1AorIiKOjiawvwdONMasAU4MfcYYM9gY81qozABgoTFmKfApMN9a+0Zr64tIbDJzM/nZA5dw38LbyczL4JYz7+TWs++ieJPTzT6NVpKbMEGCpJIZz1ATqp4afsl0PucV/NRTTzUBfNRRSR1VzOVOHuFKAFbwH1Ji+Cmso4plvN3h2B7hSuZyJ3VUUUclAXzUU42fej7nFX7JdOrbaDV/4rbnychO56xrZ7a+sw0b2k5eAXw+mD+/Hd9CJP4KsnffB6tuxCIi0qhDCay1tsRae7y1dlzofVdo/jZr7YzQ9Hpr7YGh137W2v9ra30RaZ+J0/bhoc/v5OI7LmDxG19y8cRreeZ3L3GE7/yYElMPqQzngAREmhiv8xe2sy7qvb/11PA+T7CWz2jAH3PXYF8r9xLHYh2LeZ8noiaofurZzjpe489Rt7FkwTIWvvgJ3/v5aeT0yW59h34/pMT4M+9TgiBdS7OBnCo1kJOIiDg62gIrIl2E2+Pm/Nln8reV9zHl5AN57OanefbAlfjeLmh1PS/pfJfrcOFKUKTxFSTIq9yDP8K9ruF81PEKdzOEfUmJ4bunkslIJnUotnn8ER9R7kUN8VPLfO4lGKFLs6/Ox/0//RsDR/Xn3BtOj76RhgYoLITU1NgSU2PgwAPbLieSQOEDOe3QSMQiIhKiBFakhxkwoh+3vnAD/zf/JmzAUnrSIdSefwTBrektynrJYAJHMosbkhBpfJRTHNN9pJYgq/mAiRxDOjkxbNkyjXM7FNtqFrZ5ry049++WRxjT7vFb5rB59VZ+9uClpKantlyxrAx+9SsoKIAxY2DiRKcF1pjWd5iRAddeG+vXEEmIgqywLsRV6iEgIiIOd7IDEJH4OPTUg5i87B6e+8Ncnv79C1S/Noi0m1aTfc1WbJqfNLI4nZ8zk2txdeGfAotlNR/yLo+xi230YQjH8d/swxEYWiZmlmDE+dG2nUIKl/IQ93E+viittqlk8H3uIK2D9wnHkrwCGEyLsss/XM2/7p7HqRcfzyEnT2650vbtcNhhUFTkPDonVunpcNRRcMQRsa8jkgB9w7oQF6sLsYiIhHTdq1YR6TBvmpcf3XIOx//wKP7688f56CZD2v87nHN/fzKzzj0Ll+na3YZLKeS3nEIR6/BRg8ViMCziOQYylpt5g3wGNlsnlwF4SIuajO5mGMNUAA7hdK7k7zzExQDUUw04LdSWIOdyGzO5usPfZwxTWcyr0MY9tx7SyGVA0+fS4nJ+e949DBzVn8vv/nHklc46C7Ztg0Ag+oZdLqd7MYDH43yeMQOefLLtVlqRBBuUl9Y0va0shsHIRESkV1ACK9ILDB4zkN+8fCNLFizj//38cR76/hze+/OXXHHPRex72LhkhxdRLVXczBGUsIUgu5Myi6WeajazgpuZxt0sIz1stGUXLk7hJ8zjrqiDOIHTqnp6WNfp6ZzHFL7LQp7mK97G0sB4juA7XEQ2fTrlO53G9SzjnaYEORIPqZzKVU33JDc0NPC7H/yJyl1V/HbR/5GZk9FypeXL4csvW09ewelOfPbZzn2x48bBpZc67yJd0NC83bc9bFUCKyIiIboHViTerIWPP4bzz4f994cpU+C3v4Xilvc4xtvBxx/Ag5//gWtvPY3CL9fws2k38bs+J1N8+bWwMcnPgd26FX75Szj4YDjgAN558jjKG7Y3S17DBQlQTjH/4R8tlp3Gz8ljIC48EddNJYPJnMwEjmw2P41MTuBSruM5fs6/OI3rOi15BdiXo5jMKaQSIQkFXHjIZSDf5bqmeY/d9AxfLFjGT++/hLGTR0Xe8AsvxDZYU2qq0+L60ktw551KXqVLG5IflsCWKoEVERGHEliReKqthZkz4YQT4PnnYcUKWLIE7rgDRo50um4mUjCI63/+hxm/v4J/1M3jAruKhaWZ/NcjG3lk7BlU/Pq3iY2n0f33w9ix8Mc/whdfwPLlzD/yM3yu1kfs9VHDK9zdYn4medzBJ4zjMLykN93j6yUdD6kczY+5ludivle2sxgM1/AMR/NjPKTixblAd+HGSzrjOIzf8ymZ5AHw8l9eZ85dc/nu5Sdyyn8fF33Du3bt7hrcGr8fKtoe4EqkKxiUm97Us72osg5fILZ7yEVEpGdTF2KReDr/fHj3XajbIxGrDbUmXHYZ9OsHJ5+cmHhuuAGeegpqa8kA/osVzGQ9j9n9eT4whld/u5jvfXEjZz11S+SuqvHw7LNw440t6mjnsNhW38mmiPPzGMDtfMAWVvEpL1FNGQUM5wjOI5d+HY16r7nxcBkPcR6/YRHPsZNNZJLHoZzJUPZtKvfKQ2/ywNWPMf2MQ7jqLxe3vtGRIyEtreVxtievF4YM6fiXEEkArzuF/tmpFFXUYy1sL69jeN8E/S6JiEiXZaxtfTCRrmjq1Kl28eLFyQ5DpHUrVsAhh+xOVqPZf39Ytiz+8ZSUOMlLlBFqvyWHx5nIQjOU7D5ZnHvDLGZddQrpmWkRy3cKa2HYMKf78B6+XweBCE+K2ZMbL8+0cq9rm4JBePttuOsu5z7SlBQ48ki4/nqYNm3vt9sB8x9+i/uueJhpp0/lV3Ouw+ON3BW6yfbtMGpU2wlsdrbTdT0tjv+m3ZQx5nNr7dRkx9GdxePcfNaDH7JkUxkAT196GEeMaf251iIi0nNEOzerC7FIvPz1r7Hdl7h+PaxcGf94nn7aSc6iGEkFt/AxD6R/xIRRufztF0/xozFXctefbmNu3T28z5NUU965MX34IZRH3uY+H8a2iT3vY22X+nqni/dZZzlJ7M6dToL38stOt+/LLnOS7AR68U/zue+Khzls5sH88rkYkleAgQPhgguc57lGk5HhtHQreZVuZGj+7mN6U0lNEiMREZGuQgmsSLysWhXbfYkeD3z7bdzD4Ztv2m4NBsbbXdz236M47oM8avbbwL+vXc6DY9/hvrv+wCUVQ3mUqwjg75yYWvneZ/4eUqtaXz2VTM7gxr3f/2WXwXvvQfUeowIHg1BT43S3/s1v9n777RDwB3jg6sd46Np/cORZh/Hr53+ONzWG5LXRQw/B8cdDZmbzR+KkpDjJ60UXwU03dXrcIvE0ut/uZy+v29HGD4KIiPQKSmBF4iU7O7Zy1rbectZZcnJietZng9vwh9MeZfn0p8hc8B8y336PlAmVVN44gZ3Dj+eV2Yv4TeEsGoghOW9LRkbUVuFJb8FRT0VPYlPJ4Gh+yCRO3Lt9FxbCc8+1ntTX1DgDS7XVNbeDSovLufGk23n5L69z9jUz+eVz1+JN87ZvI14vzJ0Lr74Kp57qtMoOHgznnusk6Q88oGe9Srczrv/u39E1xUpgRURECaxI/Jx/fmxJbDAIhx8e/3hOPz2mRPnT0/ysHrweH05i5zluB1lvv0/WpwvwnLydmj+O5qOR6cz+r9ms/6qDj9459tio3awNcNkV8P2bIWcHpAcyyCCHdLLJoR8X8Dsu5aG9H0n4mWdiK2cMzJ+/d/uIwRfvLOPKKf/L6k/WMPufP+OKey7C5XLt3caMge98x4m3sNC5t/iZZ2Cqbu2U7mls/93PeF6rBFZERNAoxCLxc+aZcOWVrZdJT4fLL0/MfYmHHgojRsDq1U7SHInXy8u3pVOf0vK+VPfUUtzPfULD2uXU/2kcX/0dLn/8eg4+4QDOvvY0pp58ICmt3GMbUX4+nHMOzJkTcXApA8z8i+GUeUNZs+4pKighh36M43Bc7GWS12jjxqgDWjXj9zvJYCfz1fv5+83P8K97XmHYPoO5/ZXZ0Z/zKtJLjSzIwJViaAhatpbVUuMLkOHVpYuISG+mFliRePF64bXXICsrctfN9HSYNAl+m6BnrxoD8+Y5SWOkFr7QI1a+HdN6K4drbDUZf/mS3E1vcPEdF7Bx5RZunnkH/zXhaubcNZeyHe0c6OmBB2DMGEiNMORwSgrk5OB69XUmpBzFoZzBBKZ3PHkF6NMH3DFcCLvdkJvb8f2FWb5wFT855Eb+dc8rnHbFSTz4+Z1KXkUiSHW7GNHH6TliLazfUd3GGiIi0tMpgRWJp8MOg08/hTPOcFpZc3KchLZvX/jFL5x7ExM5KuyYMc6jYn70IyeBzslxujlnZTmtxUuWEGuPXNPHx/mzz+Sf6x/gF0/+jL6D8nnkxif5/tDLufXsu1g07zMC/kDbG8rOhk8+gWuucRLF7GwnrrQ0OO88+OIL2G+/jn3vSL73PWcArbYEAjBjRqfscuOqLdxy5p1ce/SvqS6r4bevzOZnD15KWkYMzwsS6aXCuxF/vb0yiZGIiEhXoOfASs9WVOR0FU1Ph4kTI7c8JsquXc6ou6mpsM8+sbX+xVNlJaxb57Ryjh/flEhfw0S2sqrN1YeyH/eyvNm8jSs38/qjC1jw9ELKisvJ65fDcRccxXE/OIrxU0Zj2hpEyOeDr792ksbRozu95bOF6dPhs8+cbsKRpKU5XZyfeKJDuynetIOn/+9FXv/bAtIy0zjvxjM465qZSly7ED0HtuPidW7+09truPftbwC4cNoIbpu1f6fvQ0REup5o52YlsNIzff45zJ4NCxc6CWNDg5OMXHcdXH99bC1vvdS7/IO/cRX1RO+ql0oml/Igx/DjiMsD/gCL31zKvx9/l49f+Ry/L8DQ8YM47oKjOPb86QwdPzhe4bdPcbHTSr59e8uRhhv/6PH++3s9SvSWNYU89/uXeOuf72MMfPeKk/jBL88mr1+cE3NpNyWwHRevc/N/vi7mor9/BsCBw/KY+5Ppnb4PERHpepTASu/x1ltOl92aCA+9z8iAadPg9deVxEbhx8evmM4mluOn5eNjPKQynAO4nQ/x0PajXipLq/jghU945+kP+Oq9lVhrGTFxKNNOm8rhp01lwqFjcbmT2DJeXg5//jP86U9QVeXcaNe/v/OHjiuuiHxvbit89X4+fW0Jbz3xHh+/shi3182pFx/PuTecTv/h/eL0JaSjelsCa4w5BfgT4AIetdb+fo/ls4DbgSAQAK6x1i5sbZvxOjeXVvs46Pa3APC6Ulh220mkJvM3Q0REEkIJrPQOlZXOsy+rWhmIKD0dfvlLuOmmxMXVzdRSxZ/5AUv5N5YgAXy48WJIYTIn81OeJJ2stje0hx1bSvjghY/56JXFLHt/FQ2BBrLyMpl83P5MOfFADjp+fwaPGdh2V+N4CAadbt4pKc5AV+2IIeAPsHzhav7z3CLef34RlaXV5A/I5eSLjuXMq2fQZ2B+HAOXztCbElhjjAv4BjgR2AJ8BnzfWrsyrEwWUG2ttcaYScAca+2E1rYbz3PzMXe9y8YS54+SL/9kOpOH5cVlPyIi0nVEOzdrLHrpWZ54wmlBa01tLdx7L9x4Y3Lvie3C0sniRuayg40sYg5lbCefQUzjXPoxfK+3229oX866eiZnXT2TqrJqFr/5JZ+/9RWfv7WUhS9+AkDfwfkccPREDjhyXyYeMZ5R+w9PTAttSgoUFMRcfNf2Ur5YsJyP53/O4je+pKqsmtR0L9PPPJQTfng0B58wKbktyyLRHQqstdauBzDGPAvMApoSWGtt+F8BM4Gk/rV78rC8pgT2sw27lMCKiPRiSmClZ3n2WaiO4TEL9fWwbBlMnhz/mLqxfoxgFjfEZdtZeZl857zpfOe86Vhr2fLNNr58dwXLPljJV++t5D/PfghAWkYq46aMZp9DxjL2oFGMnjScofsMxuNNXBfwhoYGCtcV8c3idSz7YBVL31vJ5tVbAcjrn8v0Mw7l8NOmMOXESaRnpScsLpG9NATYHPZ5C3DYnoWMMWcCvwP6AzMjbcgYcxlwGcDw4Xv/x622HD66L3O/3AbAwrU7ufTo0XHbl4iIdG1KYKVniSV5Bae1rbY2vrFIzIwxDNtnCMP2GcJpV5yEtZbtG4pZ9fE3rP50Las/XcPcB97AX++MFuxyuxgwsh9Dxg5kwIh+DBw1gIIhfcgfmEf+gFzyB+SSmZsRc5Lrq/NRUVJJRUkVpUVlFG3cSfHGHWzfWMzm1dvYuGIz9bU+ADKy09n/qAmcfNGxHPidiYyfOoaUFD2RTLqVSP3jW7SwWmtfAl4yxhyNcz/sCRHKPAw8DE4X4k6Os8mRY3f3jvh0wy58gSBet/7fiYj0RkpgpWeZMAGWLnXuZ2xNfT2MGJGYmKTdjDEMGj2AQaMHcNwFRwHOfaZbvilk/dJv2bhyC1vXFrJtXRGrP11L5a7I9zx7vG7Ss9NJzfDicqWQEnoFG4L46vzU1/rw1fqaktNwKSmGgqF9GTJuEDMvO5FRk0Yw9qCRievSLBI/W4BhYZ+HAtuiFbbWvm+MGWOMKbDW7ox7dBEM65PB8D4ZbNpVQ62/gc83ljJtTN9khCIiIkmmBFZ6lp/+FObNa7sl9rDDnMGepNtwe9yM3G8YI/cb1mJZdUUNJdtKKS0qo6yonNKicqoraqitrKWmso66mjps0BJsCBJsCOJyu/CkevCmefCmecnKzySnbza5Bdnk9sthwIh+FAzpg9ujn0jpkT4DxhljRgFbgfOBC8ILGGPGAutCgzgdDHiBkoRHGuaocQU89ckmAP69crsSWBGRXkpXZ9KzHH44TJ0KH3/stLJGkpEBf/hDYuOSuMrMySAzJ4PhE4YkOxSRLs9aGzDGXAW8ifMYncestSuMMVeElv8VOBv4sTHGD9QC59kkP7bglP0HNiWwbyzfzq9mTiQlJQkjlouISFIpgZWexRh49VX47ndh8WLnWbCN11wZGc7yOXOcFlgRkV7KWvsa8Noe8/4aNv0HoEv9pe/w0X3Jy/BQVuOnsLyOpVvKOGi4HlElItLbaAQE6XmysuDdd+HNN+Gcc2DSJCdhveMO2LoVZsxIdoQiItJOHlcKJ00c0PT5pS+2JjEaERFJFrXASs9kDEyf7rxERKRHOGPyEOYs3gLAS0u2MvvUCWR4dSkjItKbqAVWREREuoVpY/oyqiATgMr6AK8sjTp4soiI9FBKYEVERKRbMMZwwaHDmz7/Y9FGkjy2lIiIJJgSWBEREek2vjdlKGke5/JlVWEFC1YVJzkiERFJJCWwIiIi0m3kZ3r5wWEjmj7/+Z01aoUVEelFlMCKiIhIt3L50aPxup1LmK+2lPPKV4VJjkhERBJFCayIiIh0K/1z0rhw2u5W2Dvmr6LGF0hiRCIikihKYEVERKTb+enx4yjI8gKwvaKOe9/6JskRiYhIIiiBFRERkW4nJ83D/54yoenzows38PH6kiRGJCIiiaAEVkRkb5SVwV13wdCh4HJBWhqcdhosXJjsyER6je8dPJSjxhUAYC38fM5Symp8SY5KRETiSQmsiEh7bdgA++4Lt9wCW7dCMAj19TB/Ppx8Mtx8c7IjFOkVUlIMd33vQHLTPQBsLavlJ08vwd8QTHJkIiISL0pgRXqS8nIoKgK/P9mR9FwNDXDssVBcDLW1zZdZCzU1cN998MwzyYlPpJcZmJvGnd+b1PT5w7Ul/OaVlXq0johID6UEVqS7s9ZJlg44AAoKYORIyMuDK6+ETZuSHV3PM38+lJQ4ra7R1NTAr37l/NuISNydvN9Arj1hfNPnf368UYM6iYj0UB1KYI0xfYwxbxlj1oTe8yOU2ccY82XYq8IYc01o2a3GmK1hy2Z0JB6RXsda+OEP4dJLYflyCASgrs5JoB59FCZNgqVLkx1lz/Lww1BV1Xa5wkL4+uv4xyMiAPzs+LF8d9Kgps9/fmctf1mwRi2xIiI9TEdbYGcDC6y144AFoc/NWGu/ttZOttZOBqYANcBLYUXubVxurX2tg/GI9C733w8vvwzV1S2X+f1Ol+ITTwRfkgY1sRbWrYPFi2Hz5uTE0Nm2b4+tnMcDO3bENxYRaWKM4e5zD+TYffo1zbv7rW+4/dVVBINKYkVEeoqOJrCzgMdD048DZ7RR/nhgnbV2Ywf3KyLBIPzud05ra2tqa+HFFxMTUyNr4amnYPx4p2vz8cc701OmwBtvJDaWzjZgQGzl/H6nS7eIJEyq28VDP5zCkWN3/9977MMN/PSZL6jxBZIYmYiIdJaOJrADrLWFAKH3/m2UPx/Yc2STq4wxXxljHovUBbmRMeYyY8xiY8ziHWrVEIFly6Cysu1yVVXw97/HP55w110Hl18Oa9c6CXRFhdO1eckSOPtseOCBxMbTmS67DLKy2i43aBBMmNB2ORHpVGkeF49eOJVT9x/YNG/+skLOfGAR63fE0P1fRES6tDYTWGPM28aY5RFes9qzI2OMFzgdeD5s9kPAGGAyUAjcHW19a+3D1tqp1tqp/fr1i1ZMpPcoL3eePxqLXbviG0u4115z7hON1K0ZnBbjG26AFSsSF1NnmjkT+vaFlFZ+PjMy4LbbwJjExSUiTdI8Lu6/4GAuOmJk07yviyo5/f4PeemLLbovVkSkG2szgbXWnmCt3T/Cay5QZIwZBBB6L25lU6cCS6y1RWHbLrLWNlhrg8AjwKEd+zoivcjgwbHf2zpyZFxDaeaOO9ru1uz3wz33JCaezuZ2wzvvQP/+kJ7efJkxTvJ69dXwgx8kJz4RAcCVYrj19P248+xJpLqdy52q+gDXPreUy/75OcWVdUmOUERE9kZHuxDPAy4MTV8IzG2l7PfZo/twY/IbciawvIPxiPQeY8c6r7ZkZTmP1EkEvx8++qjtcoEAzG3t56KLGz0aVq6EW291/pCQkgJeL8yYAa+/7iTxItIlnHvIMF74nyMY3iejad5bK4s46d73efLjjTRogCcRkW7FdKQbjTGmLzAHGA5sAs6x1u4yxgwGHrXWzgiVywA2A6OtteVh6/8Tp/uwBb4FLm+8p7Y1U6dOtYsXL97ruEV6jPnz4ZxznPtMI/F4YOJE+OKLxHRnraqC/HwnQW1LVlZs9/B2B9aqu3A3Z4z53Fo7NdlxdGdd/dxcXR/gd6+v4smPmz8fe8LAbG45bT+mjembpMhERCSSaOfmDiWwydLVT5IiCfXgg3D99U7S6Pfvnp+VBUOHwvvvQ6LuG7cWcnJie07qPvvA6tXxj0kkBkpgO667nJsXrd3J/77wFVtKm//h79h9+nHtieOZNDQvSZGJiEi4aOfmjnYhFpFku/JK+PJLuPRSpztr374wdSo88ggsXZq45BWcVshLLnG607YmMxOuuSYxMYmIhDlibAFvX3cMPz9xPOme3QPhvfv1Dk6//0Mu/sdnLN1clsQIRUSkNWqBFZHOtXUr7LefM0pyJC6Xk2ivXBnb42hEEkAtsB3XHc/NheW13PnG17z85Vb2vBw6ZGQ+Fx85ihMnDsSVolsEREQSTS2wIpIYQ4bAe+9BQUHLBDU7G4YPh4ULlbyKSNINyk3n3vMm8+9rjua7kwY1u5X9s29LueLJJXznj+/yyPvr2VlVn7xARUSkiVpgRSQ+6upgzhx49FEoKXES25/8xHmOqtud7OhEmlELbMf1hHPz19sreeg/a3n1q0ICe4xO7E4xHL9vf86dOoxjxvfD7VIbgIhIPGkQJxERkSiUwHZcTzo3by+v458ff8tTn2yirMbfYnm/7FRmHjCIGQcMYuqIfFLUxVhEpNMpgRUREYlCCWzH9cRzc62vgVeWbmPO4s0s3lgascyAnFRO3d9JZqeMyNf9siIinSTauVn9+EREREQiSPe6OPeQYZx7yDDW7aji+cVbeGHJFnZU7r4ftqiinn8s+pZ/LPqWvAwPR4/rx3ET+nPM+H7kZ7YxIruIiLSbWmBFRKTXUwtsx/WWc3OgIcinG3bx6rJC3ly+nZJqX8RyKQYmD8vjmPH9mTamLwcOyyXV7YpYVkREWlIXYhERkSiUwHZcbzw3BxqCfLJhF/OXFbJgVRFFFdFHKk7zpDB1RB8OH92HaWP6MmloHh4NBL2isf8AABMqSURBVCUiEpW6EIuIiIh0IrcrheljC5g+tgB7xv6sLKzg3dXFvLO6mC82lzV7tmydP8jCtTtZuHYnAOkeFwcMzeXg4fkcPDyPg4bn0y87NUnfRESk+1ACKyIiItJBxhj2G5zLfoNzueq4ceyq9vHBmh18tK6Ej9aXsLGkpln5Wn8Dn27YxacbdjXNG9YnnYOG5XPQ8Dz2H5LLvoNyyErVpZqISDj9KoqIiIh0sj6ZXmZNHsKsyUMA2FZW25TMfrSuhK1ltS3W2byrls27apm3dBsAxsCovplMHJwTSo5z2G9wDn2z1FIrIr2XElgRERGROBucl87ZU4Zy9pShgPOs2S82lbJkUylLNpWxbGs5vkCw2TrWwvqd1azfWc2rXxU2zR+Yk8b4gdmM75/F+AHZjBuQxdj+WWSneRL6nUREkkEJrIiIiEiCDcxN49QDBnHqAYMA8AWCrCysYMnGUpZtLWfFtnLWFlcRjDDW5vaKOrZX1PH+NzuazR+cm8a4AdmMH5DFuAHZjOmXyci+mfTJ9GKMnk8rIj2DElgRERGRJPO6U5g8LI/Jw/Ka5tX5G1i9vZIV28pZsa2CFdsqWF1YQf0eLbWNtpXXsa28jvf2SGyz09yMKnCS2VEFzmtkQSaj+maSm6FWWxHpXpTAioiIiHRBaR5Xi6Q20BDk25Jqvimq4puiStYUV7GmqJL1O6oJRGquBSrrAny1pZyvtpS3WJaf4WFkQSbD8jMYmp/O0ND7sD4ZDM5L07NrRaTLUQIrIiIi0k24XSmM7Z/N2P7ZzAh1PwbwNwT5dqeT2K4pdhLbDTuq+bakmhpfQ9Ttldb4Kd1UxhebyiIuH5CTytD8DIaFJbdD8zMYlJfGoNw0Mry6lBSRxNKvjoiIiEg353GlMG5ANuMGZAO7E1trLTsq69mw00lmN+ysYcPOKr7dWcO3JdVRuyM3Kqqop6iins83lkZcnpPmZlBuOgNznYR293t60+fsVLfuwRWRTqMEVkRERKSHMsbQPyeN/jlpHDa6b7NlwaBle0UdG0tq2FJaw5bSWraU1rK5tIatpbUUltdGHEQqXEVdgIq6Sr4uqoxaJtPrYmAome2fnUb/7FT6hb36Z6fRLzuVnDQluiLSNiWwIiIiIr1QSophcF46g/PSgb4tlvsbghSW1TUlt5tD71tLaymsqGV7eR3+hjYyXKDa18C6HdWs21HdarlUd0oooW2Z3DbO65uVSt9ML2ke3Zsr0lspgRURERGRFjyuFIb3zWB434yIy4NBy64aH9vL6ygsr2N7eW3ovY5t5bVN89vqptyoPhBsagVuS4bXRd8sL30ynYS2b6aXPlne0HTq7mklvCI9jhJYEREREWm3lBRDQVYqBVmp7D8kN2IZay1lNX4Ky+soqqhjR2U9xZWN7/XsqKxnR1U9xRX11PqjDza1pxpfAzW7atm8q+1kF1omvH0yveSle8jP9JKb7iE/w0t+hofcjMZpL2meFHVpFumClMCKiIiISFwYY8jP9JKf6WXi4Jyo5ay1VPsaKA4luY1J7Y6q+mbJ7q7qenZV+2LquhyuvQkvOM/mzc/wkJfuJS+U2OZleMjLaPwcmg4lwnnpHnLSPWrtFYkzJbAiIiIiklTGGLJS3WT1y2J0v6xWy1prqagLsKvax67qenZW+ULTPnZWOQluSZWPktDykipf1GfktsYXCDaNwtweXncKuekectLc5KR7yElzEtvcdHfTdE6axynTbJ5T3uNKaXesIr2JElgRERER6TaMMeSmOwngqILMNsuHJ7wlVfWUVPsoq/E5z8Ct8VEeei+r8VMWNu1riO3e3T35AkGnFbmyfYlvowyvK5TUukOJcPMENzvNTVaqh6w0N9lpbrJT3WSluclKdZOd5iEr1Y0rRV2fpedSAisiIiIiPVZ7E15wkt5af0NTQuskuaHpWj+l1T7Kav1NiXBZKOmtqPO3u3vznmp8DdT4GthesffbyPC6Qgmtm6w0j5PkhhLd5klvKCEOm9eYBCsRlq5KCayIiIiISBhjDBleNxled+gxQ7Gx1lLnD1JR56ei1k95rT80HaCizk95zR6f91heUetv89m7sWhMgov3shW4UWMinJXqJjPV3fQ5I9VNVqqLDK8zP9Prct5TXWQ2zguf73WWudU9WjqBElgRERERkU5gjCHd6yLd62JATlq71w8GLdW+ABV1ASpqw5PgQNN0VX2AqroAVfUBKusDVNb5mz5X1QWo8gWwnZAEQ+clwo287hQnAW5MhJsluKEEuEXi6yYjlBhnhOo2w+siw+Mm3evC61ZS3NsogRURERER6QJSUgzZaR6y0zwMaUfLb7jGJLgxoXWS3Mak1+9MN5sX/0S4kS8QZFfAx67qztumO8XsTmq9btI9ruaJrtdJdDOa5u+RCHtdpHvcu6dD62R4XaS69SilrkgJrIiIiIhIDxGeBBP58bwxCU+Eq+sDVNc3OO++xvfI82tC69T4Gpz3+t3lO6N79J4CQUtlnZOQQ+e0FDdKMZDu2Z30hie+4UlvmseZn+5xXmmelKZ5ae7Qe2h++h7zlSS3nxJYERERERFpplki3Aka7w9ulvhGmN6d+Aaoqm9olhDX+Bqo9QVC7w3U+BtoiEdWHBK0OIm5ryFu+wCakt50j4u0sOS2WTLs2Z0oN0uGPY2JcPN1GsvvTqRTesw9yEpgRUREREQkrsLvDy7ISu2UbVpr8TUEnWS2KcF1kt4af0PT/FpfgFp/+PLQtD/QYr3GxLjG14AvsHePUmqvWn8Dtf4GSvHHdT8el2mWDKd5Ukh1706Sm0+nNJVJczcvn+pJaUqO00LlUsPKpXtd5KZ3zh8+IlECKyIiIiIi3Y4xxkmo3C7yMjp/+4GGoJNctpr0OolvfSBIXahsrb+BOn/os7+h6b3W50zX+YNNSWuikmQAf4PF39DY3Tp+xvbP4u3rjonb9pXAioiIiIiI7MHtSiHbldJp3agjaQha6gOh5DYQDEtyw5LeQJC6UGIcnhDX+Zonw+Hr1flbbiuOva2bSfPEt6uyElgREREREZEkcKXsfuZwPDV2t67zBZuS3cZW4zp/KEkOm18f1opcF3Cm6wNh85o+h2/Hee+sLuLRKIEVERERERHpwcK7W+cSvxblROgZQ1GJiIiIiIhIj6cEVkRERERERLoFJbAiIiIiIiLSLSiBFRERERERkW6hQwmsMeYcY8wKY0zQGDO1lXKnGGO+NsasNcbMDpvfxxjzljFmTeg9vyPxiIiISNuinZfDlv/AGPNV6LXIGHNgMuIUERHZU0dbYJcDZwHvRytgjHEBDwCnAhOB7xtjJoYWzwYWWGvHAQtCn0VERCRO2jgvN9oAHGOtnQTcDjyc2ChFREQi61ACa61dZa39uo1ihwJrrbXrrbU+4FlgVmjZLODx0PTjwBkdiUdERETa1Np5GQBr7SJrbWno48fA0ATHKCIiElEi7oEdAmwO+7wlNA9ggLW2ECD03j8B8YiIiPRmrZ2XI7kYeD2uEYmIiMTI3VYBY8zbwMAIi2621s6NYR8mwjwbw3p7xnEZcFnoY70xZnl7t9EFFAA7kx3EXlDcidVd44buG7viTqyuGPeIZAeQQDGfl40xx+IksEdGWR5+bq4yxrTVKysWXfH46IpUT7FRPcVG9RQb1VPbOrOOIp6b20xgrbUndHDHW4BhYZ+HAttC00XGmEHW2kJjzCCguJU4HiZ0D44xZrG1NuqgUV2V4k4sxZ143TV2xZ1Y3TXuHqS183ITY8wk4FHgVGttSaQNhZ+bO4uOj9ionmKjeoqN6ik2qqe2JaKOEtGF+DNgnDFmlDHGC5wPzAstmwdcGJq+EIilRVdERET2XmvnZQCMMcOBF4EfWWu/SUKMIiIiEXX0MTpnGmO2ANOA+caYN0PzBxtjXgOw1gaAq4A3gVXAHGvtitAmfg+caIxZA5wY+iwiIiJxEu28bIy5whhzRajYr4G+wIPGmC+NMYuTFK6IiEgzbXYhbo219iXgpQjztwEzwj6/BrwWoVwJcPxe7Lq7DuevuBNLcSded41dcSdWd427x4h0XrbW/jVs+hLgkkTHFaLjIzaqp9ionmKjeoqN6qltca8jY227x1MSERERERERSbhE3AMrIiIiIiIi0mFdNoE1xpxjjFlhjAkaY6KOZGWMOcUY87UxZq0xZnbY/D7GmLeMMWtC7/kJirvN/Rpj9gndU9T4qjDGXBNadqsxZmvYshkt95KcuEPlvjXGLNvznqguXt/DjDHvGmNWhY6pq8OWJbS+ox2vYcuNMebPoeVfGWMOjnXdJMf9g1C8XxljFhljDgxbFvGY6SJxf8cYUx727//rWNdNctw3hMW83BjTYIzpE1qWzPp+zBhTbKI85qyrHt/Sdeg42C3S/+XWznnGmF+E6u1rY8zJyYs8viL9zuxNvRhjpoTqd23odynS46W6rSj1FPWapxfXU8RrRB1Tu7VSR8k7nqy1XfIF7AvsA/wHmBqljAtYB4wGvMBSYGJo2Z3A7ND0bOAPCYq7XfsNfYftwIjQ51uB65NQ3zHFDXwLFHT0eycybmAQcHBoOhv4Juw4SVh9t3a8hpWZAbyO85zGw4FPYl03yXEfAeSHpk9tjLu1Y6aLxP0d4NW9WTeZce9R/jTgnWTXd2jfRwMHA8ujLO9yx7deXeel46BFfbT4vxztnAdMDNVXKjAqVI+uZH+HONVLi9+ZvakX4FOcQUhN6Hfp1GR/twTU061EuObp5fUU8RpRx1RMdZS046nLtsBaa1dZa9t6IPqhwFpr7XprrQ94FpgVWjYLeDw0/ThwRnwibaG9+z0eWGet3RjXqNrW0frqsvVtrS201i4JTVfijLo5JEHxhWvteG00C3jCOj4G8ozzjORY1k1a3NbaRdba0tDHj3GeK5lsHamzLl3fe/g+8ExCImuDtfZ9YFcrRbri8S1dh46DtkU7580CnrXW1ltrNwBrceqzx4nyO9Ouegn97uRYaz+yzlX1EyTuuiUhYvg9Dteb6ynaNaKOqZC9uI6Oex112QQ2RkOAzWGft7C7QgdYawvBqXigf4Jiau9+z6flxedVoe51j5kEdcUl9rgt8G9jzOfGmMv2Yv3O1q79GmNGAgcBn4TNTlR9t3a8tlUmlnXjpb37vhjnr2qNoh0z8RZr3NOMMUuNMa8bY/Zr57rxEPO+jTEZwCnAC2Gzk1XfseiKx7d0HToOmov0fznaOa+3111762VIaHrP+b1BpGse1RMtrhF1TEUQ4To6KcdThx6j01HGmLeBgREW3WytnRvLJiLMi/uwyq3F3c7teIHTgV+EzX4IuB3ne9wO3A38995F2mJ/nRH3dGvtNmNMf+AtY8zq0F/54qYT6zsL50L/GmttRWh23Oo7UggR5u15vEYrk5RjPSTmfRtjjsVJYI8Mm53wY6YxnAjz9ox7CU73/arQvRsvA+NiXDde2rPv04APrbXhf2VPVn3Hoise39J16DhorsX/5VbKqu4i029Oc9GueXp9Pe15jdjKrZm9tq4i1FHSjqekJrDW2hM6uIktwLCwz0OBbaHpImPMIGttYajJuriD+2rSWtzGmPbs91RgibW2KGzbTdPGmEeAVzsj5tC2Oxy3dZ7xi7W22BjzEk4Xpffp4vVtjPHg/Kd7ylr7Yti241bfEbR2vLZVxhvDuvESS9wYYyYBj+Lcz1DSOL+VYybe2ow77A8ZWGtfM8Y8aIwpiGXdOGrPvlv04EhifceiKx7f0nUk8/9dlxPl/3K0c15vr7v21ssWmt/q0ivqq5Vrnl5dT1GuEXVMhYlUR8k8nrp7F+LPgHHGmFGh1szzgXmhZfOAC0PTFwKxtOh2hvbst8W9a6H/JI3OBCKO5hkHbcZtjMk0xmQ3TgMnhcXXZes7NMLZ34BV1tp79liWyPpu7XhtNA/4sXEcDpSHuq7Esm7S4jbGDAdeBH5krf0mbH5rx0xXiHtg4wh4xphDcX4TS2JZN5lxh+LNBY4h7JhPcn3Hoise39J16DgIaeX/crRz3jzgfGNMqjFmFE5Pkk8TG3VStateQr87lcaYw0PngB+TuOuWpGnlmqfX1lMr14g6pkKi1VFSjyfbBUa3ivQKVcQWoB4oAt4MzR8MvBZWbgbOaFjrcLoeN87vCywA1oTe+yQo7oj7jRB3Bs6Fcu4e6/8TWAZ8FToABnWVuHFGhlwaeq3oLvWN053Vhur0y9BrRjLqO9LxClwBXBGaNsADoeXLCBuBO9qxnqB6bivuR4HSsPpd3NYx00XivioU11KcwaeO6A71Hfp8Ec4gCeHrJbu+nwEKAT/O7/fF3eH41qvrvHQcNNVDxP/L0c55oWU3h+rta3rI6KdR6ibS70y76wWYinPBvQ64HzDJ/m4JqKeo1zy9uJ4iXiPqmIqpjpJ2PJnQxkRERERERES6tO7ehVhERERERER6CSWwIiIiIiIi0i0ogRUREREREZFuQQmsiIiIiIiIdAtKYEVERERERKRbUAIrIiIiIiIi3YISWBEREREREekWlMCKiIiIiIhIt/D/AZncN9NCfKGQAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "draw_regularization_example(X, Y)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Regularyzacja\n", + "\n", + "Regularyzacja jest metodą zapobiegania zjawisku nadmiernego dopasowania (*overfitting*) poprzez odpowiednie zmodyfikowanie funkcji kosztu.\n", + "\n", + "Do funkcji kosztu dodawane jest specjalne wyrażenie (**wyrazenie regularyzacyjne** – zaznaczone na czerwono w poniższych wzorach), będące „karą” za ekstremalne wartości parametrów $\\theta$.\n", + "\n", + "W ten sposób preferowane są wektory $\\theta$ z mniejszymi wartosciami parametrów – mają automatycznie niższy koszt.\n", + "\n", + "Jak silną regularyzację chcemy zastosować? Możemy o tym zadecydować, dobierajac odpowiednio **parametr regularyzacji** $\\lambda$." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Regularyzacja dla regresji liniowej – funkcja kosztu\n", + "\n", + "$$\n", + "J(\\theta) \\, = \\, \\dfrac{1}{2m} \\left( \\displaystyle\\sum_{i=1}^{m} h_\\theta(x^{(i)}) - y^{(i)} \\color{red}{ + \\lambda \\displaystyle\\sum_{j=1}^{n} \\theta^2_j } \\right)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "* $\\lambda$ – parametr regularyzacji\n", + "* jeżeli $\\lambda$ jest zbyt mały, skutkuje to nadmiernym dopasowaniem\n", + "* jeżeli $\\lambda$ jest zbyt duży, skutkuje to niedostatecznym dopasowaniem" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Regularyzacja dla regresji liniowej – gradient\n", + "\n", + "$$\\small\n", + "\\begin{array}{llll}\n", + "\\dfrac{\\partial J(\\theta)}{\\partial \\theta_0} &=& \\dfrac{1}{m}\\displaystyle\\sum_{i=1}^m \\left( h_{\\theta}(x^{(i)})-y^{(i)} \\right) x^{(i)}_0 & \\textrm{dla $j = 0$ }\\\\\n", + "\\dfrac{\\partial J(\\theta)}{\\partial \\theta_j} &=& \\dfrac{1}{m}\\displaystyle\\sum_{i=1}^m \\left( h_{\\theta}(x^{(i)})-y^{(i)} \\right) x^{(i)}_j \\color{red}{+ \\dfrac{\\lambda}{m}\\theta_j} & \\textrm{dla $j = 1, 2, \\ldots, n $} \\\\\n", + "\\end{array} \n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Regularyzacja dla regresji logistycznej – funkcja kosztu\n", + "\n", + "$$\n", + "\\begin{array}{rtl}\n", + "J(\\theta) & = & -\\dfrac{1}{m} \\left( \\displaystyle\\sum_{i=1}^{m} y^{(i)} \\log h_\\theta(x^{(i)}) + \\left( 1-y^{(i)} \\right) \\log \\left( 1-h_\\theta(x^{(i)}) \\right) \\right) \\\\\n", + "& & \\color{red}{ + \\dfrac{\\lambda}{2m} \\displaystyle\\sum_{j=1}^{n} \\theta^2_j } \\\\\n", + "\\end{array}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Regularyzacja dla regresji logistycznej – gradient\n", + "\n", + "$$\\small\n", + "\\begin{array}{llll}\n", + "\\dfrac{\\partial J(\\theta)}{\\partial \\theta_0} &=& \\dfrac{1}{m}\\displaystyle\\sum_{i=1}^m \\left( h_{\\theta}(x^{(i)})-y^{(i)} \\right) x^{(i)}_0 & \\textrm{dla $j = 0$ }\\\\\n", + "\\dfrac{\\partial J(\\theta)}{\\partial \\theta_j} &=& \\dfrac{1}{m}\\displaystyle\\sum_{i=1}^m \\left( h_{\\theta}(x^{(i)})-y^{(i)} \\right) x^{(i)}_j \\color{red}{+ \\dfrac{\\lambda}{m}\\theta_j} & \\textrm{dla $j = 1, 2, \\ldots, n $} \\\\\n", + "\\end{array} \n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Implementacja metody regularyzacji" + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [], + "source": [ + "def J_(h,theta,X,y,lamb=0):\n", + " \"\"\"Funkcja kosztu z regularyzacją\"\"\"\n", + " m = float(len(y))\n", + " f = h(theta, X, eps=10**-7)\n", + " j = 1.0/m \\\n", + " * -np.sum(np.multiply(y, np.log(f)) + \n", + " np.multiply(1 - y, np.log(1 - f)), axis=0) \\\n", + " + lamb/(2*m) * np.sum(np.power(theta[1:] ,2))\n", + " return j\n", + "\n", + "def dJ_(h,theta,X,y,lamb=0):\n", + " \"\"\"Gradient funkcji kosztu z regularyzacją\"\"\"\n", + " m = float(y.shape[0])\n", + " g = 1.0/y.shape[0]*(X.T*(h(theta,X)-y))\n", + " g[1:] += lamb/m * theta[1:]\n", + " return g" + ] + }, + { + "cell_type": "code", + "execution_count": 65, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "slider_lambda = widgets.FloatSlider(min=0.0, max=0.5, step=0.005, value=0.01, description=r'$\\lambda$', width=300)\n", + "\n", + "def slide_regularization_example_2(lamb):\n", + " draw_regularization_example(X, Y, lamb=lamb)" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "9a01a44941e544cb9df51b38c035da62", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "interactive(children=(FloatSlider(value=0.01, description='$\\\\lambda$', max=0.5, step=0.005), Button(descripti…" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 66, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "widgets.interact_manual(slide_regularization_example_2, lamb=slider_lambda)" + ] + }, + { + "cell_type": "code", + "execution_count": 67, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "def cost_lambda_fun(lamb):\n", + " \"\"\"Koszt w zależności od parametru regularyzacji lambda\"\"\"\n", + " theta = np.matrix(np.zeros(X.shape[1])).reshape(X.shape[1],1)\n", + " thetaBest, err = SGD(h, J, dJ, theta, X, Y, alpha=1, adaGrad=True, maxEpochs=2500, batchSize=100, \n", + " logError=True, validate=0.25, valStep=1, lamb=lamb)\n", + " return err[1][-1], err[3][-1]\n", + "\n", + "def plot_cost_lambda():\n", + " \"\"\"Wykres kosztu w zależności od parametru regularyzacji lambda\"\"\"\n", + " plt.figure(figsize=(16,8))\n", + " ax = plt.subplot(111)\n", + " Lambda = np.arange(0.0, 1.0, 0.01)\n", + " Costs = [cost_lambda_fun(lamb) for lamb in Lambda]\n", + " CostTrain = [cost[0] for cost in Costs]\n", + " CostCV = [cost[1] for cost in Costs]\n", + " plt.plot(Lambda, CostTrain, lw=3, label='training error')\n", + " plt.plot(Lambda, CostCV, lw=3, label='validation error')\n", + " ax.set_xlabel(r'$\\lambda$')\n", + " ax.set_ylabel(u'cost')\n", + " plt.legend()\n", + " plt.ylim(0.2,0.8)" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7AAAAHmCAYAAABK9WIBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde3Rc5WHv/d+jkTQajUb3i23ZxgYMNjY2GGM7gRLMLSTnhFxIiMmttCfQ0CYk7XsotF1NKKc9TdM0K6EtZUFKkpVD4VBybQ4hlDSEkJDEdgAHzMXG+CLLtu7SaKTRzGie94+95ybLsizNntFI389as2bv0Z6ZR5A4/uZ59t7GWisAAAAAAOa6smIPAAAAAACA6SBgAQAAAAAlgYAFAAAAAJQEAhYAAAAAUBIIWAAAAABASSBgAQAAAAAlwdOANcZca4x5zRizzxhz5yQ/rzPG/Icx5kVjzMvGmN/zcjwAAAAAgNJlvLoPrDHGJ+l1SVdL6pC0Q9KN1to9Wcf8uaQ6a+0dxpgWSa9JWmStjXkyKAAAAABAyfJyBnazpH3W2v1ukD4i6d0TjrGSQsYYI6lGUp+khIdjAgAAAACUKC8Dtl3S4az9Dve1bP8kaY2kTkm/lfRpa23SwzEBAAAAAEpUuYefbSZ5beJ65bdLekHSFZLOkvSfxpifWWuHcj7ImFsk3SJJwWDwotWrV3swXAAAAABAse3atavHWtsy2c+8DNgOScuy9pfKmWnN9nuSPm+dE3H3GWPelLRa0q+zD7LW3i/pfknatGmT3blzp2eDBgAAAAAUjzHm4Ml+5uUS4h2SVhljVhpjKiVtl/T9CcccknSlJBlj2iSdK2m/h2MCAAAAAJQoz2ZgrbUJY8wnJf1Ikk/Sg9bal40xn3B/fp+k/yXp68aY38pZcnyHtbbHqzEBAAAAAEqXl0uIZa19XNLjE167L2u7U9I1Xo4BAAAAADA/eBqwAAAAAOCVeDyujo4ORaPRYg8FM1BVVaWlS5eqoqJi2u8hYAEAAACUpI6ODoVCIa1YsULGTHYTFMxV1lr19vaqo6NDK1eunPb7vLyIEwAAAAB4JhqNqqmpiXgtQcYYNTU1nfbsOQELAAAAoGQRr6VrJv/uCFgAAAAAmIGBgQHde++9M3rvO9/5Tg0MDEx5zGc/+1k99dRTM/r8+YqABQAAAIAZmCpgx8fHp3zv448/rvr6+imPufvuu3XVVVfNeHyna+KYT/U7nO5x+UDAAgAAAMAM3HnnnXrjjTd0wQUX6Pbbb9fTTz+tbdu26UMf+pDOP/98SdJ73vMeXXTRRVq7dq3uv//+9HtXrFihnp4eHThwQGvWrNHNN9+stWvX6pprrtHo6Kgk6aabbtJjjz2WPv5zn/ucNm7cqPPPP1+vvvqqJKm7u1tXX321Nm7cqD/4gz/QGWecoZ6enhPG+uSTT+otb3mLNm7cqA984AMaHh5Of+7dd9+tSy+9VP/+7/9+wv7DDz+s888/X+vWrdMdd9yR/ryamhp99rOf1ZYtW/Tcc8958w94ElyFGAAAAEDJW3Hn//Pssw98/r9N+vrnP/95vfTSS3rhhRckSU8//bR+/etf66WXXkpfWffBBx9UY2OjRkdHdfHFF+v6669XU1NTzufs3btXDz/8sB544AHdcMMN+ta3vqWPfOQjJ3xfc3OzfvOb3+jee+/VF7/4RX31q1/VX/3VX+mKK67Qn/3Zn+mJJ57IieSUnp4e/fVf/7WeeuopBYNB/d3f/Z2+9KUv6bOf/awk53Y2zz77rCQnylP7nZ2d2rp1q3bt2qWGhgZdc801+u53v6v3vOc9ikQiWrdune6+++6Z/4OdAWZgAQAAACBPNm/enHNbmHvuuUcbNmzQ1q1bdfjwYe3du/eE96xcuVIXXHCBJOmiiy7SgQMHJv3s973vfScc8+yzz2r79u2SpGuvvVYNDQ0nvO+Xv/yl9uzZo0suuUQXXHCBvvGNb+jgwYPpn3/wgx/MOT61v2PHDl1++eVqaWlReXm5PvzhD+uZZ56RJPl8Pl1//fXT+UeSV8zAAgAAAECeBIPB9PbTTz+tp556Ss8995yqq6t1+eWXT3rbGL/fn972+XzpJcQnO87n8ymRSEhy7qd6KtZaXX311Xr44YdPOebs/ak+u6qqSj6f75TfnW8ELAAAAICSd7Jlvl4KhUIKh8Mn/fng4KAaGhpUXV2tV199Vb/85S/zPoZLL71Ujz76qO644w49+eST6u/vP+GYrVu36o/+6I+0b98+nX322RoZGVFHR4fOOeecKT97y5Yt+vSnP62enh41NDTo4Ycf1qc+9am8/w6ngyXEAAAAADADTU1NuuSSS7Ru3TrdfvvtJ/z82muvVSKR0Pr16/WXf/mX2rp1a97H8LnPfU5PPvmkNm7cqB/+8IdavHixQqFQzjEtLS36+te/rhtvvFHr16/X1q1b0xeBmsrixYv1t3/7t9q2bZs2bNigjRs36t3vfnfef4fTYaYz5TyXbNq0ye7cubPYwwAAAABQZK+88orWrFlT7GEU1djYmHw+n8rLy/Xcc8/p1ltvTV9UqhRM9u/QGLPLWrtpsuNZQgwAAAAAJerQoUO64YYblEwmVVlZqQceeKDYQ/IUAQsAAAAAJWrVqlV6/vnniz2MguEcWAAAAABASSBgAQAAAAAlgYAFAAAAAJQEAhYAAAAAUBIIWAAAAAAokJqaGklSZ2en3v/+9096zOWXX65T3Tr0y1/+skZGRtL773znOzUwMJC/gc5RBCwAAAAAFNiSJUv02GOPzfj9EwP28ccfV319fT6GdkqJRGLK/em+byYIWAAAAACYgTvuuEP33ntvev+uu+7SP/zDP2h4eFhXXnmlNm7cqPPPP1/f+973TnjvgQMHtG7dOknS6Oiotm/frvXr1+uDH/ygRkdH08fdeuut2rRpk9auXavPfe5zkqR77rlHnZ2d2rZtm7Zt2yZJWrFihXp6eiRJX/rSl7Ru3TqtW7dOX/7yl9Pft2bNGt18881au3atrrnmmpzvSenu7tb111+viy++WBdffLF+/vOfp3+3W265Rddcc40+9rGPnbB/8OBBXXnllVq/fr2uvPJKHTp0SJJ000036U/+5E+0bds23XHHHbP+Z859YAEAAACUvrvqPPzswUlf3r59uz7zmc/oD//wDyVJjz76qJ544glVVVXpO9/5jmpra9XT06OtW7fquuuukzFm0s/5l3/5F1VXV2v37t3avXu3Nm7cmP7Z3/zN36ixsVHj4+O68sortXv3bt1222360pe+pJ/85Cdqbm7O+axdu3bpa1/7mn71q1/JWqstW7bobW97mxoaGrR37149/PDDeuCBB3TDDTfoW9/6lj7ykY/kvP/Tn/60/viP/1iXXnqpDh06pLe//e165ZVX0p/97LPPKhAI6K677srZf9e73qWPfexj+t3f/V09+OCDuu222/Td735XkvT666/rqaeeks/nm9k//ywELAAAAADMwIUXXqiuri51dnaqu7tbDQ0NWr58ueLxuP78z/9czzzzjMrKynTkyBEdP35cixYtmvRznnnmGd12222SpPXr12v9+vXpnz366KO6//77lUgkdPToUe3Zsyfn5xM9++yzeu9736tgMChJet/73qef/exnuu6667Ry5UpdcMEFkqSLLrpIBw4cOOH9Tz31lPbs2ZPeHxoaUjgcliRdd911CgQC6Z9l7z/33HP69re/LUn66Ec/qj/90z9NH/eBD3wgL/EqEbAAAAAAMGPvf//79dhjj+nYsWPavn27JOmhhx5Sd3e3du3apYqKCq1YsULRaHTKz5lsdvbNN9/UF7/4Re3YsUMNDQ266aabTvk51tqT/szv96e3fT7fpEuIk8mknnvuuZxQTUlF8cn2s2X/PlMdd7oIWAAAAACl7yTLfL22fft23Xzzzerp6dFPf/pTSdLg4KBaW1tVUVGhn/zkJzp48OCUn3HZZZfpoYce0rZt2/TSSy9p9+7dkpzZz2AwqLq6Oh0/flw//OEPdfnll0uSQqGQwuHwCUuIL7vsMt1000268847Za3Vd77zHX3zm9+c9u9zzTXX6J/+6Z90++23S5JeeOGF9KztVN761rfqkUce0Uc/+lE99NBDuvTSS6f9naeDizgBAAAAwAytXbtW4XBY7e3tWrx4sSTpwx/+sHbu3KlNmzbpoYce0urVq6f8jFtvvVXDw8Nav369vvCFL2jz5s2SpA0bNujCCy/U2rVr9fu///u65JJL0u+55ZZb9I53vCN9EaeUjRs36qabbtLmzZu1ZcsWffzjH9eFF1447d/nnnvu0c6dO7V+/Xqdd955uu+++6b9vq997Wtav369vvnNb+orX/nKtL/zdJipppjnok2bNtlT3RMJAAAAwPz3yiuvaM2aNcUeBmZhsn+Hxphd1tpNkx3PDCwAAAAAoCQQsAAAAACAkkDAAgAAAABKAgELAAAAoGSV2jV9kDGTf3cELAAAAICSVFVVpd7eXiK2BFlr1dvbq6qqqtN6H/eBBQAAAFCSli5dqo6ODnV3dxd7KJiBqqoqLV269LTeQ8ACAAAAKEkVFRVauXJlsYeBAmIJMQAAAACgJBCwAAAAAICSQMACAAAAAEoCAQsAAAAAKAkELAAAAACgJBCwAAAAAICSQMACAAAAAEoCAQsAAAAAKAkELAAAAACgJBCwAAAAAICSQMACAAAAAEoCAQsAAAAAKAkELAAAAACgJBCwAAAAAICSQMACAAAAAEoCAQsAAAAAKAkELAAAAACgJBCwAAAAAICSQMACAAAAAEoCAQsAAAAAKAkELAAAAACgJBCwAAAAAICSQMACAAAAAEoCAQsAAAAAKAkELAAAAACgJBCwAAAAAICSQMACAAAAAEoCAQsAAAAAKAkELAAAAACgJHgasMaYa40xrxlj9hlj7pzk57cbY15wHy8ZY8aNMY1ejgkAAAAAUJo8C1hjjE/SP0t6h6TzJN1ojDkv+xhr7d9bay+w1l4g6c8k/dRa2+fVmAAAAAAApcvLGdjNkvZZa/dba2OSHpH07imOv1HSwx6OBwAAAABQwrwM2HZJh7P2O9zXTmCMqZZ0raRvneTntxhjdhpjdnZ3d+d9oAAAAACAuc/LgDWTvGZPcuy7JP38ZMuHrbX3W2s3WWs3tbS05G2AAAAAAIDS4WXAdkhalrW/VFLnSY7dLpYPAwAAAACm4GXA7pC0yhiz0hhTKSdSvz/xIGNMnaS3Sfqeh2MBAAAAAJS4cq8+2FqbMMZ8UtKPJPkkPWitfdkY8wn35/e5h75X0pPW2ohXYwEAAAAAlD5j7clOS52bNm3aZHfu3FnsYQAAAAAAPGCM2WWt3TTZzzybgV2Q3vgvaeCQNNovbbhRCi0q9ogAAAAAYN4gYPPpp1+QDj3nbC/dTMACAAAAQB55eRGnhaeqPrM92l+8cQAAAADAPETA5lOgIbNNwAIAAABAXhGw+UTAAgAAAIBnCNh8CmQtIY4OFG8cAAAAADAPEbD5xAwsAAAAAHiGgM0nAhYAAAAAPEPA5lOAqxADAAAAgFcI2Hyqyp6B5RxYAAAAAMgnAjafmIEFAAAAAM8QsPkUYAYWAAAAALxCwOZTVZ0k42yPDUrJ8aIOBwAAAADmEwI2n8p8UlVtZj86WLyxAAAAAMA8Q8DmG7fSAQAAAABPELD5RsACAAAAgCcI2Hyryr4SMRdyAgAAAIB8IWDzjRlYAAAAAPAEAZtvBCwAAAAAeIKAzbfsgI2yhBgAAAAA8oWAzbdA9jmwzMACAAAAQL4QsPnGEmIAAAAA8AQBm28ELAAAAAB4goDNN26jAwAAAACeIGDzjRlYAAAAAPAEAZtvBCwAAAAAeIKAzbeJVyG2tnhjAQAAAIB5hIDNt4qAVF7lbCfjUnykuOMBAAAAgHmCgPUCy4gBAAAAIO8IWC8QsAAAAACQdwSsF3ICllvpAAAAAEA+ELBeqJpwIScAAAAAwKwRsF5gCTEAAAAA5B0B64WJt9IBAAAAAMwaAeuF7ICNcg4sAAAAAOQDAesFlhADAAAAQN4RsF4gYAEAAAAg7whYL3AbHQAAAADIOwLWC9xGBwAAAADyjoD1AjOwAAAAAJB3BKwXOAcWAAAAAPKOgPWCv1aScbZjYWk8XtThAAAAAMB8QMB6oaxswr1gB4s3FgAAAACYJwhYr7CMGAAAAADyioD1ChdyAgAAAIC8ImC9wq10AAAAACCvCFivsIQYAAAAAPKKgPUKAQsAAAAAeUXAeiU7YKOcAwsAAAAAs0XAeiXAObAAAAAAkE8ErFdYQgwAAAAAeUXAeoWABQAAAIC8ImC9knMbHc6BBQAAAIDZImC9wgwsAAAAAOQVAesVAhYAAAAA8oqA9Ur2VYijA5K1xRsLAAAAAMwDBKxXyv1SRbWznUxIseHijgcAAAAAShwB6yWWEQMAAABA3hCwXiJgAQAAACBvCFgv5QQst9IBAAAAgNkgYL1UVZfZZgYWAAAAAGaFgPUSS4gBAAAAIG8IWC9lB2yUJcQAAAAAMBueBqwx5lpjzGvGmH3GmDtPcszlxpgXjDEvG2N+6uV4Ci77XrDMwAIAAADArJR79cHGGJ+kf5Z0taQOSTuMMd+31u7JOqZe0r2SrrXWHjLGtHo1nqJgCTEAAAAA5I2XM7CbJe2z1u631sYkPSLp3ROO+ZCkb1trD0mStbbLw/EUHgELAAAAAHnjZcC2Szqctd/hvpbtHEkNxpinjTG7jDEf83A8hcdtdAAAAAAgbzxbQizJTPKaneT7L5J0paSApOeMMb+01r6e80HG3CLpFklavny5B0P1SFX2ObAELAAAAADMhpczsB2SlmXtL5XUOckxT1hrI9baHknPSNow8YOstfdbazdZaze1tLR4NuC8YwkxAAAAAOSNlwG7Q9IqY8xKY0ylpO2Svj/hmO9J+h1jTLkxplrSFkmveDimwiJgAQAAACBvPFtCbK1NGGM+KelHknySHrTWvmyM+YT78/usta8YY56QtFtSUtJXrbUveTWmgvOHJOOT7LgUj0iJmFReWexRAQAAAEBJ8vIcWFlrH5f0+ITX7puw//eS/t7LcRSNMc69YEd6nf3ogFQzv+4UBAAAAACF4uUSYkgsIwYAAACAPCFgvcatdAAAAAAgLwhYr+XcSocZWAAAAACYKQLWaywhBgAAAIC8IGC9RsACAAAAQF4QsF7LDtgo58ACAAAAwEwRsF4LcA4sAAAAAOQDAes1lhADAAAAQF4QsF7jNjoAAAAAkBcErNe4jQ4AAAAA5AUB6zWWEAMAAABAXhCwXiNgAQAAACAvCFivZV+FODogJZPFGwsAAAAAlDAC1mu+Cqmyxtm2SSkWLu54AAAAAKBEEbCFwDJiAAAAAMVmrRSPSiN9xR7JjJUXewALQqBeGjzsbI8OSA1THw4AAAAAaclxKTYsjYWzHkMT9rNfzzo2Npx7bDIh1Z8hfWZ3sX+rGSFgC4Fb6QAAAAALz3jCOYVwLCxFhyaJz4kRepIojQ3nd1xjpXtaIwFbCCwhBgAAAEpHMinFI1nROeQ8ohPCM70/OCFQ3WPiI8X+TU5UViGV+4s9ihkjYAuBgAUAAAAKYzzhxubgidEZHZoQm6nXJgbpkCRb7N8kV2VI8ockf43kr3W3Q1nbNc5zZY1UVZfZTh/nPko4XiUCtjCyAzY6ULxxAAAAAHPZZPGZE6KT7U94nlOznmZCbIakqsniM7WditMJ76mskcq4/q5EwBZGgHNgAQAAMM9Z65yrmROd7mxndCB3/2Tb8Uixf4uMimBWcNZOCNC6KWK0NvN6RZDwzDMCthBYQgwAAIC5LjnuhmR2fE58TIzTgdzXbLLYv4XSs56p8Jz4nB2lqaW22dGZmgH1kUpzEf9WCiEnYFlCDAAAAA9kB+hMHrE5cGVaU+ZGZF1ueFbVnSRKJ3m9skYypti/CTxCwBYCAQsAAIBTsVaKRbJmNgedvzumttP7g7nHZM+aFltFcEJw1p0YoFV1mUCdeBzxiVMgYAuB+8ACAAAsDMmkc5XbVHimg3Oy7UkiNZko4uBNVkxmBWZOZNad5Ofuvq+iiOPHQkDAFgLnwAIAAJSO9FLcgQkhOuD8Xe6E17JnSYt8+xX/hMgM1GcCNLU92SN13icXHMIcR8AWArfRAQAAKKzUFXFTwTnaP0mATvxZKkSLGKHlgdzwTD/qJ7xWnxupqRnSMl9xxg0UCAFbCJVBqazcWRISH5HiUamiqtijAgAAmPvG4xOCM/uR9VrOz93ji7Uc11/rBmdWaAbq3e36zHY6SLO2y/3FGTNQIgjYQjDGmYWNdDv70QGpYlFxxwQAAFBIibFJAnSSx0hfbpwW68q46SW3E4NzqucGbr8CeIz/dhVKdsCO9kshAhYAAJSg8XhWaPZNEp59WftZs6LxSOHHWlGdCctAQ25wBtzXqyZuE6HAXMZ/MwuFW+kAAIC5JHW13JG+EwM0O05ztoswI2rKciMz+5F+vfHECA3UsxwXmIcI2ELhVjoAAMAriVgmPEd6cyN0ZEKIZgepTRZujGXlJwZo+tE4eaCmZkO5Mi4AFwFbKNxKBwAATEc8mhWhve4jO0h7J4RqgWdFTVlueFY3ZkWoO/OZfi3rdX/IuS4IAMwCAVso3EoHAICFJzGWic3sx2j/ia+NuK8V8lzRypBUnR2ibmxOtl2dmhGtY0YUQNEQsIXCDCwAAKUtmZwQnj25M6SRnglB2le4mVHjmxCbTSeG6cTnQINUXlmY8QFAnhCwhRLgHFgAAOaUeNSJ0EiPG6N9Wdu97nZfJlYLdc5oWbkToOkQdaNz4mupGK1uZFYUwIJBwBYKM7AAAHjHWik27ERnKkLTM6I9UqQ3N1YjBVqqa3yZ+EzHaNOEGG3KDVV/LeeKAsBJELCFwm10AACYPmulsXAmNiPdmQDNDtRId2a2dHzM+3H566RgdnQ250ZpsDk3SJkZBYC8ImALhdvoAAAWuvioE5yR7kyIZu+P9OT+zOsgLavIis5GJ0ZzAjQ7SN1jfBXejgkAMCUCtlBYQgwAmG+S4+55o91SpGtCkHafGKixYW/HU14lBVuywjMrSCfbZ6kuAJQcArZQuI0OAKAUxEZyI3S4KytG3e3h7szSXVnvxpIK0nR8tjjLd4MtWTHa7C7pbZYqgwQpAMxzBGyhVNVltkcHnEvxc04MAMBr1kpjQ250dmWCNB2mE7a9nCUtq8gEaTAVpBMDNStMCVIAwAQEbKH4yp2lSmNDkqw0Npg7KwsAwHRZK0UH3fg8nhulw8dzo3S4y9tzSQMNUrA1Kz5bJsRpVqRW1RGkAIBZIWALKVDvBqyc82AJWABASuo2MMNdmRAd7nJnTY87M6jZcepVlKZmSWtaMmFak4rQ1kyc1rQ655JyUSMAQAERsIUUaJAGDjnb3EoHABaGRCwToeHjWRF6PBOpqef4iDdjqKjORGewNROnNakgzdquqmeWFAAwZxGwhcStdABgfrDW+XN8uEsaPuY8h49lRWlWmHr1531F0AnRmrYJcdp6Yqj6a7wZAwAABUbAFhK30gGAuW084cyWZsdo+LgTqeHjuYE6Hsv/95dXOcEZasuN0XSQthGlAIAFjYAtJAIWAIojHs2K0CmeIz3K+21hTJk7Q9rmPlon2XYjlfuSAgAwJQK2kIItme39T0ubby7aUABgXoiPOrOl4WNS+Kg7Y3rUCdL0/jFv7r9dGXJmStMBuigTpOnX25wLHZX58v/9AAAsQARsIa15l/TMF5ztV38gHd4hLbu4uGMCgLkoMZYbptnPw1mvRwfz/MXGuZBRzaLcCA0tytp2nyuDef5uAABwKgRsIS1eL627XnrpW87+U3dJN/2A5WIAFo7xhHMF3nBnbpQOHc0N1dG+/H6v8WXFpxunocW5cRpa5Jxb6uN/GgEAmKv4X+lC2/YX0p7vScmEdPBZad+PpVVXFXtUADA71jrLdIeOZuI0HaVHpSH3tUiXZJP5+96ycjdIsx7Z+6kwrW6Wysry970AAKAoCNhCazpL2vi70s5/dfafuks66wr+YgVg7krEnGW7qTjNec6K08Ro/r4zPWO6yJkpTT+35e4HGvnzEwCABYSALYa3/an04sPODeuP/1Z6+dvS+e8v9qgALETRITdCjzghekKkdjpLfvOputmJz9rFEwJ1iRuoS5zzULnwEQAAmICALYbQImnrrdLP/sHZ/6//Ja25TiqvLO64AMwf1kojvVlhmvVIhenQUSkWzt93VgTdKF2cFahLnD/zapdklvfyZx0AAJghArZY3nqbtONfnXPG+g9Iv/kGt9UBMD3JcWm4y43QI7nP6dnUo9L4WJ6+0DjLeVNBmorU2iW5sco9TAEAgMcI2GIJ1Eu/8/9J//mXzv5PvyBtuFHy1xR3XACKKznuXgApFaVZgTqYFal2PD/fV17lBmh7Vpi258ZqTZvkq8jP9wEAAMwCAVtMm2+WfnWf8xfTSJf0q3+RLru92KMC4JX0zOkRabAjK0yztsPH8henVXVOjKZmS9OPrNcCDcyaAgCAkkHAFlNFQLr8Tun7n3L2f36PtOl/SNWNxR0XgNNnrRTpkYY63JnSVJhmz5x2OrfQyofqpkyMZodpdqCyogMAAMwz0wpYY8wHrLX/fqrXMAMbPiT94h+lntelsSHnwk5v/5tijwrARNFBJ0gHj2QiNR2o7gxqvs45rW52QrRu6YQ4zYrVikB+vgsAAKCETHcG9s8kTYzVyV7D6fKVS1f8pfToR539Xz8gbfmEVL+suOMCFpLEWCZE02Hakbufr6v1Bhqlunap1o3TiduhJVJFVX6+CwAAYJ6ZMmCNMe+Q9E5J7caYe7J+VCspT+vgoDXvktovko7scmZwfvp56d3/XOxRAfNDMumcYz54RBo8nDVrejgTqJGu/HxX6pzT2vZMmNal9pcycwoAADBLp5qB7ZS0U9J1knZlvR6W9MdeDWrBMUa66i7pG+9y9l/4N+ktn5JaVxdzVEBpiEXcEE0FafbjsBOoyfjsv6c8kBujdUszoVq3zIlTf2j23wMAAICTmjJgrbUvSnrRGPNv1tq4JBljGiQts7/hfeYAACAASURBVNb2F2KAC8bKy6SzrpTe+LFkk9L/uV760CPSovOLPTKgeJJJafjYSQLV3R/Nwx9FpsxZulvXnhWmy7L2lzoXV+NqvQAAAEU13XNg/9MYc517/AuSuo0xP7XW/slUbzLGXCvpK5J8kr5qrf38hJ9fLul7kt50X/q2tfbu0xj//HLVXdKbzzizRUMd0oPXSu9/UDrn7cUeGeCNyWZPBw5nXhvqzM/saVW9c1557dLM7Gn2o2aRcz46AAAA5rTp/o2tzlo7ZIz5uKSvWWs/Z4zZPdUbjDE+Sf8s6WpJHZJ2GGO+b63dM+HQn1lr//tpj3w+WrxeuvER6d9vci4YExuWHt4uvf1/Oxd2YvYHpSSZlEZ63CA9nBuqA4fc2dO+2X+Pr9K9ANIy9zEhTmvbuZ0MAADAPDHdgC03xiyWdIOkv5jmezZL2met3S9JxphHJL1b0sSARbZVV0n/40np3z4oDR5ylhM/cafUs1d6xxeYJcLcEY86F0NKxWj6kdo/kp/bylQ3ORFavzwrTLNiNdgilZXN/nsAAAAw5023hu6W9CNJP7fW7jDGnClp7yne0y7pcNZ+h6Qtkxz3FmPMi3IuGPU/rbUvTzzAGHOLpFskafny5dMccglrO0+6+cfSIx+SOnY4r+38V6n/TekDX3eudAp4yVpppM+dMT2ctbQ3ayY10j377ymrcGZP03G6bEKkLpUqq2f/PQAAAJgXjLXWmw825gOS3m6t/bi7/1FJm621n8o6plZS0lo7bIx5p6SvWGtXTfW5mzZtsjt37vRkzHNOfFT67h9KL38781rLaulD/1dqWFG0YWEeSMSy7ns6cXmv+5wYnf33VNU7IVq/bEKgus81bcyeAgAAIIcxZpe1dtNkP5vWDKwxZqmkf5R0iSQr6VlJn7bWdkzxtg5Jy7L2l8qZZU2z1g5lbT9ujLnXGNNsre2ZzrjmvYqAdP2/Sk1nS898wXmt+1XpgSuld31ZOve/8Zd/nMhaaaQ3cwuZya7gO3xczn+VZ8H43HNPs4K0flnW8t52bisDAACAvJruEuKvSfo3SR9w9z/ivnb1FO/ZIWmVMWalpCOStkv6UPYBxphFko5ba60xZrOkMkm90x/+AlBWJl3xF07Efv+T0njMuTDO//2IMxt7yWek898v+SqKPVIUythwZvY0PYua9Rg6IiWis/+eypB75d72CWHqhipX7gUAAECBTfdvny3W2q9l7X/dGPOZqd5grU0YYz4p59xZn6QHrbUvG2M+4f78Pknvl3SrMSYhaVTSduvVmuZSt+GDUsMZznmxI27jd78qffcT0k/+t/TWT0kXfoTzBUtdPCqFO52Z06FO53ZKgxNiNTow++9J3/d0qXuv02UnzqRyrjUAAADmmGmdA2uMeUrS1yU97L50o6Tfs9Ze6d3QJregzoGdzHCX9It/lHY+6NxmJ1t1s7T1E9LFH5cCDcUZH04uNiKFj7ph2ukEaep5sMPZHsnT6nl/3cnjtG6pFFrM7CkAAADmpKnOgZ1uwC6X9E+S3iLnxLlfSLrNWnsonwOdjgUfsCmj/dKOr0q//JfMjGxKZUg6793S2VdIZ26TqhuLM8aFInXOafioNHTUmUENH3OCNB2sR5x/Z/mQuu9pbXagZoVqbbtUVZuf7wIAAAAKLB8B+w1Jn7HW9rv7jZK+aK39/byOdBoI2AliI9Lz/0f6xT3OhXpOYKT2jdJZV0pnXym1b2LmbbqSSWm0z4nR4WPOc/iYcwGk1PPQUedn47H8fKfxObOjtUucKK1tz0RpKlKrm7l4FwAAAOatfATs89baC0/1WiEQsCcxHpd++5j08y8758aejL9OOvMyadlWqeVc51G7dOEE0XjCidJIj7Ncd7jLeUS6pOFuJ0pT25EuKZnI33eXlTvnnda6gVrb7sRqXbvz76B2iXNbGf4PBgAAACxgs76NjqQyY0zDhBlY/pY9l/gqpAtulNZ/UDqyS3rjx9K+H0tHdko2mTlubFB65T+cR0pFtdS8Smo+NxO1DSulYItU3TR3gyo+6izLHe2XRgeytlOPVKj2ZoJ1dECzvn3MZPx1TpiG3Ef2dl27E67BloXzfxQAAAAAHphumfyDpF8YYx6T87f/GyT9jWejwsyVlUnLLnYel9/phNz+p52YfeO/nHMxJ4qPSEdfdB6TCTQ48RVskYLNznOgQSqvch4VVZnt1L6v0nmvtW5AW2c79Wytc6uXxJj7nPWIu8+xYeeWMWNh5xELZ7bHwvlbtjsVf50UWiSF2pzbxqSfU4/FznNl0PuxAAAAAAvctJYQS5Ix5jxJV0gykn5srd3j5cBOhiXEs2Ct1P2a9OZPpa49UvfrznLj0b5ij6ywAg3OeaTVTVJNq/toc8J84nZFoNijBQAAABaUfCwhlhusRYlW5IkxUutq55Et0uOEbc9rznP3a87VcyPd0kifPFlymw++SidGU4+q+tz9QL0zY1zd5ARrsFkKNM7dJdEAAAAApsTf5OEuC26WVlxy4s+yL3oU6XYfPVJ0ILMEOD7qLgUezeyPxyTjnu9pjCTjPJuyzHZ5lVTul8oD7nP2cmS/VFkj+UO5j8oayV/rbJf73c8GAAAAsBAQsJiarzyzzBYAAAAAiohLogIAAAAASgIBCwAAAAAoCQQsAAAAAKAkELAAAAAAgJJAwAIAAAAASgIBCwAAAAAoCQQsAAAAAKAkELAAAAAAgJJAwAIAAAAASgIBCwAAAAAoCQQsAAAAAKAkELAAAAAAgJJAwAIAAAAASgIBCwAAAAAoCQQsAAAAAKAkELAAAAAAgJJAwAIAAADAApJM2mIPYcbKiz0AAAAAAMDpGU9aDY3GNTAa1+DEx0jsxNdGExpyt1tDfv3X/7y82L/CjBCwAAAAAFAE1lqFxxIaHMmNzQF3f2A05kTqSO7PhkbjCo8lZvy9leWluxCXgAUAAACAWYglkm5gxjLxOeLOjo7E0rOkqdecKI1pKJrQeBGW8w5HE7LWyhhT8O+eLQIWAAAAACRF4+NuZDoh6sRoLB2eOfvpUI0pEhsv+FiNkUL+ctVVV6g+UKm6QIXqAhWqdZ/rAhWqr85sZ/885C8vyXiVCFgAAAAA88xobDwdof0jMQ2OxNXvhungSPyESE1tjyWSBR9rdaVP9VnhWZ8K0gnxmR2j9YFK1VSVy1dWmhE6GwQsAAAAgDkpPp7MCdDMdkz97mxof8R9fdR5LkaIlhmpvroyJzbrAxWqr65UbSC1PTFEneNL+XzUYiBgAQAAAHjKWquhaEIDbnhODNGBnOfMzOjwLC5UNBMVPqP66sqs4KzMitEK1WX9rN79WV11hWoqy1W2AGdDi4GABQAAADBtifFkerazfySu/kgssz0S00Akrr6R2IQojRf0YkWVvjInMrNCs766Qg3VlelzRlNhWue+Xl9doUCFr2TPDV0oCFgAAABggYqPJ9PnifZFnOjsc5fk9kdi6nOfU3HaH3GunFsoqaW5qdh0QrPSjdGKrG1nOW5DsFINhOi8RsACAAAA88B40qaX4PZF4uqLxNJhmorRgZHc18MFjNFgpU/11ZVqCGZmQBuqK9Mh2hB0n91lug3VlQpVsTQXuQhYAAAAYI6x1io8llDfcGYWNBOecfVFxk6YKR0cjcsWYJWuMXJmO934TM2KNmYFaHaUNrrLdv3lPu8Hh3mPgAUAAAA8Fh9PpkMzFaV9kZh6h50o7Y1kIjUVqvFx72vUGKVnOxuDmRB1luJWqrG6Mr0sN/VaXaBiQd6+BXMDAQsAAACcptHYuHojY06ERpwoTYVo37D77P68r4DnjdZXV2RFpxOeqTBtCubGaKN7ixdiFKWEgAUAAMCCZq3VSGw8HaO9w2NugOZup2ZM+yIxjcbHPR9XdaUvPTM68dHgzpQ2Bv3ppbv1gQqV+7inKOY3AhYAAADzTjQ+nonR4UyY9kVi6hmOZWZP3e1oPOnpeMqM0jHaEMzMhja5MdpUkx2mzqOqgnNGgYkIWAAAAMx5ifGk+kfi6o04QdqTDtPUfma7LxLT8Ji3S3YrfEaNwUo1Bf058dkUrFRjjfsc9KdjlPNGgfwgYAEAAFBw1lpFYuPqHR5Tz/CYE6DpMB1TTySmnvBYeuZ0wOMr7FaWl6nZjc/GoN8NUGdmNDtGU4Ea8pdzn1GgCAhYAAAA5EUyaTUwGneCNDym7uHc2dKenDD1dtluaoa0MehXsxuhTTX+dJA2Bf3pmdKmGr+ClT6CFCgBBCwAAABOajxp3fNGx9QdTs2WOjOmqUjtceO0LxLTeNKbaVLjnkPalJoVrfE7M6buEt5m97VUkNZWMUMKzEcELAAAwAJzsih1tnNf74vE5FGTyl9epuYaZ4a0OTU7WuNPv9aUjlO/Gqq5wi4AAhYAAGBesNZqaDSh7uGousJOgHanZkjDMXVnRWnv8JhnURqqKldLKkJDbpgGne2moF8t7nNziGW7AE4fAQsAADCHRePj6RDtGhpLh2huoDrbsXFvziltqK5wZ0Wd8EzNmLZkRWpqBtVfzq1fAHiHgAUAACgwa60GRuLpmdKucDRre0zdWfvhqDe3g6l3o7TFjdJUjObsu1FawdJdAHMEAQsAAJAnifGkeiMxdQ1lojRnO+ztbGmNv1wtodyZ0daQXy0hJ0Zb3O2moF+V5UQpgNJDwAIAAJxCfDyp7vCYjg+lojQ3To8POXHaF8n/uaXlZSYdni1uhGZHaWutXy01VWoOVaq6kr/aAZjf+FMOAAAsWPHxpHqGx3R8yI3ToWhm2w3W7vCYeiOxvH93yF+ullonSltrq9xnf+Y55FdrqEr1gQqVlXGhIwCQCFgAADAPJZNWvZGYjg9F3UcqSqM6NpiaMY2qNxKTzfOMaVOw0onP2iq1urOlre5+S3q/SoFKLnYEAKeLgAUAACUlHI2no/TYYFTHw1EdH4zqWFaodofHlMjjWl5jlD6ftK22Sm21frWEqnLitNVd0su5pQDgHQIWAADMCeNJq+7wmI4NpWZJ3Sh14zS1HYmN5+07jZGagn611WbitNUN1NaQ89xWW6WmYKXKuRIvABQdAQsAADw3GhvXsaGojg6O6vhQVEcHs8N0TMcGR9Udzu8FkOqrK9QWqlJrrV+LaqvSM6etWdvNNX5uEQMAJYSABQAAM2atVXgsoWODTpQeGxx1n6PpmdSjg1ENjsbz9p3+8jK11VY5UVpXpUW1qWW9VenXW2v9qqrgHFMAmG8IWAAAMClrrYaiCR11o/ToQFagDkXVOTCqY3le0tsUrNSiuuw4zSzjTb1eF6iQMVyVFwAWIgIWAIAFangsoaMDo+ocjOY8exGn5WUmE6FuiC6uq8oJ09Zav/zlzJoCAE6OgAUAYB6Kxsd1bDCqzsFRdQ5kBergqDoHnFnUcDSRl+/yl5dpsRumS+oCaqtz4tSJ1IDa6vxqDvq5lykAYNYIWAAASkwyadUzPKYjA06cdg6MuqGa2e+NxPLyXVUVZVpSF9Di+iotqg1kQjVrv76aJb0AgMIgYAEAmGNGYgl1DkTdQHUeRwZGdaTfCdVjg1HFx2d/ud7K8jItqXNmSRfXVWlxvbO9pD7zGuebAgDmEgIWAIACstaqLxJLB+mRrDhNBWv/yOyv2OsrM2oL+bWkPqDF9QE3VKu0pD7gvFZXpcZgJXEKACgpngasMeZaSV+R5JP0VWvt509y3MWSfinpg9bax7wcEwAAXhpPWnWFozrSP6oON0ozzyPqHBhVNJ6c9fc0VFe4IRpQe30mTFOzp60hv8q5vykAYJ7xLGCNMT5J/yzpakkdknYYY75vrd0zyXF/J+lHXo0FAIB8SYwndXQwmhOl2bHaOTCqRHJ2y3srfCa9lHdJfUDt7iMVqEvqA6quZBEVAGDh8fJ//TZL2met3S9JxphHJL1b0p4Jx31K0rckXezhWAAAmJbEeFLHhqI63OfEaYcbp6ntY0NRjc8yUEP+crU3ZKI0Z7s+oJaQXz6u2AsAwAm8DNh2SYez9jskbck+wBjTLum9kq4QAQsAKIBk0qorPKaO/hEd7h/R4b5RHe5z4vRw/4iODs4+UJuClWpvCGhpQ2b2tL2h2n0OqC5QkaffBgCAhcXLgJ3s/zqe+DeCL0u6w1o7PtVFJIwxt0i6RZKWL1+etwECAOanwZG4DvWlAtV5PtQ3qo6+EXUMjCqWmN05qC0hv5Y2BLTUjdKlDZlHe321ApW+PP0mAAAgm5cB2yFpWdb+UkmdE47ZJOkRN16bJb3TGJOw1n43+yBr7f2S7pekTZs2zf6+AQCAkjaWGNeR/lE3Up0Z1EO9I+loDUcTs/r87EDNxGm1ljU4y3yrKghUAACKwcuA3SFplTFmpaQjkrZL+lD2AdbalaltY8zXJf1gYrwCABae1K1mDvU5UZqK00N9zozq0aGo7Cz+78z66gota6jWssaAljVUa2mjE6rL3GAlUAEAmJs8C1hrbcIY80k5Vxf2SXrQWvuyMeYT7s/v8+q7AQBzX+pqvgd7R3SwL6JDvSPuthOpw2Mzn0WtqijT8sZqN1LdOE3vBxSq4hxUAABKkafX4LfWPi7p8QmvTRqu1tqbvBwLAKDwovFxHe4b0YHeER3sjaQD9VBvRB39M7/djDHSkrqAljUG0qG6vKlaSxuqtbyxWs01lZrq2goAAKA0cRM5AMCsDI8l0nF6oDeigz3OjOrBXueKvjNVXenT8sZqndHkROnyRmc29YymoJbUV8lfzjJfAAAWGgIWAHBK4Wg8HagHeiLpGdU3e0bUMzw2489tCfl1RqMze3pGY9CJVTdYm4LMogIAgFwELABAkjQSS+hAjxOpb/akQtXZ7hmOzegzy4y0tMGZRT3DjdTlTZlZ1epK/mcIAABMH39zAIAFJJZI6nD/iN7sdsJ0f09Eb/YM60DPiI4NzWy5b4XPOEt7G6u1ojmoFU1BN1iDaq8PqLK8LM+/BQAAWKgIWACYZ6y1OjYU1ZvdEb3RE9Gb3RHt7xnWmz3OhZPGZ3DhpAqf0fLGaq1sDuqMpqBWNGVidXFdlcp9RCoAAPAeAQsAJWp4LOFEavew9ncPa39PRPvdmdXR+Phpf155mTOTmorTlW6grmwOakl9QL4yzkcFAADFRcACwByWTFodGRh1I9WZSX2jy3k+PjSziye11we0otmZTV3ZXKMz3VhtbwiogplUAAAwhxGwADAHROPj2u/OpjqPiPZ1DevNnmFF48nT/ry6QIXObAnqzOYa9zmYXvIbqOT2MwAAoDQRsABQQP2RmN7oHta+Lvfhbh8ZGJU9zVNTK3xGZzQ5cXpmixOqZ7U4s6qNwUpvfgEAAIAiImABIM+steoKj2nv8WHt6wprb9ew9nYN642uYfVGTv92NE3BSjdOa3SWG6pnttRoWUOAiycBAIAFhYAFgBmy1qpzMKrXj4e177gzk7rXDdZwNHFan1VmpDOanBnUVKie1eosAW5gNhUAAEASAQsAp2St1VE3VPceH9brx8N6vWtY+46HFYmd3tV+qyrKdGZzjc5uzX2c0VQtfznnpgIAAEyFgAUAl7VW3cNjev3YsF47Htbe42H3eVjDY6c3oxryl+vsthqtaq3RqtZQOlTb6wMq43Y0AAAAM0LAAliQBkfjev14WK8eC+v1Y06ovn48rIGR+Gl9Tl2gQue01ejs1pDOaXNidVVbjVpDfhlDqAIAAOQTAQtgXhtLjGtf17BecyP1tWPO4+hg9LQ+JxWqq9pCOqfVeV7VVqOWGkIVAACgUAhYAPNC6oJKrx4d0qvHnJnVV48OaX9PROPJ6d+fJljp0zmLQjqnNaRzFoV0bpszs9rCjCoAAEDREbAASs5ILKHXjoX1ytGwXj02pFePhvXKsaHTuvJvpa9MZ7XWaPWikM5pC+ncRTU6py2k9voAoQoAADBHEbAA5qzU1X9fOTrkPsJ65eiQ3uyNyE5/UlXLG6t17qKQVi8KpZ/PaAqqgnuoAgAAlBQCFsCcEB9Pan93RHuODmpP55D2HB3Sns4h9Z/GRZVqq8q1enGt1iwKafXiWp3rLgEO+vmjDgAAYD7gb3UACm4kltArR8Pa0zmolzuH9HLnkF47HlYskZzW+8uMtLI5qDWLa91HSGsW12pRbRXLfwEAAOYxAhaApwZH4nq5c1AvZcXq/u5hTfe6SiF/udYsrtV5S2q1epETque0hRSo9Hk7cAAAAMw5BCyAvOkdHtNLnUN66cigXjoyqN8eGVRH/+i0399eH0jH6nmLa7V2Sa2WNnBRJQAAADgIWAAz0js8pt1HBvXbjsF0sHZO896qZUY6q6VGa5fUal17nc5zo7W+utLjUQMAAKCUEbAATmlwJK7fHhnU7iMD2n3YmVk9MjC9mdVKX5nOXRTSuvZanbekTmuX1GrNolqWAAMAAOC0EbAAcozEEnq5c0gvHh7Qix2D2t0xoIO9I9N6r7+8TGsW12pde63Ob6/TuvY6ndMW4nY1AAAAyAsCFljA4uNJvX48rBcPO6H6wuEBvX48PK0LLPnLy3TeEidUU7F6dmsNsQoAAADPELDAAmGtVedgVC8cGtALh/v1wuEB/fbIoKLxU9+6psJntHpRrdYvrdP6pXU6v71eq9qIVQAAABQWAQvMU5GxhF50Z1VfODSg5w8PqDs8dsr3GSOd3VKjDcvqtWFpndYvrdfqxSH5yzlnFQAAAMVFwALzgLVWB3tH9JtD/c7j4IBePTY0raXA7fUBbVhWpw1L67V+ab3OX1qnGj9/NAAAAGDu4W+pQAkajY3rxY4B7TrYr+cP9ev5QwPqjcRO+b4af7nWL63TBcvqncfyerWGqgowYgAAAGD2CFigBHSFo9p1oF87DzqPl48MKnGK6VVjpHPbQrpwuRuryxp0dmuNfGWmQKMGAAAA8ouABeYYa632dQ3r1wf6tPNAv3Ye7NPhvlPfc7UuUKELl9dr4/IGbVzeoA3L6hSqqijAiAEAAIDCIGCBIouPJ/XSkUHtONCnX7/Zr10H+9Q/Ej/l+85urdFFyxt00RkN2nhGvc5srlEZs6sAAACYxwhYoMCi8XH95lC/frW/TzsO9On5QwMajY9P+R5/eZk2LKvXpjPcYF3eoIZgZYFGDAAAAMwNBCzgsZFYQrsOOsH6qzd79eLhQcXGp773amOwUpvOaNDmlY266IwGrV1Sp8py7rkKAACAhY2ABfJsJJbQzgP9em5/r361v1e7O059waWlDQFtXtGoTSsatXllg85qqZExLAcGAAAAshGwwCxF4+N6/tCAnnujR8/t79ULhwcUH586WM9qCWrLmU3asrJRm1c2anFdoECjBQAAAEoXAQucpsR4UruPDOoX+3r0izd6tetgv8YSUy8JPrctpC1nNmrLyiZtXtmolpC/QKMFAAAA5g8CFjgFa63290T08309enavM8sajiamfM85bTV661nN2npmozavbFIjF1wCAAAAZo2ABSbRMzymZ/f26Nl9Pfr5vh4dHYxOefyZzUFtPatJbz2rSVvPbFJzDTOsAAAAQL4RsICkWCKpXQf79czebj3zerde7hya8vhFtVW6dFWz3npWk956VrMW1VUVaKQAAADAwkXAYsE60BNJB+tzb/QqEjv5vVhD/nJtPatJv7OqWZec3awzm4NcJRgAAAAoMAIWC0Y0Pq5fvdmnn7zapadf69KB3pGTHlteZrRxeYMTrKuatb69TuU+7sMKAAAAFBMBi3ntyMBoOlh/vq9Xo/GTz7IuawzoslUtuuycFr31rCaFqioKOFIAAAAAp0LAYl4ZT1q9cHhAT71yXP/1SpdeOx4+6bGBCp/eelaTLjvHidYVTdUsCwYAAADmMAIWJW8kltCze3ucaH21Sz3DsZMeu7I5qG3ntmrb6hZtXtkof7mvgCMFAAAAMBsELEpS11BUP361S0/tOa5n9/VoLJGc9LhKX5m2nNmoK1a36vJzW7WyOVjgkQIAAADIFwIWJeNw34ieeOmYnnj5mH5zqF/WTn5cc41fV61p1ZVr2nTJ2U2qruQ/5gAAAMB8wN/sMaftPR5OR+tU92Y9ty2kq85r1VVr2rRhab3KyjiXFQAAAJhvCFjMKdZavXI0rP/320498dIxvdEdmfQ4X5nR5hWNuvq8Nl21pk3Lm6oLPFIAAAAAhUbAYk7Yezys/9h9VD/Y3an9J4nWSl+ZLl3VrGvXLtJV57WpMVhZ4FECAAAAKCYCFkVzoCeiH+zu1A92H9Wrxya/3U2gwqdtq1v09rWLdMXqVu7NCgAAACxgBCwKqisc1fdf6NT3XujUb48MTnpMdaVPV61p0zvPX6y3ndOiQCW3ugEAAABAwKIARmIJPfnycX37+SN6dm+3kpNcPdhfXqYrVrfqv69foitWtxKtAAAAAE5AwMIT40mrX+7v1bd+06EfvXRMkdj4CcdU+Izedk6L3rVhia5c06YaP/9xBAAAAHByFAPy6kBPRI/uPKxv/+aIjg1FJz1my8pGvffCdr3j/MWqC3BOKwAAAIDpIWAxa9H4uH708jE98uvDem5/76TH/P/t3XuQVvV9x/H3dxdWLoabiAK7oCKCqKCwmstolNRESdowGjNqUrUZLTUmbTLJ5DqZtFM7uZjJxKTxMlStNWlDLtXETjVWk1hNlYqLKPECbrguVoiA3GHZ5dc/drssy1Ie8DnPOc/u+zXjDOcyzGfwO8t8+J3zO6ccP5TLzxnP3LPH0zDKT95IkiRJOnIWWB21V17fyoJn1vLAc+vYsmvvQddHDa3jgzPGcdk545leP5yIyCGlJEmSpL7CAqsjsqu1nQefX8e/PLOW59e+edD1moDZU8Zw5bkNzJ46hoG1NTmklCRJktQXWWBVkrWbdvLDhatZsGhtr6utDaMGc2VjA1fMauDE4YNySChJkiSpr7PA6pBSSjz9+43c+9QqHnt5/UGfv6mrreF9Z5zA9LE8FgAADsVJREFUVedO4F2TjqOmxkeEJUmSJGXHAquD7Gxt4/7F67jv6VUsX7/9oOsNowZz7TtO4kOz6hk1tK7yASVJkiT1SxZYddm4fQ//+F+ruO/pVWzd3XbQ9Qsmj+a6d57E7KljqHW1VZIkSVKFWWDFujd38Q9PrGDBojXs3rvvgGtD6mr50Mx6rnvXRE4d87acEkqSJElSxgU2Ii4FvgvUAnellL7R4/pc4GZgH9AGfDql9NssM2m/5g3buOPxFfxiyTraerzgOvG4IVz3zpO4orGeYYMG5pRQkiRJkvbLrMBGRC1wG/BeoAVYFBEPppRe6nbbr4AHU0opIqYDPwGmZpVJHZ5f+ya3P97Mf7y0ntRjY6ZpY4dx0+xJzDlzrI8JS5IkSSqULFdgzwOaU0orACJiATAX6CqwKaXuOwQNBXrUKZXTste38a1HXuGxlzccdO28k0dx00WTuPC044mwuEqSJEkqniwL7HhgbbfjFuDtPW+KiMuArwNjgA/09htFxDxgHsCECRPKHrSva9m8k+88+ir3P9dy0IrrxaeP4eMXTWLWxFH5hJMkSZKkEmVZYHtbxjtohTWl9ADwQES8m473YS/u5Z75wHyAxsZGV2lLtGlHK7f9ppkfPL2a1vb9mzNFwB9PH8cnZk9i6onDckwoSZIkSaXLssC2AA3djuuB1w51c0rpiYiYFBGjU0pvZJirz9vZ2sbdT65k/hMr2LbnwM/hXDTleD5/yVSmjbO4SpIkSaouWRbYRcDkiDgZWAdcBXyk+w0RcSrw+85NnGYCdcDGDDP1aSklftrUwrceWcYftu054NrZDSP44pypvOOU43JKJ0mSJElvTWYFNqXUFhGfBB6h4zM696SUXoyIGzuv3wl8CLg2IvYCu4ArU+r5lqZKseIP2/nyA0tZuGLTAecnHT+Uz10ylUvOOMHNmSRJkiRVtai2vtjY2JieffbZvGMURmvbPuY/8Xu+9+tmWtv2v+d64rBBfPriyVwxq54BtTU5JpQkSZKk0kVEU0qpsbdrWT5CrIw1rd7El+5fyvL1+79GVFsT3HD+yXzq4skMqfN/ryRJkqS+w4ZThbbu3sstv3yFf/7vNQd8Fues8cP5+uVnceb44fmFkyRJkqSMWGCrzGMvrefLDyxlQ7dNmobU1fLZ903hundO9HFhSZIkSX2WBbZKtO9LfOfR5Xz/N80HnH/P1DH87dwzqB85JKdkkiRJklQZFtgqsGXXXj694Dl+s+wPXedGH3sMf/PBaXzgrLHuLixJkiSpX7DAFtyr67cx7wdNrHxjR9e5CyaP5u+vPocRQ+pyTCZJkiRJlWWBLbBHXnydz/x4CTta27vO3XjhJD53yRRqa1x1lSRJktS/WGALaN++xK2/epXv/erVrnODB9ZyyxXT+ZMZ43JMJkmSJEn5scAWzNbde/nMj5fw2Msbus7VjxzM/GsamTZuWI7JJEmSJClfFtgC2bh9D1fOX0jzhu1d584/teN915FDfd9VkiRJUv9mgS2IPW3t3PjDpgPK67x3n8LnL5nit10lSZIkCQtsIaSU+MoDv2PRqs0ARMC3PzyDy2fW55xMkiRJkorDpb0CuOvJlfy0qaXr+EtzplpeJUmSJKkHC2zOfv3Ker728Mtdxx+eVc+fX3BKjokkSZIkqZgssDla9vo2/upHS0ip4/jck0byd5edSYTfeJUkSZKkniywOdm4fQ/X/9Mitu9pA2D8iMHc8aezOGZAbc7JJEmSJKmYLLA5aG3bx8d/uJiWzbsAGFpXy91/1sjoY4/JOZkkSZIkFZcFtsJSSnzl50t5ZtUmoGPH4e9edQ5TTxyWczJJkiRJKjYLbIXd/duV/OTZ/TsOf+HSqVw87YQcE0mSJElSdbDAVlDT6s187aH9Ow5fPnM8f/FudxyWJEmSpFJYYCvoZ00t7OvccXjWxJF8/fKz3HFYkiRJkkpkga2gxas3d/36s+89zR2HJUmSJOkIWGArZMuuvSzfsA2A2prg7Akjck4kSZIkSdXFAlshz63ZTOp8fHja2GEMqRuQbyBJkiRJqjIW2Arp/vjwrIkjc0wiSZIkSdXJAlshTWv2F9iZFlhJkiRJOmIW2Apoa9/HkjVvdh27AitJkiRJR84CWwGvvL6NHa3tAIwdPojxIwbnnEiSJEmSqo8FtgIW+/iwJEmSJL1lFtgKaOq+gdMEC6wkSZIkHQ0LbAU0uQOxJEmSJL1lFtiMrd+6m5bNuwAYNLCGaeOG5ZxIkiRJkqqTBTZj3VdfZ9SPYGCtf+SSJEmSdDRsUxnz8WFJkiRJKg8LbMYssJIkSZJUHhbYDO3e286Lr23pOp7pDsSSJEmSdNQssBlaum4Le9sTAJOOH8rIoXU5J5IkSZKk6mWBzZCPD0uSJElS+VhgM/TsKgusJEmSJJWLBTYjKSUWr7HASpIkSVK5WGAzsmrjTjbtaAVg+OCBnDL62JwTSZIkSVJ1s8BmpOf7rzU1kWMaSZIkSap+FtiMuIGTJEmSJJWXBTYjTas3df3a779KkiRJ0ltngc3All17Wb5+OwC1NcGMhuE5J5IkSZKk6meBzcBz3XYfPmPcMIbUDcgxjSRJkiT1DRbYDCzu9v6rjw9LkiRJUnlYYDPQ5PdfJUmSJKnsLLBl1ta+j+fWvNl1bIGVJEmSpPKwwJbZK69vY2drOwDjhg9i3IjBOSeSJEmSpL7BAltmi7s9PjzT1VdJkiRJKhsLbJk1rfb9V0mSJEnKggW2zCywkiRJkpQNC2wZrd+6m5bNuwAYPLCW08cOyzmRJEmSJPUdFtgy6r76OqNhOANr/eOVJEmSpHKxYZWRjw9LkiRJUnYssGVkgZUkSZKk7FhgyySlxGknHEvDqI7vvs6cYIGVJEmSpHIakHeAviIiuOWKGQBs3L6HEUPqck4kSZIkSX2LK7AZOO7YY/KOIEmSJEl9jgVWkiRJklQVLLCSJEmSpKpggZUkSZIkVYVMC2xEXBoRyyKiOSK+2Mv1j0bEC53/PRURM7LMI0mSJEmqXpkV2IioBW4D5gDTgKsjYlqP21YCF6aUpgM3A/OzyiNJkiRJqm5ZrsCeBzSnlFaklFqBBcDc7jeklJ5KKW3uPFwI1GeYR5IkSZJUxbIssOOBtd2OWzrPHcr1wMMZ5pEkSZIkVbEBGf7e0cu51OuNEbPpKLDnH+L6PGBe5+H2iFhWloTZGQ28kXcIqZPzqCJxHlUkzqOKxHlUkeQ9jxMPdSHLAtsCNHQ7rgde63lTREwH7gLmpJQ29vYbpZTmU0Xvx0bEsymlxrxzSOA8qlicRxWJ86gicR5VJEWexywfIV4ETI6IkyOiDrgKeLD7DRExAbgfuCaltDzDLJIkSZKkKpfZCmxKqS0iPgk8AtQC96SUXoyIGzuv3wl8FTgOuD0iANqK2vQlSZIkSfnK8hFiUkoPAQ/1OHdnt1/fANyQZYacVM3jzuoXnEcVifOoInEeVSTOo4qksPMYKfW6r5IkSZIkSYWS5TuwkiRJkiSVjQX2KEXEpRGxLCKaI+KLvVyPiPhe5/UXImJmHjnVP5Qwjx/tnMMXIuKpiJiRR071D4ebx273nRsR7RFxRSXzqX8pZR4j4qKIWBIRL0bEf1Y6o/qPEv6+Hh4R/xYRz3fO48fyyKn+ISLuiYgNEfG7Q1wvZJ+xwB6FiKgFbgPmANOAqyNiWo/b5gCTO/+bB9xR0ZDqN0qcx5XAhSml6cDNFPi9BlW3Eufx/+77Jh0b/UmZKGUeI2IEcDvwwZTSGcCHKx5U/UKJPx8/AbyUUpoBXAR8u/NrHlIW7gUu/X+uF7LPWGCPznlAc0ppRUqpFVgAzO1xz1zgvtRhITAiIsZWOqj6hcPOY0rpqZTS5s7DhXR8l1nKQik/HwH+EvhXYEMlw6nfKWUePwLcn1JaA5BSciaVlVLmMQFvi47PcxwLbALaKhtT/UVK6Qk6ZuxQCtlnLLBHZzywtttxS+e5I71HKocjnbXrgYczTaT+7LDzGBHjgcuAO5GyVcrPx9OAkRHxeEQ0RcS1FUun/qaUefw+cDrwGrAU+FRKaV9l4kkHKWSfyfQzOn1Y9HKu53bOpdwjlUPJsxYRs+kosOdnmkj9WSnzeCvwhZRSe+c3wKWslDKPA4BZwB8Bg4GnI2JhSml51uHU75Qyj5cAS4D3AJOARyPiyZTS1qzDSb0oZJ+xwB6dFqCh23E9Hf9SdqT3SOVQ0qxFxHTgLmBOSmljhbKp/yllHhuBBZ3ldTTw/ohoSyn9vDIR1Y+U+vf1GymlHcCOiHgCmAFYYFVupczjx4BvpI7vXDZHxEpgKvBMZSJKByhkn/ER4qOzCJgcESd3vlh/FfBgj3seBK7t3L3rHcCWlNL/VDqo+oXDzmNETADuB65xVUEZO+w8ppROTimdlFI6CfgZcJPlVRkp5e/rXwAXRMSAiBgCvB14ucI51T+UMo9r6HgagIg4AZgCrKhoSmm/QvYZV2CPQkqpLSI+ScfumbXAPSmlFyPixs7rdwIPAe8HmoGddPyLmlR2Jc7jV4HjgNs7V73aUkqNeWVW31XiPEoVUco8ppRejohfAi8A+4C7Ukq9flJCeitK/Pl4M3BvRCyl4/HNL6SU3sgttPq0iPgRHbtdj46IFuCvgYFQ7D4THU8oSJIkSZJUbD5CLEmSJEmqChZYSZIkSVJVsMBKkiRJkqqCBVaSJEmSVBUssJIkSZKkqmCBlSRJkiRVBQusJEmSJKkqWGAlSSqoiDgrIlZHxMfzziJJUhFYYCVJKqiU0lLgKuDavLNIklQEFlhJkoptA3BG3iEkSSoCC6wkScX2DeCYiJiYdxBJkvJmgZUkqaAi4lJgKPDvuAorSZIFVpKkIoqIQcAtwE3AUuDMfBNJkpQ/C6wkScX0FeC+lNIqLLCSJAEWWEmSCicipgDvBW7tPGWBlSQJiJRS3hkkSZIkSTosV2AlSZIkSVXBAitJkiRJqgoWWEmSJElSVbDASpIkSZKqggVWkiRJklQVLLCSJEmSpKpggZUkSZIkVQULrCRJkiSpKvwvra0Q0eeHEMoAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plot_cost_lambda()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## 3.6. Krzywa uczenia się" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "* Krzywa uczenia pozwala sprawdzić, czy uczenie przebiega poprawnie.\n", + "* Krzywa uczenia to wykres zależności między wielkością zbioru treningowego a wartością funkcji kosztu.\n", + "* Wraz ze wzrostem wielkości zbioru treningowego wartość funkcji kosztu na zbiorze treningowym rośnie.\n", + "* Wraz ze wzrostem wielkości zbioru treningowego wartość funkcji kosztu na zbiorze walidacyjnym maleje." + ] + }, + { + "cell_type": "code", + "execution_count": 69, + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "outputs": [], + "source": [ + "def cost_trainsetsize_fun(m):\n", + " \"\"\"Koszt w zależności od wielkości zbioru uczącego\"\"\"\n", + " theta = np.matrix(np.zeros(X.shape[1])).reshape(X.shape[1],1)\n", + " thetaBest, err = SGD(h, J, dJ, theta, X, Y, alpha=1, adaGrad=True, maxEpochs=2500, batchSize=100, \n", + " logError=True, validate=0.25, valStep=1, lamb=0.01, trainsetsize=m)\n", + " return err[1][-1], err[3][-1]\n", + "\n", + "def plot_learning_curve():\n", + " \"\"\"Wykres krzywej uczenia się\"\"\"\n", + " plt.figure(figsize=(16,8))\n", + " ax = plt.subplot(111)\n", + " M = np.arange(0.3, 1.0, 0.05)\n", + " Costs = [cost_trainsetsize_fun(m) for m in M]\n", + " CostTrain = [cost[0] for cost in Costs]\n", + " CostCV = [cost[1] for cost in Costs]\n", + " plt.plot(M, CostTrain, lw=3, label='training error')\n", + " plt.plot(M, CostCV, lw=3, label='validation error')\n", + " ax.set_xlabel(u'trainset size')\n", + " ax.set_ylabel(u'cost')\n", + " plt.legend()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Krzywa uczenia a obciążenie i wariancja\n", + "\n", + "Wykreślenie krzywej uczenia pomaga diagnozować nadmierne i niedostateczne dopasowanie:\n", + "\n", + "\n", + "\n", + "Źródło: http://www.ritchieng.com/machinelearning-learning-curve" + ] + }, + { + "cell_type": "code", + "execution_count": 70, + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7AAAAHgCAYAAACcrIEcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3yW9b3/8deVASHMMFSGyBAkgixxtFjFvQCtE6tt1To6bXvOsbY9v6PW03Pa/tp6Wn+2x6pVOyzWrbjFusUKOBCZIsOAIHsGyLh+f3wDdxYhxNy5ciev5+NxP3J97+t73/kkKOSd74riOEaSJEmSpOYuK+kCJEmSJEmqDwOsJEmSJCkjGGAlSZIkSRnBACtJkiRJyggGWEmSJElSRjDASpIkSZIyQk7SBeyr7t27x/369Uu6DEmSJElSGsycOXNNHMc9aruXcQG2X79+zJgxI+kyJEmSJElpEEXR0j3dcwqxJEmSJCkjGGAlSZIkSRnBACtJkiRJyggZtwZWkiRJkgBKSkooKipi+/btSZeiBsjLy6NPnz7k5ubW+zUGWEmSJEkZqaioiI4dO9KvXz+iKEq6HO2DOI5Zu3YtRUVF9O/fv96vcwqxJEmSpIy0fft2unXrZnjNQFEU0a1bt30ePTfASpIkScpYhtfM1ZA/OwOsJEmSJDXAhg0b+P3vf9+g155xxhls2LChzj7XX389U6dObdD7t1QGWEmSJElqgLoCbFlZWZ2vfeqpp+jSpUudfW666SZOOumkBte3r6rXvLevYV/7NQYDrCRJkiQ1wA9/+EMWLVrEyJEjufbaa3nppZc4/vjj+dKXvsRhhx0GwNlnn83hhx/O0KFDuf3223e/tl+/fqxZs4YlS5ZQWFjIlVdeydChQznllFMoLi4G4NJLL+XBBx/c3f+GG25g9OjRHHbYYcybNw+A1atXc/LJJzN69GiuvvpqDjroINasWVOj1ueee47Pfe5zjB49mvPPP58tW7bsft+bbrqJY445hgceeKBGe/LkyRx22GEMGzaM6667bvf7dejQgeuvv56jjjqKadOmpecbXAt3IZYkSZKU8fr98Mm0vfeSn59Z6/M///nPmT17Nu+++y4AL730Em+99RazZ8/evbPuXXfdRdeuXSkuLuaII47g3HPPpVu3blXeZ+HChUyePJk77riDCy64gIceeohLLrmkxufr3r07b7/9Nr///e/51a9+xZ133slPfvITTjjhBH70ox/xzDPPVAnJu6xZs4af/vSnTJ06lfbt2/OLX/yCm2++meuvvx4Ix9m89tprQAjlu9orVqzg6KOPZubMmRQUFHDKKafw6KOPcvbZZ7N161aGDRvGTTfd1PBvbAM4AitJkiRJjeTII4+scizMLbfcwogRIzj66KP5+OOPWbhwYY3X9O/fn5EjRwJw+OGHs2TJklrf+5xzzqnR57XXXmPSpEkAnHbaaRQUFNR43ZtvvsmcOXMYO3YsI0eO5E9/+hNLly7dff/CCy+s0n9Xe/r06YwbN44ePXqQk5PDxRdfzCuvvAJAdnY25557bn2+JY3KEVhJkiRJaiTt27ffff3SSy8xdepUpk2bRn5+PuPGjav12Ji2bdvuvs7Ozt49hXhP/bKzsyktLQXCeap7E8cxJ598MpMnT95rzZXbdb13Xl4e2dnZe/3cjc0AK0mSJCnj7Wmabzp17NiRzZs37/H+xo0bKSgoID8/n3nz5vHmm282eg3HHHMM999/P9dddx3PPfcc69evr9Hn6KOP5lvf+hYffvghBx98MNu2baOoqIjBgwfX+d5HHXUU3/3ud1mzZg0FBQVMnjyZ73znO43+NewLpxBLkiRJUgN069aNsWPHMmzYMK699toa90877TRKS0sZPnw4//Ef/8HRRx/d6DXccMMNPPfcc4wePZqnn36anj170rFjxyp9evTowT333MNFF13E8OHDOfroo3dvAlWXnj178rOf/Yzjjz+eESNGMHr0aM4666xG/xr2RVSfIefmZMyYMfGMGTOSLkOSJElSwubOnUthYWHSZSRqx44dZGdnk5OTw7Rp0/jGN76xe1OpTFDbn2EURTPjOB5TW3+nEEtSfW34GGbeA8vehDGXwWHnJV2RJElq5ZYtW8YFF1xAeXk5bdq04Y477ki6pLQywEpSXcrLYfFL8NadsOBpiMvD88tnwKCTIa9zouVJkqTWbdCgQbzzzjtJl9FkDLCSVJvi9fDuZJh+J6xbVPN+6XZY8BwMP7/pa5MkSWqlDLCSVNkn74XQOusBKK1lC/tOfWBTUbie+7gBVpIkqQkZYCWpdAd88GgIrkVv1bzfthOM/BKM+RoQw++ODM9/OBV2boM2+U1ariRJUmtlgJXUem1YBjPuhrf/DNvW1Ly//zA44goYfgG0qXTAd/fBsGYBlGyDRf+AwvFNV7MkSVIr5jmwklqX8vIwcjr5IvjtCHjt5qrhNSsXDjsfLn8Wvv5a2G24cngFKJyYup77eNPULUmSWoQOHToAsGLFCs47r/YTDcaNG8fejg79zW9+w7Zt23a3zzjjDDZs2NB4hTZTjsBKah2K18M798KMP8K6j2re79QHxlwKo78KHfar+70KJ8CrvwrX85+B0p2Q06bRS5YkSS1Xr169ePDBBxv8+t/85jdccskl5OeHpUxPPfVUY5W2V6WlpeTk5OyxXd/XNYQBVlLLtuLdsLb1/Qdr35RpwPFhmvDg0yC7nn8l9hwBnfvCxmWwYyMseQUOPqlx65YkSc3eddddx0EHHcQ3v/lNAG688UY6duzI1VdfzVlnncX69espKSnhpz/9KWeddVaV1y5ZsoTx48cze/ZsiouLueyyy5gzZw6FhYUUF6d+ZvnGN77B9OnTKS4u5rzzzuMnP/kJt9xyCytWrOD444+ne/fuvPjii/Tr148ZM2bQvXt3br75Zu666y4ArrjiCr73ve+xZMkSTj/9dI455hjeeOMNevfuzWOPPUa7du2q1LV69Wq+/vWvs2zZMiAE5bFjx3LjjTeyYsUKlixZQvfu3Rk8eHCV9s9+9jMuv/xyVq9eTY8ePbj77rvp27cvl156KV27duWdd95h9OjR/PrXv/5M33MDrKSWp2Q7zNm1KdP0mvfbdoZRF8OYy6H7oH1//ygKo7Bv/i60504xwEqSlLQb03g2+40ba3160qRJfO9739sdYO+//36eeeYZ8vLyeOSRR+jUqRNr1qzh6KOPZuLEiURRVOv7/O///i/5+fnMmjWLWbNmMXr06N33/uu//ouuXbtSVlbGiSeeyKxZs7jmmmu4+eabefHFF+nevXuV95o5cyZ33303//znP4njmKOOOorjjjuOgoICFi5cyOTJk7njjju44IILeOihh7jkkkuqvP673/0u3//+9znmmGNYtmwZp556KnPnzt393q+99hrt2rXjxhtvrNKeMGECX/nKV/jqV7/KXXfdxTXXXMOjjz4KwIIFC5g6dSrZ2dkN+/5XYoCV1HKsXwoz7oJ3/gLb1ta8f8BhcMSVcNh5Nde17qvKAXbek3DmzZD12f9SliRJmWPUqFF8+umnrFixgtWrV1NQUEDfvn0pKSnhxz/+Ma+88gpZWVksX76cVatWccABB9T6Pq+88grXXHMNAMOHD2f48OG7791///3cfvvtlJaW8sknnzBnzpwq96t77bXX+OIXv0j79uFnnXPOOYdXX32ViRMn0r9/f0aOHAnA4YcfzpIlS2q8furUqcyZM2d3e9OmTWzevBmAiRMnVhmxrdyeNm0aDz/8MABf/vKX+cEPfrC73/nnn98o4RXSGGCjKLoLGA98GsfxsDr6HQG8CVwYx3HDJ4FLap3Ky8NOwNPvgAXPAnHV+9lt4NCz4cgroc8RYfS0MRx4JLTfD7Z+CltXw8f/hIM+3zjvLUmSMsZ5553Hgw8+yMqVK5k0aRIA9957L6tXr2bmzJnk5ubSr18/tm/fXuf71DY6u3jxYn71q18xffp0CgoKuPTSS/f6PnEc7/Fe27Ztd19nZ2dXmaq8S3l5OdOmTasxtRjYHYr31K6s8tdTV799lc4R2HuAW4E/76lDFEXZwC+AZ9NYh6SWaNs6ePdemP5HWL+45v3OB4YdhEd9BTr0aPzPn5Udjs+ZEdaXMHeKAVaSpCTtYZpvuk2aNIkrr7ySNWvW8PLLLwOwceNG9ttvP3Jzc3nxxRdZunRpne9x7LHHcu+993L88ccze/ZsZs2aBYTRz/bt29O5c2dWrVrF008/zbhx4wDo2LEjmzdvrjGF+Nhjj+XSSy/lhz/8IXEc88gjj/CXv/yl3l/PKaecwq233sq1114LwLvvvrt71LYun//857nvvvv48pe/zL333ssxxxxT78+5L9IWYOM4fiWKon576fYd4CHgiHTVIamFWf52CK2zH4TSWn4DOfCEME148Knpn9JbOKFqgD31vxtvhFeSJGWEoUOHsnnzZnr37k3Pnj0BuPjii5kwYQJjxoxh5MiRDBkypM73+MY3vsFll13G8OHDGTlyJEceeSQAI0aMYNSoUQwdOpQBAwYwduzY3a+56qqrOP300+nZsycvvvji7udHjx7NpZdeuvs9rrjiCkaNGlXrdOHa3HLLLXzrW99i+PDhlJaWcuyxx3LbbbfV63WXX345v/zlL3dv4pQOUV1DzJ/5zUOAfaK2KcRRFPUG/gacAPyxot9epxCPGTMm3tuZSJJamJLt8MEjYZrw8pk17+d1hlFfDpsydRvYdHWVlcAvB8L2it/4Xvki9B5d92skSVKjmTt3LoWFhUmXoc+gtj/DKIpmxnE8prb+SW7i9BvgujiOy/a0G9cuURRdBVwF0Ldv3yYoTVKzsH5JGOF8+y9QvK7m/Z4jwmjrsHOhTX6Tl0d2LhxyBrw3ObTnTjHASpIkpVGSAXYMcF9FeO0OnBFFUWkcx49W7xjH8e3A7RBGYJu0SklNq7wcPpwajsBZ+By1bso09JxwdmufMclP2S2cUCnAPg4nXp98TZIkSS1UYgE2juP+u66jKLqHMIW4RniV1EpsWxeOv5lxVxh5ra5zXzji8jBVuH33mveTMvAEyM2Hkm2w9kNYPR/2q3udiyRJkhomncfoTAbGAd2jKCoCbgByAeI43vsqYEmtw/KZFZsyPVT7pkwHnxSmCQ86uXmes5rbLtQ257HQnjvFACtJUhOK47jWI2jU/DVkP6Z07kJ80T70vTRddUhqhkqKYfbDYZrwirdr3s/rAqMuafpNmRqqcGKlAPsYHHdtsvVIktRK5OXlsXbtWrp162aIzTBxHLN27Vry8vL26XVJroGV1NqsWwwz/gjv/BWK19e833MkHFmxKVNuzcOzm61Bp4S1uWU7YeX74evs2n/vr5MkSZ9Jnz59KCoqYvXq1UmXogbIy8ujT58++/QaA6yk9CovC5syvXVH+FhjU6a2IbAecQX0OTyREj+zvE4w4HhY+Gxoz3sCPv+dZGuSJKkVyM3NpX9/f2ncmhhgJaXH1rWpTZk2LK15v0tfGPO1ik2ZujV9fY2tcEIqwM6dYoCVJElKAwOspMYTx7D8bZh+R1jjWrajWocobHh0xBVhc6bmuClTQx1yBkRZEJfDx/+EzSuh4wFJVyVJktSiGGAlfXYlxWEX4bfugE/erXm/XUEYaR1zGXQd0PT1NYX23eCgsbDk1dCe90QI6pIkSWo0BlhJDbd2UZgi/M5fYfuGmvd7jQ6bMg39YmZtytRQh56VCrBzHjfASpIkNTIDrKR9t30jPPZtmPt4zXs5eRWbMn0NemfopkwNNeRMeOrfwvWS12DbOsjvmmxNkiRJLYgBVtK+KS+Dh66Ahc9Vfb6gX8WmTJe03tDWqRf0OQKKpkNcBvOfhlEXJ12VJElSi2GAlbRvXvzvquF10KlhmvDAEyErK7m6movCCSHAQtiN2AArSZLUaPxpU1L9ffAovPqrVHvs9+Di+8POwobXYMj41PWif8COzcnVIkmS1ML4E6ek+lk1Bx79Zqo98EQ48frk6mmuug2E/YeF67IdsPD5ZOuRJElqQQywkvZu2zq47yIo2RraBf3hvD+2rHNcG1PhxNR1bRtdSZIkqUEMsJLqVl4GD30N1i8J7dz2cNHkcLaralc4IXW94Dko2Z5cLZIkSS2IAVZS3V74SVjLucsXb4P9CpOrJxPsVwhdB4brkq3w0YvJ1iNJktRCGGAl7dn7D8Lrv021j70WDp245/4KoqjqKOzcKcnVIkmS1IIYYCXV7pNZ8Ni3U+3Bp8G4HydXT6apvA52/lNQVpJcLZIkSS2EAVZSTVvXwn0XQ2lxaHcbBOfc7lE5+6LXKOjUO1wXr4elrydbjyRJUgvgT6OSqiorhQe+ChuXhXabjjDpb5DXOdm6Mk1WVtVpxHPcjViSJOmzMsBKqur5/4Alr6ba594BPQYnV08mqxxg5z0B5eXJ1SJJktQCGGAlpbw7Gd78fap9/L/DIacnV0+m6/s5yO8erresgqLpydYjSZKU4QywkoLlb8OU76baQ8bDF/4tuXpagqxsGHJGqj3XacSSJEmfhQFWEmz5FP5+CZTtCO0eQ8J5r27a9NlV3o147hSI4+RqkSRJynD+dNrYNi6HDcuSrkKqv9KdcP9XYdPy0M7rHDZtatsx2bpaiv7HQttO4XrDUlj5frL1SJIkZTADbGNa8BzcdkwIA6U7k65Gqp9nfwzL3qhoRHDuXdBtYKIltSg5bcMZurvMnZJcLZIkSRnOANtYNq2Av18Mxetgxdsw9cakK5L27u0/w/Q7Uu2TboBBJyVXT0tVeTdi18FKkiQ1mAG2sXTqBSfekGq/+TuY92Ry9Uh78/F0ePJfU+2hX4Sx30uunpbs4BMhp124Xj0PVi9Ith5JkqQMZYBtTJ/7FgyudOTIo99wPayap80rKzZtqpjqvv8wOOt3EEXJ1tVStWkfQuwu85xGLEmS1BAG2MYURXD276HzgaG9fSM8cJnrYdW8lO6Av38ZtqwM7XYFMOneELKUPtV3I5YkSdI+M8A2tvyucN5dkJUT2stnwAs/SbYmaZc4hqf+DYreCu0oC867Gwr6JVpWqzD41NTfCyvegQ0fJ1uPJElSBjLApsOBR1ZdDzvtVpj/dHL1SLvMuCts3LTLyf8JA49Prp7WpF0X6H9cqj3vieRqkSRJylAG2HT53Ldh0Kmp9iNfd8RFyVr6Bjz9g1T7sAvCum01nUMrTSOe427EkiRJ+8oAmy5ZWfDF26BT79DevgEevBzKSpKtS63TxuVw/1egvDS0e46Aibe4aVNTO+RMoOJ7vmwabPk00XIkSZIyjQE2nfK7hvWFUXZoF70FL9yUbE1qfUq2hzOKt64O7fzucOG9kNsu2bpaow494KDPVzRij9qSJEnaRwbYdOt7FJx4far9xi2w4Nnk6lHrEsfwxPfDpkEQfplywZ+gy4HJ1tWaFU5IXbsbsSRJ0j4xwDaFz18DB5+caj9yNWwsSq4etR7//AO897dU+7SfQ79jkqtHMGR86nrxy1C8IblaJEmSMowBtilkZcEX/wAde4V28XrXwyr9Fr8Cz/441R55CRx5ZXL1KOhyIPQaFa7LS52RIUmStA8MsE2lfbdwPuyu9bAf/xP+8dNka1LLtWEZPHApxGWh3ftwOPPXbtrUXBRW2o14rrsRS5Ik1ZcBtikd9Dk44d9T7dd/AwufT64etUw7t8F9F8O2taHdfj+48K+Qm5dsXUqpHGA/nAo7tyZXiyRJUgYxwDa1sd+HgSem2o9cHY44kRpDHMOUa2DlrNDOyoUL/wKdeiVbl6rqfjD0KAzXpdtDiJUkSdJeGWCbWlYWnHM7dOwZ2tvWwkNfg7LSZOtSyzDtVnj/gVT7jF9C36OTq0d75m7EkiRJ+8wAm4T23eHcP0JU8e1fNg1e+u9ka1LmW/QPeL7SkU2HXwZjLkuuHtWtcoBd8CyU7kiuFkmSpAxhgE1Kv7FwfKUdYl/9tdMI1XDrFsMDl0FcHtoHHg2n/99ka1LdDjgMuhwUrndsCrtGS5IkqU4G2CQd868w4PhU++GrYNOK5OpRZtqxJWzatL3iPNGOPeGCP0NOm2TrUt2iCA6ttJnTnMeSq0WSJClDGGCTlJUF59wBHQ4I7W1r4aErXA+r+otjeOyb8OkHoZ3dJuw43HH/ZOtS/VTejXj+U/6/L0mStBcG2KR16AHn3plaD7v0dXj558nWpMzx2s1VR+7G/w/0GZNcPdo3vcdU/QXWsmnJ1iNJktTMGWCbg/5fgHE/SrVf+VXYkEeqy4Ln4IX/TLWPvApGXZJcPdp3WVlQOD7VdjdiSZKkOhlgm4sv/CsMGFfRiOGhK2HTJwkWpGZtzYdhujlxaB80Fk51J+uMVP04nfLy5GqRJElq5tIWYKMouiuKok+jKJq9h/sXR1E0q+LxRhRFI9JVS0bIyq5YD1uxdnHbGnj4SigvS7YuNT/bN8F9X4IdG0O7Ux84/0+QnZtsXWqYg8ZCu4JwvXkFrHgn2XokSZKasXSOwN4DnFbH/cXAcXEcDwf+E7g9jbVkhg77hRBLFNpLXoWXf5FoSWpmysvh0W/AmvmhnZMHk/4a1lIrM2XnwiFnptpzH0+uFkmSpGYubQE2juNXgHV13H8jjuP1Fc03gT7pqiWjDDgOxv0w1X75/8KiF5OrR83LK7+EeU+k2hNugV6jkqtHjaPKNOLHw+7SkiRJqqG5rIH9GvB00kU0G8deC/2PrWjE4XzYzasSLUnNwLyn4KVK61w/920YcWFy9ajxDBgHbTqE63UfwadzkqxGkiSp2Uo8wEZRdDwhwF5XR5+roiiaEUXRjNWrVzddcUnJyoZz7oT2+4X21k/hoa+5HrY1Wz0//CJjl/7HwUk/Sa4eNa7cPBh0SqrtbsSSJEm1SjTARlE0HLgTOCuO47V76hfH8e1xHI+J43hMjx6tZK1fx/3h3GrrYV/5ZaIlKSHFG2DyRbBzc2h36Qvn3wPZOYmWpUZWfTdiSZIk1ZBYgI2iqC/wMPDlOI4XJFVHszZgHBz3g1T7pZ/DRy8nVY2SUF4WdqNetyi0c/Nh0t8gv2uydanxDToZstuG61WzYe2iZOuRJElqhtJ5jM5kYBpwSBRFRVEUfS2Koq9HUfT1ii7XA92A30dR9G4URTPSVUtGO+466PeFikYcwsyWTxMtSU3oxf+Ghc+l2mf9Dg44LLl6lD5tO8LBJ6balTfrkiRJEpDeXYgviuO4ZxzHuXEc94nj+I9xHN8Wx/FtFfeviOO4II7jkRWPMemqJaNlZcO5d0L7iqnTW1Z5Pmxr8cGj8OqvUu1jvg/DzkmuHqVf5WnEczxOR5IkqbrEN3FSPXQ8AM65nd3rYT96CV79dZIVKd1WfQCPfjPVPvgkOOE/kqtHTWPwaRBlh+vlM2Dj8mTrkSRJamYMsJli4Alw7L+l2i/9DBa/mlw9Sp9t6+C+L0HJ1tDuOiCMwmdlJ1uX0i+/K/T/Qqo978nkapEkSWqGDLCZ5LgfwkFjw3VcDg9dAVtawbFCrUlZKTx4OaxfEtptOoRNm9oVJFqWmlCV3YidRixJklSZATaTZOfAuX+E/O6hvWVlxXrY8mTrUuN54Sfw0Yup9tn/C/sVJlePmt6Q8exeLrD0ddi6xxPGJEmSWh0DbKbp1BPO+UOq/dGL8JrrYVuE9x+EN25JtY/9ARw6Mbl6lIyOB8CBR4XruBzmP5VsPZIkSc2IATYTHXwSfOFfU+0X/xuWvJ5cPfrsPpkFj3071R58Goz7UXL1KFlOI5YkSaqVATZTjfsx9P18uI7L4aGvwdY1ydakhtm6Fu67GEqLQ7vboLDrdJb/e7ZaheNT1x+9BNs3JlaKJElSc+JPyJkqOyfsTNuua2hv/gQevsr1sJmmrBQe+CpsXBbabTvBRZMhr3OydSlZBf3ggOHhumwnLHw+0XIkSZKaCwNsJuvcu+J82AqLXoDX/ye5erTvnv8PWLLrOKQIzrkDug9KtCQ1E4WV1j87jViSJAkwwGa+QSfD2O+l2v/4L1j6RnL1qP7enQxv/j7VPv7f4ZDTkqtHzUvldbALn4eS4uRqkSRJaiYMsC3BCf8HDjw6XMdl8ODXPHqjuVv+Nkz5bqpdOKHqxlxSj0PCemiAkm2w6B/J1iNJktQMGGBbguxcOO+P0K4gtDevgEeudj1sc7XlU/j7JVC2I7R7FIbzXt20SZVFUdVjlOY4jViSJMmfmFuKzn3gi5XOh/3w+apniqp5KN0J938FNi0P7bzOMOleaNsx2brUPFWeRrzg6fDfjyRJUitmgG1JBp8Kn78m1X7hJlj2ZnL1qKZnfwTLpoXrKAvOuwu6DUy2JjVfPUdC5wPD9faNlTb8kiRJap0MsC3NiddDnyPDdVwGD14O29YlW5OCmX+C6Xem2ifeAAeflFw9av6iqOoo7NwpydUiSZLUDBhgW5rs3DCql9cltDcth0e+7nrYpH08HZ76t1R76Dkw9rt77i/tUjnAznsSysuSq0WSJClhBtiWqMuB8MXbUu2Fz8K0W5Orp7XbvLJi06aK9Yv7HwZn3RpG16S9OfAoaN8jXG/9FD5+K9l6JEmSEmSAbakOOR0+9+1Ue+qN/uCbhNId8Pcvw5aVod2uACb9Fdq0T7YuZY6sbBgyPtV2GrEkSWrFDLAt2Uk3Qp8jwnVcBg9c5nrYphTHYdpwUcUvDqIsOP8eKOiXZFXKRNXXwcZxcrVIkiQlyADbktVYD1sEj37TH36byoy74O0/p9qn/BQGjEuqGmWyfl8IRy4BbFwGn7ybbD2SJEkJMcC2dF36wtm/T7UXPA3TfpdcPa3F0jfg6R+k2sMvhKO/mVw9ymw5bWDw6am204glSVIrZYBtDYacCUd/K9WeekPYFVfpsXE53P8VKC8N7Z4jYMJv3bRJn43H6UiSJBlgW42TboTeh4fr8lLPh02Xku3w94th6+rQzu8OF94Lue2SrUuZb+AJkJsfrtcsgNXzk61HkiQpAQbY1iKnTcV62Err6B77luthG1McwxPfhxXvhHZWDlzw53CskfRZtcmHQSen2nMfT64WSZKkhBhgW5OCfnBWpfWw85+CN/83sXJanH/+Ad77WwH8hkIAACAASURBVKp92s+h39jk6lHLUzgxdT3HACtJklofA2xrUzgejvpGqv389VA0M7l6WorFr8CzP061R10CR1yRXD1qmQadAtltwvXKWbB+SaLlSJIkNTUDbGt08k3Qa1S4Li+BBy+F4g2JlpTR1i+F+78aztoF6D0Gzvi1mzap8eV1qnoU09wnkqpEkiQpEQbY1iinDZx3N7StWA+7wfWwDbZzW9i0qbhiQ6wO+8OFf4HcvGTrUsvlbsSSJKkVM8C2Vl37w1m3ptrznghrOFU/n86D52+AW8fAyvfDc1m5cMFfoFOvZGtTy3bIGRBV/NX98T9h88pk65EkSWpCBtjW7NCJcOTVqfZz/weWv51cPc3dltVh06s/HAe/Pwpe/w1sWp66f+avoO9RydWn1qF9dzho1+ZgMcx7MtFyJEmSmpIBtrU75T+h58hwXV4CD1zqetjKSrbD7Ifh3gvg14fAMz+ET96t2ie/G5z6Mzj80kRKVCtUeTdij9ORJEmtSE7SBShhOW3h/LvDqOKOTbBhKTz+nXB+aWvdhKi8HD5+E96bDB88Bjs21uyT3RYOOR1GXAQHnwjZuU1fp1qvIWfC09eG68WvwrZ1kN812ZokSZKagAFW0HUATPx/8MBXQ3vu4/DWHXDUVcnW1dTWLoL37oNZ94WNrWrT93MwYhIceja069K09Um7dO4ddrtePiPsfr3gGRj5paSrkiRJSjsDrIKhZ8OSK2D6naH93L/DgUekjttpqbatgw8eDsG1aHrtfboOgOGTYPgFYfMrqTkonBACLITdiA2wkiSpFTDAKuWU/4KP34KVs6BsZ1gPe/UrkNc56coaV+lOWPhcmCK84Nmw9re6vC4w7JwwRbjPEa13OrWar8IJMPWGcP3hC7BjC7TtkGxNkiRJaWaAVUpuHpx/T1gPu3MzrF8Cj18Tnsv0ABfHUDQjTA+e/RAUr6/ZJysXBp0SpggPPjWsD5aaq24DYf9hsGo2lO2AD5+HoV9MuipJkqS0MsCqqm4DYeJv4cHLQ3vOo2Fa8ZFXJltXQ61fCrPuD6Ot6xbV3qf3mBBah54D7bs1bX3SZ1E4IQRYgDmPG2AlSVKLZ4BVTcPOhSWvwYy7QvvZH8OBR0LPEcnWVV/bN8Kcx8K61qWv196nc9+wpnXEJOg+qGnrkxpL4QR46WfheuFz4din3Lxka5IkSUojA6xqd+rP4OPpsOr9sB72/q9WrIftlHRltSsrgUX/CKF1/lNQur1mnzYdYehZYV1r389DlscgK8Ptd2jYZGzdR7BzC3z0EhxyWtJVSZIkpY0BVrXLzYML/gR/ODb8YLx+MUz5Lpx3V/NZDxvH8Ml7IbTOfhC2rq7ZJ8oO57QOvxAOOQPa5Dd9nVK6RFEYhX39t6E9d4oBVpIktWgGWO1Zt4Ew4bfw0NdC+4OHod8xcMTXkq1r43J4/3547++wem7tfQ4YHqYHDzsPOu7ftPVJTalwYirAzn8Syn4L2f7VLkmSWiZ/ylHdDjsPlrwKM+8J7Wd+FI6V6Tm8aevYsSWMLr03GRa/AsQ1+3TsCYedH4Lr/kObtj4pKb1GQ6fesGl52F176esw4Likq5IkSUoLA6z27rSfhyNodh3X8cClcPXL0LZjej9veRksfjlMEZ47BUq21eyTmx9GoEZcCP2Pg6zs9NYkNTdZWTBkPLz1h9Ce+7gBVpIktVgGWO1dbrvU+bAlW8NxNFO+B+femZ71sKvmhJHW9x+AzZ/U0iEKP6APnxTW/7Xt0Pg1SJmkcEKlAPsEnP5LNymTJEktkgFW9dN9EEz4DTxccR7s7Aeh/xfg8Esb5/03rwrv+d5kWPl+7X16DAnTgw+7ADr3bpzPK7UEfT8H+d1g21rYshKWzwhHX0mSJLUwBljV3/ALwnrYt/8c2k9fB73HwAHDGvZ+JcUw78kwRXjRPyAuq9mnfY+wrnX4heEc2uayA7LUnGTnhF223/lLaM993AArSZJaJAOs9s1pvwjrYT+dE85afeCrcNVL9V8PW14eNpmZdR988Bjs3FyzT3ZbGHJmGG0deAJk5zbmVyC1TIUTKwXYKXDyf/oLH0mS1OKkLcBGUXQXMB74NI7jGkN0URRFwG+BM4BtwKVxHL+drnrUSNrkw/l/gtvHhfWwaz+EJ/4Fzrm97h+W1ywMI62z/g4bP669z0Fjw0jroWdBuy5pKV9qsQYcB207wY5NsH5J2HTtgMOSrkqSJKlRpXME9h7gVuDPe7h/OjCo4nEU8L8VH9Xc9RgM42+GR64O7ffvD+thR3+lar+ta8PZse9NhuUza3+vrgNhxEUw/Hwo6JfWsqUWLactDD41bH4GYRTWACtJklqYtAXYOI5fiaKoXx1dzgL+HMdxDLwZRVGXKIp6xnFc27azam5GTArrYd/5a2g/dS30Phy6HQwLnoH3/g4Ln4Xy0pqvbVcAw84Nuwj3GeM0R6mxFE5IBdg5j8PxP062HkmSpEaW5BrY3kDluaRFFc8ZYDPF6b+Eopmwem5YD/vXc8PGTNs31OyblRtGh0ZMgkGnhNEiSY3r4JMgJy/8/7h6bpi6331Q0lVJkiQ1miQPCqxt2C2utWMUXRVF0YwoimasXr06zWWp3trkh/Nhc/NDe/MnNcNrnyPgzF/Dvy2ASfeGESLDq5QebdqHELvL3CnJ1SJJkpQGSQbYIuDASu0+wIraOsZxfHscx2PiOB7To0ePJilO9bTfkBBQK+vSF479AXx7JlwxFY64AvK7JlOf1NoUTkhdG2AlSVILk+QU4seBb0dRdB9h86aNrn/NUCO/FKYtrpoNA0+Evp+DrCR/NyK1YoNPhaycsP58xduw4WPocuDeXydJkpQB0nmMzmRgHNA9iqIi4AYgFyCO49uApwhH6HxIOEbnsnTVoiYw7JzwkJSsdgXQ/zhY9EJoz3sSjv56sjVJkiQ1knTuQnzRXu7HwLfS9fklqdUqnJAKsHMfN8BKkqQWw3mektTSDDmT3fvkLX0DtnyaaDmSJEmNxQArSS1Nh/3CWnQAYpj/VKLlSJIkNRYDrCS1RO5GLEmSWiADrCS1RIXjU9cfvQzFG/bcV5IkKUMYYCWpJerSF3qNCtflJbDwuWTrkSRJagQGWElqqSpPI57zWHJ1SJIkNRIDrCS1VIUTU9cfvgA7tyZXiyRJUiMwwEpSS9V9EPQYEq5Li0OIlSRJymAGWElqydyNWJIktSAGWElqySoH2AXPQOnO5GqRJEn6jAywktSSHTAcuhwUrndsgsWvJFuPJEnSZ2CAlaSWLIqqTSN2N2JJkpS5DLCS1NJV3o143pNQXpZcLZIkSZ+BAVaSWro+R0CH/cP1trWwbFqy9UiSJDWQAVaSWrqsLBgyPtV2N2JJkpShDLCS1BpUP04njpOrRZIkqYEMsJLUGvQ7BtoVhOtNy2HF28nWI0mS1AAGWElqDbJz4ZAzUm2nEUuSpAxkgJWk1qLyNOI5jzuNWJIkZRwDrCS1FgOOh9z24XrdIvh0brL1SJIk7SMDrCS1Frl5MPiUVNtpxJIkKcMYYCWpNam+G7EkSVIGMcBKUmsy6BTIbhuuV70P6z5Kth5JkqR9YICVpNakbUcYeEKqPfeJ5GqRJEnaRwZYSWptqkwjfjy5OiRJkvaRAVaSWptDTocoO1wXTYdNK5KtR5IkqZ4MsJLU2uR3hX7HpNrznkyuFkmSpH1ggJWk1shpxJIkKQMZYCWpNRoyPnW95HXYuja5WiRJkurJACtJrVGnnnDgUeE6LoMFTydbjyRJUj0YYCWptao8jXiO04glSVLzZ4CVpNaq8jTij16E7ZuSq0WSJKkeDLCS1Fp17Q8HHBauy3bCwueSrUeSJGkvDLCS1JoVTkxdz52SXB2SJEn1YICVpNas8jrYhc9DSXFytUiSJO2FAVaSWrMeQ6DboHBdshUWvZhsPZIkSXUwwEpSaxZFVUdh57obsSRJar4MsJLU2lUOsPOfgrKS5GqRJEmqgwFWklq7XqOgU59wvX0jLHk12XokSZL2wAArSa1djWnE7kYsSZKaJwOsJKlagH0CysuSq0WSJGkPDLCSJOh7NLTvEa63fgpF05OtR5IkqRYGWEkSZGXDkDNT7TnuRixJkpofA6wkKai+DjaOk6tFkiSpFgZYSVLQ71ho2zlcb1wGn7yXbD2SJEnVGGAlSUFOGzjktFTb3YglSVIzY4CVJKV4nI4kSWrG0hpgoyg6LYqi+VEUfRhF0Q9rud85iqIpURS9F0XRB1EUXZbOeiRJezHwRMjND9dr5sPq+cnWI0mSVEnaAmwURdnA74DTgUOBi6IoOrRat28Bc+I4HgGMA34dRVGbdNUkSdqLNvlw8EmptqOwkiSpGUnnCOyRwIdxHH8Ux/FO4D7grGp9YqBjFEUR0AFYB5SmsSZJ0t4UTkxdz/U4HUmS1HykM8D2Bj6u1C6qeK6yW4FCYAXwPvDdOI7L01iTJGlvBp8CWbnh+pP3YP3SZOuRJEmqkM4AG9XyXPVDBU8F3gV6ASOBW6Mo6lTjjaLoqiiKZkRRNGP16tWNX6kkKSWvMwwYl2rPeyKpSiRJkqpIZ4AtAg6s1O5DGGmt7DLg4Tj4EFgMDKn+RnEc3x7H8Zg4jsf06NEjbQVLkiq4G7EkSWqG0hlgpwODoijqX7Ex0ySg+mKqZcCJAFEU7Q8cAnyUxpokSfUx5EyIKv6JWPYmbF6VbD2SJEmkMcDGcVwKfBt4FpgL3B/H8QdRFH09iqKvV3T7T+DzURS9D7wAXBfH8Zp01SRJqqf23eGgsRWNGOY/mWg5kiRJADnpfPM4jp8Cnqr23G2VrlcAp6SzBklSAxVOgCWvhus5j8OYy5OtR5IktXrpnEIsScpkQ85MXS95FbatS64WSZIkDLCSpD3p3Ad6Hx6uy0thwbPJ1iNJklo9A6wkac/cjViSJDUj9QqwURSdX5/nJEktTOHE1PWiF2DHluRqkSRJrV59R2B/VM/nJEktSbeBsN/QcF26HT6cmmw9kiSpVatzF+Ioik4HzgB6R1F0S6VbnYDSdBYmSWomCifApx+E67mPw9Czk61HkiS1Wns7RmcFMAOYCMys9Pxm4PvpKkqS1IwUToCXfx6uFzwLS9+A3HaQ0w5y8yA3H3LywnPZucnWKkmSWrQ6A2wcx+8B70VR9Lc4jksAoigqAA6M43h9UxQoSUrY/kOhoD+sXww7t8Ddp++5b5RdEW7zqn6s7bkq9yrCcL36VPqYnQtR1HTfC0mSlKi9jcDu8nwURRMr+r8LrI6i6OU4jv8lfaVJkpqFKIJh58Crv95737gshNydTbTZU5RVEWgrh+S82sNuvYL0nu7lQ9uOhmVJkhJW3wDbOY7jTVEUXQHcHcfxDVEUzUpnYZKkZuSYf4Edm2H1fCgphtJiKNkeNnYqKU59jMuatq64HEq2hke6ZeVAuwJo1xXyu0F+19DO71rtua6pj+0KILu+/9RKkqS9qe+/qjlRFPUELgD+PY31SJKao7Yd4Ixf7r1fWQmUbKsIt8V1fCyuFHxr6VOyrWY4rt63ZFvTBubyUti6Ojz2RV7nqqG2StAtSN3L75a6zm2Xnq9BkqQMV98AexPwLPB6HMfToygaACxMX1mSpIyUnQvZnUNoawplJbWE3F0f9xak69Fn12jzzq2hb0Ns3xge6xfX/zU57SpCbbUR3epBd1cIzu8GbTs5xVmS1OLVK8DGcfwA8ECl9kfAuekqSpKkesnOrdj5uFP6P1fpDti2DorXwba1la7XQfH68HHb2krPrYPiDUDcgM9VDJuWh0d9VZnivCvoFtQdfp3iLEnKMPX6VyuKoj7A/wPGEv4lfg34bhzHRWmsTZKk5iOnLXTqGR71VV4WQmzlUFtrCF5f9bnykn2vr6FTnNt23sNob7cwurvfoXDgUZCVve81SZLUyOr7a9e7gb8B51e0L6l47uR0FCVJUouQlQ3tu4VHfcVx2MW5StBdv/fw29Cdn3dsDI+6pjjnd4chZ8CQCTDguBDmJUlKQBTHe5/aFEXRu3Ecj9zbc01hzJgx8YwZM5r600qS1LxVmeJc6eO2takpztVDcEOmOLfpCINPgSHjYdDJ4XghSZIaURRFM+M4HlPbvfqOwK6JougSYHJF+yJgbWMUJ0mSGkFDpzhv37jntb2bV8KiF2DLqtRrdm6G2Q+FR3ZbGHh8CLOHnLFvI82SJDVAfQPs5cCtwP8QflX7BnBZuoqSJElNICs7tf51T8rLoWg6zJsCc6fA+iWpe2U7YMEz4RFlwUFjQ5gdciZ0OTDt5UuSWp/6TiH+E/C9OI7XV7S7Ar+K4/jyNNdXg1OIJUlKSBzDqg9g3hMhzK6avee+vUaFMFs4AXoc0nQ1SpIyXl1TiOsbYN+J43jU3p5rCgZYSZKaiXWLK8LsE/DxP9njetpug0KQLRwPvUZ7Xq0kqU6NEWDfA8ZVG4F9OY7jwxq10nowwEqS1AxtXgXznwxhdvHL4Vif2nTqXTEyOx76ft5zaCVJNTRGgP0K8CPgQcKvVy8A/iuO4780ZqH1YYCVJKmZK94AC58L04w/nAol22rv165r2PypcDwMOB5y85q2TklSs/SZA2zFmxwKnABEwAtxHM9pvBLrzwArSVIG2bkNPnoxhNn5T8P2DbX3y20fjuUpnBA+5nVu2jolSc1GowTY5sIAK0lShiorgaWvh2nG856AzZ/U3i8rFwaMCyOzh5wJHXo0ZZWSpIQZYCVJUvNSXg4r3g4js3OnwLpFe+gYQd/PhTA7ZDwUHNSkZUqSmp4BVpIkNV9xDKvnhZHZuY/Dyll77nvA8DDNeMh42K/QHY0lqQUywEqSpMyxfinMezKMzC6bxh6P5+k6sGJkdgL0Phyyspq0TElSehhgJUlSZtqyGuY/FcLs4pehbGft/Tr2hCFnhpHZfsdAdm7T1ilJajQGWEmSlPm2bwrH88x7AhY8ByVba++X1wUOOT2E2YEnQJv8pq1TkvSZGGAlSVLLUrIdPnoJ5k2BeU9B8bra++Xmw8EnhmnGg0+Fdl2atExJ0r4zwEqSpJarrDSslZ33RNgIalNR7f2ycqD/sWFkdsiZ0PGApq1TklQvBlhJktQ6xDGseKcizE6BNQv20DGCA48MYbZwPHQd0KRlSpL2zAArSZJap9ULwjTjuU+Ec2f3ZL+h4XiewvGw/zCP55GkBBlgJUmSNhaljudZ+jrE5bX3K+gHg04NHzvuDx32hw4HhOu2HZuyYklqlQywkiRJlW1dCwueDiOzi/4BZTvq97rc9hWh9oA6Ph4A7QocxZWkBjLASpIk7cmOzfDh1BBmFzwLOzd/9vfMyg0jt3sLu+17QHbOZ/98qqp0B2xbB8XrKz2qtXff3xA+RlEYde/aHwr6h3XRXQeEtiPvUpMywEqSJNVH6Q5Y/Ap88i5sXgVbVlb9WN+R2nqLQojdFWjrCr25eY38uTNAyfZ6BNBaHiXbGreO9j1CmK0ebLsOcLRdSgMDrCRJ0mcVx7B9I2xZBZtX1v1xx6bG//x5nWsPttVDb9tOzS9QlWyvRwBdlxoN3XW/tDjpyveubedUmO1aKeAW9A/TyZvbn4WUAeoKsM5ZkSRJqo8ognZdwqPHIXX33bktjNpu+bTusLttTf0///aN4bFmft39ctrtfepyh/0hvxtkZdX/8wOUFO8lgO663lD1flMH0aycMDLarmvFx4pHfteKP8OCmvfLSmD9Yli3GNZ9FB7rF8P6JVC2c8+fa8fGMGL/ybs17+XmV0xLHlBzanLnPpCVna7vgNRiOQIrSZKUlLKSEHIrT1WuLfRuWQXlpY37ubNyoP1+VcNt+/3CNOndAXVD1YBaur1xa6hPjbtCZn61MFrbY1efNh0ab+SzvAw2LU8F2/W7Am5F2C3Z2sCvLRcKDqp9WnKXvpDTtnHqlzKQU4glSZIyWXl5CJCbV9Zcl7sr4O4Ku429/rMxZOXWEkCrjYbWdr9N++Y9BTeOwy8cqgTbj1Jhd/uGhr1vlAWd+lQE2mrTkrv2D98XqQUzwEqSJLUGcRx2Va4xZbl66F0ZpiPvq91BtPrU3NpGRCv1ae5BNF22ras0Lbna1OQtqxr+vh32rxlqd43gtitovPqlhLgGVpIkqTWIIsjrFB7dB9Xdt2R7zdHbLZ+G3Y5rXTtaENZ0tsYg2lD5XcOj9+E17+3YEtbXVg61u0ZvNxYBdQwy7fpzWzat5r12BdV2TK40gtu+h39+yniOwEqSJEnNSekOWL+09mnJG5Y2fD10bvuKMNuv0rrbAbD/sBC0pWbCEVhJkiQpU+S0hR6Dw6O6slLYVFQ11K5bnJqqXNeOzyVbYdX74VFdQX/oPTqMFvcaDT2Hu9ZWzZIBVpIkScoU2TnhaJ6CfjCw2r3y8rC+ucaOyRUht67ziddXhODZD4V2lAX7HQq9RoVQ23t0aGfnpukLk+rHKcSSJElSSxfHYVOp6utt18yHVR/UfdbtLjl5cMDwVKDtNTpMQd7X84SlvUhsCnEURacBvwWygTvjOP55LX3GAb8BcoE1cRwfl86aJEmSpFYniqB9t/A48Iiq90p3wKrZsPzt8FjxNqyeT42NpEq3Q9Fb4bFLXufUKG2viinInXqm/ctRA+zcFn5xsfbD0B56drL1NFDaRmCjKMoGFgAnA0XAdOCiOI7nVOrTBXgDOC2O42VRFO0Xx/Gndb2vI7CSJElSmm3fBJ+8F8Ls8pmw/B3YuKx+r+3YsyLMVjx6jfJ4n6ZSuiPsbr32Q1i7CNYtCh/XLoLNK1L9egyBb/0zsTL3JqkR2COBD+M4/qiiiPuAs4A5lfp8CXg4juNlAHsLr5IkSZKaQF4n6P+F8Nhly+pKgbbiY/G6mq/d/AnMfzI8duk6sOrU457DIbdd+r+OlqisNOxGXTmgrlsUQuvGIojL9/4e6xaHNdMZOP07nQG2N/BxpXYRcFS1PoOB3CiKXgI6Ar+N4/jPaaxJkiRJUkN06AGDTw0PCOtqNyxNhdkV78CKd8Nux9WtqwhZ798f2lk5sF9h1anHPYaETaoUwuWmoorR0w8rpv5WXDf0KKUoO2z+1W1g+IVCaXFG7jSdzv9Cajslufp85RzgcOBEoB0wLYqiN+M4XlDljaLoKuAqgL59+6ahVEmSJEn7JIpSOyIPOyc8V14W1s8un1kxWvt2WF9bPXCVl8LK98Nj5j3hudx86Dmi6vTjgv7h87REcQybV6ZGT9cuSq1RXbcYynY04E0j6HxgCKm7gmq3g8N1l74tYhfpdAbYIuDASu0+wIpa+qyJ43grsDWKoleAEYS1s7vFcXw7cDuENbBpq1iSJElSw2Vlw/6HhsfoL4fnSrZXbBJVaerx2oU1X1uyDZZNC49d2hVUCrQVo7Ud92+ar6UxxDFsW1tpJLXylN+Pah+tro+OPUMw7TqgIqweHMJqQT/IzWvUL6G5SWeAnQ4MiqKoP7AcmERY81rZY8CtURTlAG0IU4z/J401SZIkSWpKuXnQZ0x47LJ9Y5hyXHn68ablNV9bvB4WvRAeu3TqA71HpaYe9xoZdkNOUvGGqhsm7R5V/Qh2bGzYe+Z3T42eVgmqAzJy6m9jSVuAjeO4NIqibwPPEo7RuSuO4w+iKPp6xf3b4jieG0XRM8AsoJxw1M7sdNUkSZIkqRnI6wwDxoXHLptXpo7x2TVau31DzdduKgqPuVNSz3UfnAq0vUfD/sMafyRyx5ZKU3wrRlB3BdVtaxv2nnmdq07z7TowNf036VDeTKXtGJ108RgdSZIkqRWI4xAYV7yTCrSfvBc2H9qbrFzYf2jVqcc9DglTnOtSsh3WL6425bcitG5Z2bCvI7d9tTWplab85ndtuWt8P4O6jtExwEqSJEnKDGWlsHpupanHb8OqORCX7f21ue3DdONeo0KobdO+5nmpG4uoue9sPWS3rTTNt9rmSR32N6TuIwOsJEmSpJZp57awm3HlqcfrFjX+58nKqTiGpmL0tNuA1HWn3hl5pmpzVVeA9aAlSZIkSZmrTT70PSo8dileX2nqccXH+kwBjrIqjqGpZU1q576eU9sM+CcgSZIkqWVpVwADTwiPXTatSE09/uTdcBZt12pTfgsOgpy2ydWtvTLASpIkSWr5OvUKj8LxSVeiz8CJ2pIkSZKkjGCAlSRJkiRlBAOsJEmSJCkjGGAlSZIkSRnBACtJkiRJyggGWEmSJElSRjDASpIkSZIyggFWkiRJkpQRDLCSJEmSpIxggJUkSZIkZQQDrCRJkiQpIxhgJUmSJEkZwQArSZIkScoIBlhJkiRJUkYwwEqSJEmSMoIBVpIkSZKUEQywkiRJkqSMYICVJEmSJGUEA6wkSZIkKSMYYCVJkiRJGcEAK0mSJEnKCAZYSZIkSVJGMMBKkiRJkjKCAVaSJEmSlBEMsJIkSZKkjGCAlSRJkiRlBAOsJEmSJCkjGGAlSZIkSRnBACtJkiRJyggGWEmSJElSRjDASpIkSZIyggFWkiRJkpQRDLCSJEmSpIxggJUkSZIkZQQDrCRJkiQpIxhgJUmSJEkZwQArSZIkScoIBlhJkiRJUkYwwEqSJEmSMoIBVpIkSZKUEQywkiRJkqSMYICVJEmSJGWEtAbYKIpOi6JofhRFH0ZR9MM6+h0RRVFZFEXnpbMeSZIkSVLmSluAjaIoG/gdcDpwKHBRFEWH7qHfL4Bn01WLJEmSJCnzpXME9kjgwziOP4rjeCdwH3BWLf2+AzwEfJrGWiRJkiRJGS6dAbY38HGldlHFc7tFUdQb+CJwWxrrkCRJkiS1AOkMsFEtz8XV2r+B/9/enQfHed/3Hf98916cSwKgRVKiSIFkFNmVZYuWLdKSXaduZOdQnbjjo62nbjoaJ1Xc6TStMz0yaTzT8dR/pDnsuqrHST1xqsmkjqsmSuyMxw1lHZEoi6Yum5cuHpIAkABx7vntH8+zxT2HpQAAHqJJREFUB0BguQR3F3iA92tmZ/d5nt/ufokfF8AHv9/ze/Q5dy83fSGz+8zsiJkdGRsba1uBAAAAAIDoSHTwtc9IuqFh+3pJ55a0OSDpQTOTpGFJHzazkrt/q7GRuz8g6QFJOnDgwNIQDAAAAADYBDoZYJ+StM/M9kg6K+njkj7Z2MDd91Qfm9kfSvrzpeEVAAAAAACpgwHW3Utmdr+C1YXjkr7m7s+b2WfC45z3CgAAAABoWSdHYOXuD0t6eMm+ZYOru//TTtYCAAAAAIi2Ti7iBAAAAABA2xBgAQAAAACRQIAFAAAAAEQCARYAAAAAEAkEWAAAAABAJBBgAQAAAACRQIAFAAAAAEQCARYAAAAAEAkEWAAAAABAJBBgAQAAAACRQIAFAAAAAEQCARYAAAAAEAkEWAAAAABAJBBgAQAAAACRQIAFAAAAAEQCARYAAAAAEAkEWAAAAABAJBBgAQAAAACRQIAFAAAAAEQCARYAAAAAEAkEWAAAAABAJBBgAQAAAACRQIAFAAAAAEQCARYAAAAAEAkEWAAAAABAJBBgAQAAAACRQIAFAAAAAEQCARYAAAAAEAkEWAAAAABAJBBgAQAAAACRQIAFAAAAAEQCARYAAAAAEAkEWAAAAABAJBBgAQAAAACRQIAFAAAAAEQCARYAAAAAEAkEWAAAAABAJCTWugAAAAAA6LR8qaypuaIm54uanCvK3dWbTgS3VFy96YR6UnGZ2VqXiiYIsAAAAAAiwd01Wyhrcq6gybmipsIwOjVf1OR8IQioc8Hj6v5qm/li+Yqvbyb1JOP1YJuOqyeVUN+SoFsLvOmE+lZsEzw/EWfSazsRYAEAAAB0VbniujRfHQ0tLAqa1QBaHy1dfLxU8Y7V5S7NFsqaLZSl6XxbXjOViKkvHN2thtzq4yD4xsMgHITf6uPFbRLqSQfb6URsU48SE2ABAAAArErjtNx6AC3UHy8zEjo5V9ClhVLXa03ETLmepAazwS0eM83ky5orlDSbL2k2X25plPZqFUoVXSgVdGG2Pa8Xj1lDuK2H3OqIce8yAbg3HQ9HhKsjxXHdNNLXnoK6jAALAAAAbGLVablT1dHOhvNEG6flVqfpNk7d7UTgu5JsMr4oiOZ6ksplU8G+xseNx3tS6m3h/NZyxTVbKGkuX9ZMvqS5Qim4z5c12/C4fqwegIN95Xr7fEmzhZLaPWBcrrimF0qavoY/AmSTcb34+XvaWFX3EGABAAAaTMzkdX5qQYPZpIb6UupJ8esSosfddWm+pLGZBb05nddY9TbT8Hg6r/GZgibnCh2dlruS/kxicfhcEkYHsknlskH4DPYH+zLJeMdqisdMA5mkBjLJtryeu2uhWFkchsOQO7tCGK4dawjDM2EYnsuXVShXrrmu3nTnvoadxndkAACwqU0vFPXkSxf06MkJPXZqXD96fXrR8UwypqHetIb6UtraG9yGelMa6kvXHgf36TDwsoopOmehWF42iFa335zOazzc146gcyXxmCmXrY58hmEz3B5sCKBLj/dnEpticSMzUzYVVzYVl5Ruy2sWSpVlw/BsOOq7XBieLZQXBeZcT3sC+logwAIAgE1loVjWD165qMdOTejRU+M6dmZK5SajTwvFis5Ozuvs5HxLr59OxIJQ25fS1t50EHbD7SDsBsF3OAzEfekEgXeTK1dcF2YLenN6YcWR0ur2tUwbbSaTjF02EjpYDZ9Lp+k2HOf/b/elEjGlEinlelJrXcqaIMACAIANrVSu6NmzU0FgPTmuI69cVKG08shUImbaM9yr6YWSLswWrnoUK1+q6NzUgs5NLbTUPhWP1Ud2G0JudcR3KNxfDb4DGQJDFLi7pvOlxSF0hWA6MZNv+3mSktSbimukP12/9aWXbGc03J/Slp5UR6flAu1EgAUAABuKu+vHb0zrsXBK8N+evqDp/MqjVmbSLdsHdHB0SAf3Dutdu7eqL52ovdZMPgiyE7MFTcwUdGE2r4nZgi7MFHRhtqDx2WDfhZmgTb5JOF5OoVzR65cW9Pql1gJvMm5h4E3Xpi/XR3TTtSC8tTel4d60BrIE3nbKl8oanyksE0wXalN4q/uu9v9CKxIx03BfWtsGlguk9cfDfWn1pvlVHxtPR/9Xm9k9kn5HUlzSV939C0uO/yNJnws3ZyT9srv/sJM1AQCAjefViTk9empcj54c1xOnJzQ+U2ja/qbhXh3cO6SDo8O686Yhbeldfiqemak/k1R/Jqkbh3qvWIe7a65Q1sRMQROz+VrwvRDexmfytcfVNgvFqws5xbLrjUt5vXGptWtUJmKmLY3n6vYtDr5L9ycT7T8vsRPxud2ZfL5QbrrYUXV7ar7Y3jcObelJNh0prT7OZZOKxfiDBDavjgVYM4tL+pKkD0o6I+kpM3vI3V9oaPaSpPe5+0Uz+5CkByS9u1M1AQCAjeHNSwt6/HQwJfixUxM6c7H5+anXDWR0cO+QDo0O6+DeIW0fzHakLjMLr8WY0K6hnpaeM1cohSO7hYaR3sXht3HfXOHqLltSqngthKG7MsmYtvUH4XPbSuG0P62h3rRSHfjDAbARdXIE9g5JJ939tCSZ2YOS7pVUC7Du/lhD+yckXd/BegAAQERNzRf1xOkJPR6ex3rizZmm7XM9SR0cHdKdo8M6NDqkPcO963YabU8qoZ6tCd2wtbXAu1As16Ywj4dTl+thNx+O9NYD8UyT6dO4evGYabgvdXkY7UtrpD+zKJi2ct1RAFenkwF2p6TXGrbPqPno6i9J+svlDpjZfZLuk6Rdu3a1qz4AALBOzRfKOvJKcGmbx0+N69mzU00XuelJxXXHnq06NDqsO0eHdMv2gQ07zTKTjGtnLqududZGkReKZV2cq05ZDs/hnWmcxlwPvhdmC01XZF6NTlxd1DvwoqlEbFEo3dZ/+UjpSF9aW3pSG/b/FhAFnQywy32yl/12Y2Z/V0GAfe9yx939AQXTi3XgwIHuX2UZAAB0VLFc0Q9fm6ytFPzMq5NNV/9Nxk3v2LVFh0aHdWjvkG69PscUzBVkknFtH8x2bNo0AHRTJwPsGUk3NGxfL+nc0kZmdqukr0r6kLtPdLAeAACwTlQqrhdfv1RbKfjJly5otsm5nWbS39k5qIOjwzo4OqR37d6qbIrLfgDAZtPJAPuUpH1mtkfSWUkfl/TJxgZmtkvSNyX9E3c/3sFaAADAGnJ3vTQ+q8dOBYH18VMTujjXfDXXvdv6dCi8tM179gxpsCfZpWoBAOtVxwKsu5fM7H5J31ZwGZ2vufvzZvaZ8PhXJP2GpCFJXw5PcC+5+4FO1QQAALrn9amF2irBj50a1/mp5tc53ZnL6lB4aZuDo0PaNpDpUqUAgKgw78RZ8B104MABP3LkyFqXAQAAlpicKwSrBJ8KQuvpsdmm7Yd6U7pzdEiH9gaBddfWHlZsBQDIzJ5eaWCzk1OIAQDABjabL+nJly/ULm3zwvlLTVeH7Usn9O49W3Vwb7Dw0v5t/azmCgC4KgRYAACamCuUNDFTUDxmSsRNyVhMibgpUbu3TTNqWChV9MyrF/XoqeDSNs+8OqlSk0uupBIxHbhxiw6G57HeunNQiTgrBQMAVo8ACwDYtMoV19h0Xmcn53Wu4XZ2ckHnJud1fmr+igsNSVIiZorHTMl4Q7itBt547LLjyVisHojj9bbVUJxsCMeJJc9JxsN9sfrx5KJAHbaPm+KxmJINrxE8J7b4NeL1fbUaw3orLj1/bqp2aZsjL1/UfHHllYJjJt16fa52HuvtN25RJslKwQCA9iHAAgA2rEsLRZ0Pw+jikLqgs5PzeuPSQtMRxFaVKq5SxZUvrXzd0iiKmXSlL8/N1/UH57GODuuOm7ZqIMNKwQCAziHAAgAiqViu6PWpBZ2fWj6gnpuc13S+dM3vk4ybhvvScpdKlUoQVsuuYjl4XG5DAF6vlvun7drao0N7h3Tn6LDuvGlII/3p7hcGANi0CLAAgHXH3TU5V6yF0stD6oLemF5oumBQq4Z6U9qRy2pHLhPcD2Zr2ztzWQ33pZsuNOTu9VBbqahU9iDoluv7ypUw8DYeW7Svvr8ajIPXrNRfo+wqVveF71eqVFQsB/vKlfrxYnis8X2LFVd56XuUF79v42tUw+tIf1oHwxHWO0eHdMPWnmv/ogMAsEoEWABA1y0Uy3q9IZQuF1CbnWvZqnQipp257OKAWgupwfa1nqNpFpxvmoxLWW2c8z0rYYhOxjfPIlUAgPWPAAsAaCt31/hMoWFBpHpArS6QND6Tv+b3MZNG+tLakcuGIbUeUHfmsto+mNHW3hTha5ViMVOKS9wAANYZAiwA4KrMFUq1c0xrt4aAem5qQYU2LGbUm4rXR0xzWe1cElDfMpBRKsElWQAA2EwIsACAFRXLFf3wtUkdPj6m758c10vjsy1dVuZK4jHTdQMZ7chltH3w8oC6I5fVQCbB6CkAAFiEAAsAWOTViTkdPjGmw8fH9PipiVWt5DuYTdZC6fYliyLtyGW1rT+tRJzRUwAAcHUIsACwyU0vFPX4qQk9cmJcj5wY08sTc03bJ+MWhtJMw/mm9YC6PZdVX5ofLwAAoP34DQMANplyxfXc2Sk9cmJMh4+P6wevXlSpybVMdwxmdPf+Ed21b0S337hF2/qbX1YGAACgUwiwALAJnJ+a1yPHx3X4xJgePTne9DzWbDKu99y0tRZaR0d6ORcVAACsCwRYANiA5gtl/e1LwbTgw8fHdOLNmabt37pjQHftG9Hd+4Z1++4tSic2zvVMAQDAxkGABYANwN31o9endfj4mB45Ma4nX77Q9FI2w31p3b1vWHfvH9GhvcMa6U93sVoAAIDVIcACQESNz+T1/XCE9ZGT4xqbzq/YNpWI6Y7dW3VXGFpvvq6facEAACByCLAAEBH5UllPv3JRh48HqwU/f+5S0/b7tvUF04L3D+vde4aUTTEtGAAARBsBFgDWKXfXqbHZcLXgMT1x+oLmi+UV2+d6knrv3mHdvW9Ed+0f1vbBbBerBQAA6DwCLACsI1NzRX3/ZDDC+siJcZ2dnF+xbSJmeueuLbVpwW/bOag4l7cBAAAbGAEWANZQqVzR0dcmdTg8l/XYmUk1uSSrbhzqCUZY9w3rztEh9WeS3SsWAABgjRFgAaDLXrswp785PqZHTozpsZMTms6XVmzbl07o4OhQeE3WYd041NvFSgEAANYXAiwAdNhMvqTHT03UzmV9eWJuxbZm0q3X5/S+fcO6a/+Ibrshp2Q81sVqAQAA1i8CLAC0WaXieu7clA4fH9PhE+P6wSsXVWoyL3j7YEZ37xsJr8k6pFxPqovVAgAARAcBFgDa4PWpBR0OR1gfPTmui3PFFdtmk3G956attUvcjI70cU1WAACAFhBgAWAV5gtlPfnyBR0Oz2U9/sZM0/a3bB/QXfuH9b59I7p99xalE1yTFQAA4GoRYAEg5O6ayZc0OVfU1Hxwm5wranK+oMm5oi6F269dnNORVy6qUKqs+FrDfWndvW9Yd+0f1qG9w9rWn+nivwQAAGBjIsAC2HCK5UotfAZBtFB7XL8vBPfzRU3NhffzRZWbXcOmiVQ8pnft2RJMC943opuv61eMa7ICAAC0FQEWwLrk7porlBeNgk5Vw2ctiBbqxxtGTWeaXJamnfZu69Nd+4Z19/4RvXvPVvWk+JYKAADQSfy2BazA3VUoVzSXL2uuWNZcvqTZQsN9oaS5QllzhbJMUjJuisdiSsRNiZgpEY8F9zEL98Vq++MxC9ubktXtWEzxuCkZC/bXnh8+Nx7R0bxyxYOptw2jnktDZzWcTs43jpIWVCyvbjT0WvSk4hrMJjWYTSrXk1Qum6o9HuwJ9m/pSem2G3Lakct2vT4AAIDNjACLDaFUroQhs6zZQknzhbJm80HAnC2UghBaqAfP2fziANrYtvG5zS590m1mCgNxPdjGY7HLgvDlgTl8XA3Wi4LyMs9vaJe4LEgvDuDFktcC6GQYQC8tOW90eqE7o6FLv1aD2aRy2aQGe1K1x0EgTWogm1SuJ1XbN5ith1MWVwIAAFi/CLDoqkrFtVAq1wLk4iBZ354Ng2XjaOdsvqz5YmnZ5+abLKazUbhLxbKrWC6vdSldk0rEtCUMlrlsSoNhAK2PiKYWbVfb9KcTnH8KAACwARFgsSpzhZLOTS7o/NS8zk8u6OzkvC7MFmqjnbMNo5tLw2aUJGKm3nRCvam4etIJ9aTi6knF1ZtKqCfcn0kGI3alSkWlsqtUcZXKlfA+3K5UVK64iuXqvS/arrZZ7vnliqtYqcjXz2DwVRvIJMLwmaqPeC4JnfVR0vqU3erXFgAAAJAIsFhGvlTWG1N5nZua17nJeZ2fWrjsfmq+uNZlLmKmIFSm4uptCJo9qYR60+F9Kq5sqh5Ga6E0GVdPOgil9bYJZVNxpRKxtf6n1VTCINsYgBuDbnGFkFxr0xiQLwvJlXqorlRULruKlWB/s1Aej9miUFq/r0/bHcgmI3v+LgAAANYXAuwmU6643pwOgmh1BLU2kjq1oHOTCxqfyXe0hkwyFo5gxmuhs2dJ+OxNJ5RNxuuBckmw7G14bm86oXQiJrONHZJiMVM6xogkAAAANi8C7Abi7pqYLSwKp42jpucn5/XGdH7V17lslIybrhvMaMdgVjtyWW0fzGhbfzqYbntZCK2PdmaTcUbjAAAAAKwKATYi3F2X5ks6NzW/eNR0ciHct6DzUwsqtGExo5hJbxnIaPtgRttzWe0YzGj7YFY7csH99lxGw71pFskBAAAA0FUE2HVi6aJI55aG08l5zbZpAaThvlQQRAcztdHTWlDNZfWW/rQS8fVz7icAAAAASATYriiUKnp9amGF0dNgim+7FkUayCQuD6XhqOmOwayuG8ywsisAAACASCLAttF3X3xDp8dma6On56fmdbaNiyJlk/FaEF06alq970vTpQAAAAA2JtJOG/3Od0/o2JmpVT23uijS9sEloTRcJGlHLqPBbHLDr7QLAAAAACshwLbR9sHMsgE2ZtK2/sxlo6c7WRQJAAAAAFpGgG2jD9y8TdcNhKOnDdN6t/WnlWRRJAAAAAC4JgTYNvrYu3atdQkAAAAAsGExLAgAAAAAiAQCLAAAAAAgEgiwAAAAAIBI6GiANbN7zOzHZnbSzH59meNmZr8bHj9mZu/sZD0AAAAAgOjqWIA1s7ikL0n6kKRbJH3CzG5Z0uxDkvaFt/sk/bdO1QMAAAAAiLZOjsDeIemku59294KkByXdu6TNvZK+7oEnJOXMbHsHawIAAAAARFQnA+xOSa81bJ8J911tG5nZfWZ2xMyOjI2Ntb1QAAAAAMD618kAa8vs81W0kbs/4O4H3P3AyMhIW4oDAAAAAERLJwPsGUk3NGxfL+ncKtoAAAAAANDRAPuUpH1mtsfMUpI+LumhJW0ekvSpcDXi90iacvfzHawJAAAAABBRiU69sLuXzOx+Sd+WFJf0NXd/3sw+Ex7/iqSHJX1Y0klJc5I+3al6AAAAAADR1rEAK0nu/rCCkNq47ysNj13Sv+hkDQAAAACAjaGTU4gBAAAAAGgbAiwAAAAAIBIIsAAAAACASCDAAgAAAAAigQALAAAAAIgECxYCjg4zG5P0ylrXcQXDksbXugh0FX2++dDnmwv9vfnQ55sPfb750Ofr143uPrLcgcgF2CgwsyPufmCt60D30OebD32+udDfmw99vvnQ55sPfR5NTCEGAAAAAEQCARYAAAAAEAkE2M54YK0LQNfR55sPfb650N+bD32++dDnmw99HkGcAwsAAAAAiARGYAEAAAAAkUCAXSUzu8fMfmxmJ83s15c5fq+ZHTOzo2Z2xMzeuxZ1on2u1OcN7d5lZmUz+2g360P7tfA5f7+ZTYWf86Nm9htrUSfap5XPedjvR83seTP7m27XiPZq4XP+bxo+48+F39+3rkWtaI8W+nzQzP6vmf0w/Jx/ei3qRPu00OdbzOzPwt/dnzSzt61FnWgNU4hXwcziko5L+qCkM5KekvQJd3+hoU2fpFl3dzO7VdKfuPvNa1Iwrlkrfd7Q7q8lLUj6mrv/abdrRXu0+Dl/v6Rfc/efXZMi0VYt9nlO0mOS7nH3V81sm7u/uSYF45q1+r29of3PSfpX7v6B7lWJdmrxc/7vJA26++fMbETSjyVd5+6FtagZ16bFPv+ipBl3/09mdrOkL7n7T61JwbgiRmBX5w5JJ939dPjN7EFJ9zY2cPcZr/91oFcSfymItiv2eehXJf1vSfxCG32t9jk2jlb6/JOSvunur0oS4TXyrvZz/glJ/6srlaFTWulzl9RvZiapT9IFSaXulok2aqXPb5H0XUly9x9J2m1mb+lumWgVAXZ1dkp6rWH7TLhvETP7iJn9SNJfSPpnXaoNnXHFPjeznZI+IukrXawLndPS51zSneE0s780s7d2pzR0SCt9vl/SFjP7f2b2tJl9qmvVoRNa/ZzLzHok3aPgj5SIrlb6/Pcl/aSkc5KelfQv3b3SnfLQAa30+Q8l/YIkmdkdkm6UdH1XqsNVI8Cuji2z77IRVnf/s3Da8D+Q9PmOV4VOaqXP/6ukz7l7uQv1oPNa6fMfSLrR3d8u6fckfavjVaGTWunzhKTbJf2MpJ+W9B/NbH+nC0PHtPTzPPRzkh519wsdrAed10qf/7Sko5J2SLpN0u+b2UCnC0PHtNLnX1Dwx8mjCmbTPSNG3detxFoXEFFnJN3QsH29gr/SLcvdD5vZqJkNu/t4x6tDJ7TS5wckPRjMONKwpA+bWcndCTXRdMU+d/dLDY8fNrMv8zmPtFY+52ckjbv7rKRZMzss6e0Kzq9C9FzNz/OPi+nDG0Erff5pSV8ITwU7aWYvSbpZ0pPdKRFt1urP809LUjh1/KXwhnWIEdjVeUrSPjPbY2YpBT/UHmpsYGZ7ww+AzOydklKSJrpeKdrlin3u7nvcfbe775b0p5J+hfAaaa18zq9r+JzfoeB7Kp/z6Lpin0v6P5LuMrNEOKX03ZJe7HKdaJ9W+lxmNijpfQr6H9HWSp+/KumnJCk8D/InJJ3uapVop1Z+nufCY5L0zyUdbvwjNdYXRmBXwd1LZna/pG9LiitYbfZ5M/tMePwrkn5R0qfMrChpXtLHGhZ1QsS02OfYQFrs849K+mUzKyn4nH+cz3l0tdLn7v6imf2VpGOSKpK+6u7PrV3VuBZX8b39I5K+E468I8Ja7PPPS/pDM3tWwfTTzzGzJrpa7POflPR1MytLekHSL61ZwbgiLqMDAAAAAIgEphADAAAAACKBAAsAAAAAiAQCLAAAAAAgEgiwAAAAAIBIIMACAAAAACKBAAsAQAvC6wT+yiqf+7CZ5dpYy21m9uGrfM5j7Xp/AADWCgEWAIDW5CQtG2DNLN7sie7+YXefbGMtt0m6qgDr7gfb+P4AAKwJAiwAAK35gqRRMztqZl80s/eb2ffM7I8lPStJZvYtM3vazJ43s/uqTzSzl81s2Mx2m9mLZvY/wjbfMbNs2OazZvaCmR0zswfDfb1m9jUze8rMnjGze80sJem3JH0srOVjjUWa2VvN7Mnw2DEz2xfunwnvfys8dtTMzprZH4T7/3HD8/77lUI5AABrwdx9rWsAAGDdM7Pdkv7c3d8Wbr9f0l9Iepu7vxTu2+ruF8JQ+pSk97n7hJm9LOmApD5JJyUdcPejZvYnkh5y9z8ys3OS9rh73sxy7j5pZv9Z0gvh8ZykJyW9Q9I/DF/j/mXq/D1JT7j7N8KwG3f3eTObcfe+hnaDkh6R9GlJc5L+i6RfcPeimX05fI2vt/erCADAtUmsdQEAAETYk9XwGvqsmX0kfHyDpH2SJpY85yV3Pxo+flrS7vDxMUnfMLNvSfpWuO/vS/p5M/u1cDsjadcVanpc0r83s+slfdPdTyxtYGYm6RuSftvdnzaz+yXdLump4JCykt68wvsAANB1TCEGAGD1ZqsPwhHZvyfpTnd/u6RnFATOpfINj8uq/zH5ZyR9SUGQfNrMEpJM0i+6+23hbZe7v9isIHf/Y0k/L2le0rfN7APLNPtNSWfc/Q+q5Uv6nw3v8xPu/pvN3gcAgLVAgAUAoDXTkvqbHB+UdNHd58zsZknvafWFzSwm6QZ3/56kf6tgwag+Sd+W9KvhiKnM7B1XqsXMbpJ02t1/V9JDkm5dcvxnJX1Q0mcbdn9X0kfNbFvYZquZ3dhq/QAAdAsBFgCAFrj7hKRHzew5M/viMk3+SlLCzI5J+rykJ67i5eOS/sjMnlUwcvvb4arFn5eUlHTMzJ4LtyXpe5JuWW4RJ0kfk/ScmR2VdLOkpeex/mtJOyRVF2z6LXd/QdJ/kPSdsP6/lrT9KuoHAKArWMQJAAAAABAJjMACAAAAACKBAAsAAAAAiAQCLAAAAAAgEgiwAAAAAIBIIMACAAAAACKBAAsAAAAAiAQCLAAAAAAgEgiwAAAAAIBI+P9ANtKSMitnyQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plot_learning_curve()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## 3.7. Warianty metody gradientu prostego\n", + "\n", + "* Batch gradient descent\n", + "* Stochastic gradient descent\n", + "* Mini-batch gradient descent" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### _Batch gradient descent_\n", + "\n", + "* Klasyczna wersja metody gradientu prostego\n", + "* Obliczamy gradient funkcji kosztu względem całego zbioru treningowego:\n", + " $$ \\theta := \\theta - \\alpha \\cdot \\nabla_\\theta J(\\theta) $$\n", + "* Dlatego może działać bardzo powoli\n", + "* Nie można dodawać nowych przykładów na bieżąco w trakcie trenowania modelu (*online learning*)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### *Stochastic gradient descent* (SGD)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "#### Algorytm\n", + "\n", + "Powtórz określoną liczbę razy (liczba epok):\n", + " 1. Randomizuj dane treningowe\n", + " 1. Powtórz dla każdego przykładu $i = 1, 2, \\ldots, m$:\n", + " $$ \\theta := \\theta - \\alpha \\cdot \\nabla_\\theta \\, J \\! \\left( \\theta, x^{(i)}, y^{(i)} \\right) $$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "notes" + } + }, + "source": [ + "**Randomizacja danych** to losowe potasowanie przykładów uczących (wraz z odpowiedziami)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "#### SGD - zalety\n", + "\n", + "* Dużo szybszy niż _batch gradient descent_\n", + "* Można dodawać nowe przykłady na bieżąco w trakcie trenowania (*online learning*)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "#### SGD\n", + "\n", + "* Częsta aktualizacja parametrów z dużą wariancją:\n", + "\n", + "\n", + "\n", + "* Z jednej strony dzięki temu nie utyka w złych minimach lokalnych, ale z drugiej strony może „wyskoczyć” z dobrego minimum" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### _Mini-batch gradient descent_" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "#### Algorytm\n", + "\n", + "1. Ustal rozmiar \"paczki/wsadu\" (*batch*) $b \\leq m$.\n", + "2. Powtórz określoną liczbę razy (liczba epok):\n", + " 1. Powtórz dla każdego batcha (czyli dla $i = 1, 1 + b, 1 + 2 b, \\ldots$):\n", + " $$ \\theta := \\theta - \\alpha \\cdot \\nabla_\\theta \\, J \\left( \\theta, x^{(i : i+b)}, y^{(i : i+b)} \\right) $$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "#### _Mini-batch gradient descent_\n", + "\n", + "* Kompromis między _batch gradient descent_ i SGD\n", + "* Stabilniejsza zbieżność dzięki redukcji wariancji aktualizacji parametrów\n", + "* Szybszy niż klasyczny _batch gradient descent_\n", + "* Typowa wielkość batcha: między kilka a kilkaset przykładów\n", + " * Im większy batch, tym bliżej do BGD; im mniejszy batch, tym bliżej do SGD\n", + " * BGD i SGD można traktować jako odmiany MBGD dla $b = m$ i $b = 1$" + ] + }, + { + "cell_type": "code", + "execution_count": 71, + "metadata": { + "slideshow": { + "slide_type": "skip" + } + }, + "outputs": [], + "source": [ + "# Mini-batch gradient descent - przykładowa implementacja\n", + "\n", + "def MiniBatchSGD(h, fJ, fdJ, theta, X, y, \n", + " alpha=0.001, maxEpochs=1.0, batchSize=100, \n", + " logError=True):\n", + " errorsX, errorsY = [], []\n", + " \n", + " m, n = X.shape\n", + " start, end = 0, batchSize\n", + " \n", + " maxSteps = (m * float(maxEpochs)) / batchSize\n", + " for i in range(int(maxSteps)):\n", + " XBatch, yBatch = X[start:end,:], y[start:end,:]\n", + "\n", + " theta = theta - alpha * fdJ(h, theta, XBatch, yBatch)\n", + " \n", + " if logError:\n", + " errorsX.append(float(i*batchSize)/m)\n", + " errorsY.append(fJ(h, theta, XBatch, yBatch).item())\n", + " \n", + " if start + batchSize < m:\n", + " start += batchSize\n", + " else:\n", + " start = 0\n", + " end = min(start + batchSize, m)\n", + " \n", + " return theta, (errorsX, errorsY)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "Porównanie uśrednionych krzywych uczenia na przykładzie klasyfikacji dwuklasowej zbioru [MNIST](https://en.wikipedia.org/wiki/MNIST_database):\n", + "\n", + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Wady klasycznej metody gradientu prostego, czyli dlaczego potrzebujemy optymalizacji\n", + "\n", + "* Trudno dobrać właściwą szybkość uczenia (*learning rate*)\n", + "* Jedna ustalona wartość stałej uczenia się dla wszystkich parametrów\n", + "* Funkcja kosztu dla sieci neuronowych nie jest wypukła, więc uczenie może utknąć w złym minimum lokalnym lub punkcie siodłowym" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## 3.8. Algorytmy optymalizacji metody gradientu\n", + "\n", + "* Momentum\n", + "* Nesterov Accelerated Gradient\n", + "* Adagrad\n", + "* Adadelta\n", + "* RMSprop\n", + "* Adam\n", + "* Nadam\n", + "* AMSGrad" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Momentum\n", + "\n", + "* SGD źle radzi sobie w „wąwozach” funkcji kosztu\n", + "* Momentum rozwiązuje ten problem przez dodanie współczynnika $\\gamma$, który można trakować jako „pęd” spadającej piłki:\n", + " $$ v_t := \\gamma \\, v_{t-1} + \\alpha \\, \\nabla_\\theta J(\\theta) $$\n", + " $$ \\theta := \\theta - v_t $$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Przyspiesony gradient Nesterova (*Nesterov Accelerated Gradient*, NAG)\n", + "\n", + "* Momentum czasami powoduje niekontrolowane rozpędzanie się piłki, przez co staje się „mniej sterowna”\n", + "* Nesterov do piłki posiadającej pęd dodaje „hamulec”, który spowalnia piłkę przed wzniesieniem:\n", + " $$ v_t := \\gamma \\, v_{t-1} + \\alpha \\, \\nabla_\\theta J(\\theta - \\gamma \\, v_{t-1}) $$\n", + " $$ \\theta := \\theta - v_t $$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Adagrad\n", + "\n", + "* “Adaptive gradient”\n", + "* Adagrad dostosowuje współczynnik uczenia (*learning rate*) do parametrów: zmniejsza go dla cech występujących częściej, a zwiększa dla występujących rzadziej:\n", + "* Świetny do trenowania na rzadkich (*sparse*) zbiorach danych\n", + "* Wada: współczynnik uczenia może czasami gwałtownie maleć\n", + "* Wyniki badań pokazują, że często **starannie** dobrane $\\alpha$ daje lepsze wyniki na zbiorze testowym" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Adadelta i RMSprop\n", + "* Warianty algorytmu Adagrad, które radzą sobie z problemem gwałtownych zmian współczynnika uczenia" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Adam\n", + "\n", + "* “Adaptive moment estimation”\n", + "* Łączy zalety algorytmów RMSprop i Momentum\n", + "* Można go porównać do piłki mającej ciężar i opór\n", + "* Obecnie jeden z najpopularniejszych algorytmów optymalizacji" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Nadam\n", + "* “Nesterov-accelerated adaptive moment estimation”\n", + "* Łączy zalety algorytmów Adam i Nesterov Accelerated Gradient" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### AMSGrad\n", + "* Wariant algorytmu Adam lepiej dostosowany do zadań takich jak rozpoznawanie obiektów czy tłumaczenie maszynowe" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## 3.9. Metody zbiorcze" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + " * **Metody zbiorcze** (*ensemble methods*) używają połączonych sił wielu modeli uczenia maszynowego w celu uzyskania lepszej skuteczności niż mogłaby być osiągnięta przez każdy z tych modeli z osobna." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + " * Na metodę zbiorczą składa się:\n", + " * dobór modeli\n", + " * sposób agregacji wyników" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + " * Warto zastosować randomizację, czyli przetasować zbiór uczący przed trenowaniem każdego modelu." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Uśrednianie prawdopodobieństw" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "#### Przykład\n", + "\n", + "Mamy 3 modele, które dla klas $c=1, 2, 3, 4, 5$ zwróciły prawdopodobieństwa:\n", + "\n", + "* $M_1$: [0.10, 0.40, **0.50**, 0.00, 0.00]\n", + "* $M_2$: [0.10, **0.60**, 0.20, 0.00, 0.10]\n", + "* $M_3$: [0.10, 0.30, **0.40**, 0.00, 0.20]\n", + "\n", + "Która klasa zostanie wybrana według średnich prawdopodobieństw dla każdej klasy?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "Średnie prawdopodobieństwo: [0.10, **0.43**, 0.36, 0.00, 0.10]\n", + "\n", + "Została wybrana klasa $c = 2$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Głosowanie klas" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "#### Przykład\n", + "\n", + "Mamy 3 modele, które dla klas $c=1, 2, 3, 4, 5$ zwróciły prawdopodobieństwa:\n", + "\n", + "* $M_1$: [0.10, 0.40, **0.50**, 0.00, 0.00]\n", + "* $M_2$: [0.10, **0.60**, 0.20, 0.00, 0.10]\n", + "* $M_3$: [0.10, 0.30, **0.40**, 0.00, 0.20]\n", + "\n", + "Która klasa zostanie wybrana według głosowania?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "Liczba głosów: [0, 1, **2**, 0, 0]\n", + "\n", + "Została wybrana klasa $c = 3$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Inne metody zbiorcze\n", + "\n", + " * Bagging\n", + " * Boostng\n", + " * Stacking\n", + " \n", + "https://towardsdatascience.com/ensemble-methods-bagging-boosting-and-stacking-c9214a10a205" + ] + } + ], + "metadata": { + "celltoolbar": "Slideshow", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.3" + }, + "livereveal": { + "start_slideshow_at": "selected", + "theme": "white" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/wyk/3_Metody_ewaluacji_i_optymalizacji.ipynb b/wyk/3_Metody_ewaluacji_i_optymalizacji.ipynb deleted file mode 100644 index 4b104e8..0000000 --- a/wyk/3_Metody_ewaluacji_i_optymalizacji.ipynb +++ /dev/null @@ -1,1805 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "### Uczenie maszynowe\n", - "# 3. Metody ewaluacji i optymalizacji" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "## 3.1. Metodologia testowania" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "W uczeniu maszynowym bardzo ważna jest ewaluacja budowanego modelu. Dlatego dobrze jest podzielić posiadane dane na odrębne zbiory – osobny zbiór danych do uczenia i osobny do testowania. W niektórych przypadkach potrzeba będzie dodatkowo wyodrębnić tzw. zbiór walidacyjny." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Zbiór uczący a zbiór testowy" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - "* Na zbiorze uczącym (treningowym) uczymy algorytmy, a na zbiorze testowym sprawdzamy ich poprawność.\n", - "* Zbiór uczący powinien być kilkukrotnie większy od testowego (np. 4:1, 9:1 itp.).\n", - "* Zbiór testowy często jest nieznany.\n", - "* Należy unikać mieszania danych testowych i treningowych – nie wolno „zanieczyszczać” danych treningowych danymi testowymi!" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "Czasami potrzebujemy dobrać parametry modelu, np. $\\alpha$ – który zbiór wykorzystać do tego celu?" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Zbiór walidacyjny" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - "Do doboru parametrów najlepiej użyć jeszcze innego zbioru – jest to tzw. **zbiór walidacyjny**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - " * Zbiór walidacyjny powinien mieć wielkość zbliżoną do wielkości zbioru testowego, czyli np. dane można podzielić na te trzy zbiory w proporcjach 3:1:1, 8:1:1 itp." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "### Walidacja krzyżowa" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - "Którą część danych wydzielić jako zbiór walidacyjny tak, żeby było „najlepiej”?" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - " * Niech każda partia danych pełni tę rolę naprzemiennie!" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "\n", - "Żródło: https://chrisjmccormick.wordpress.com/2013/07/31/k-fold-cross-validation-with-matlab-code/" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Walidacja krzyżowa\n", - "\n", - "* Podziel dane $D = \\left\\{ (x^{(1)}, y^{(1)}), \\ldots, (x^{(m)}, y^{(m)})\\right\\} $ na $N$ rozłącznych zbiorów $T_1,\\ldots,T_N$\n", - "* Dla $i=1,\\ldots,N$, wykonaj:\n", - " * Użyj $T_i$ do walidacji i zbiór $S_i$ do trenowania, gdzie $S_i = D \\smallsetminus T_i$. \n", - " * Zapisz model $\\theta_i$.\n", - "* Akumuluj wyniki dla modeli $\\theta_i$ dla zbiorów $T_i$.\n", - "* Ustalaj parametry uczenia na akumulowanych wynikach." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Walidacja krzyżowa – wskazówki\n", - "\n", - "* Zazwyczaj ustala się $N$ w przedziale od $4$ do $10$, tzw. $N$-krotna walidacja krzyżowa (*$N$-fold cross validation*). \n", - "* Zbiór $D$ warto zrandomizować przed podziałem.\n", - "* W jaki sposób akumulować wyniki dla wszystkich zbiórow $T_i$?\n", - "* Po ustaleniu parametrów dla każdego $T_i$, trenujemy model na całych danych treningowych z ustalonymi parametrami.\n", - "* Testujemy na zbiorze testowym (jeśli nim dysponujemy)." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### _Leave-one-out_\n", - "\n", - "Jest to szczególny przypadek walidacji krzyżowej, w której $N = m$." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - "* Jaki jest rozmiar pojedynczego zbioru $T_i$?\n", - "* Jakie są zalety i wady tej metody?\n", - "* Kiedy może być przydatna?" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Zbiór walidujący a algorytmy optymalizacji\n", - "\n", - "* Gdy błąd rośnie na zbiorze uczącym, mamy źle dobrany parametr $\\alpha$. Należy go wtedy zmniejszyć.\n", - "* Gdy błąd zmniejsza się na zbiorze trenującym, ale rośnie na zbiorze walidującym, mamy do czynienia ze zjawiskiem **nadmiernego dopasowania** (*overfitting*).\n", - "* Należy wtedy przerwać optymalizację. Automatyzacja tego procesu to _early stopping_." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "## 3.2. Miary jakości" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "Aby przeprowadzić ewaluację modelu, musimy wybrać **miarę** (**metrykę**), jakiej będziemy używać.\n", - "\n", - "Jakiej miary użyc najlepiej?\n", - " * To zależy od rodzaju zadania.\n", - " * Innych metryk używa się do regresji, a innych do klasyfikacji" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "### Metryki dla zadań regresji\n", - "\n", - "Dla zadań regresji możemy zastosować np.:\n", - " * błąd średniokwadratowy (*root-mean-square error*, RMSE):\n", - " $$ \\mathrm{RMSE} \\, = \\, \\sqrt{ \\frac{1}{m} \\sum_{i=1}^{m} \\left( \\hat{y}^{(i)} - y^{(i)} \\right)^2 } $$\n", - " * średni błąd bezwzględny (*mean absolute error*, MAE):\n", - " $$ \\mathrm{MAE} \\, = \\, \\frac{1}{m} \\sum_{i=1}^{m} \\left| \\hat{y}^{(i)} - y^{(i)} \\right| $$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "W powyższych wzorach $y^{(i)}$ oznacza **oczekiwaną** wartości zmiennej $y$ w $i$-tym przykładzie, a $\\hat{y}^{(i)}$ oznacza wartość zmiennej $y$ w $i$-tym przykładzie wyliczoną (**przewidzianą**) przez nasz model." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "### Metryki dla zadań klasyfikacji\n", - "\n", - "Aby przedstawić kilka najpopularniejszych metryk stosowanych dla zadań klasyfikacyjnych, posłużmy się następującym przykładem:" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "# Przydatne importy\n", - "\n", - "import ipywidgets as widgets\n", - "import matplotlib.pyplot as plt\n", - "import numpy as np\n", - "import pandas\n", - "import random\n", - "import seaborn\n", - "\n", - "%matplotlib inline" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "def powerme(x1,x2,n):\n", - " \"\"\"Funkcja, która generuje n potęg dla zmiennych x1 i x2 oraz ich iloczynów\"\"\"\n", - " X = []\n", - " for m in range(n+1):\n", - " for i in range(m+1):\n", - " X.append(np.multiply(np.power(x1,i),np.power(x2,(m-i))))\n", - " return np.hstack(X)" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "def plot_data_for_classification(X, Y, xlabel=None, ylabel=None, Y_predicted=[], highlight=None):\n", - " \"\"\"Wykres danych dla zadania klasyfikacji\"\"\"\n", - " fig = plt.figure(figsize=(16*.6, 9*.6))\n", - " ax = fig.add_subplot(111)\n", - " fig.subplots_adjust(left=0.1, right=0.9, bottom=0.1, top=0.9)\n", - " X = X.tolist()\n", - " Y = Y.tolist()\n", - " X1n = [x[1] for x, y in zip(X, Y) if y[0] == 0]\n", - " X1p = [x[1] for x, y in zip(X, Y) if y[0] == 1]\n", - " X2n = [x[2] for x, y in zip(X, Y) if y[0] == 0]\n", - " X2p = [x[2] for x, y in zip(X, Y) if y[0] == 1]\n", - " \n", - " if len(Y_predicted) > 0:\n", - " Y_predicted = Y_predicted.tolist()\n", - " X1tn = [x[1] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 0 and yp[0] == 0]\n", - " X1fn = [x[1] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 1 and yp[0] == 0]\n", - " X1tp = [x[1] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 1 and yp[0] == 1]\n", - " X1fp = [x[1] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 0 and yp[0] == 1]\n", - " X2tn = [x[2] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 0 and yp[0] == 0]\n", - " X2fn = [x[2] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 1 and yp[0] == 0]\n", - " X2tp = [x[2] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 1 and yp[0] == 1]\n", - " X2fp = [x[2] for x, y, yp in zip(X, Y, Y_predicted) if y[0] == 0 and yp[0] == 1]\n", - " \n", - " if highlight == 'tn':\n", - " ax.scatter(X1tn, X2tn, c='r', marker='x', s=100, label='Dane')\n", - " ax.scatter(X1fn, X2fn, c='k', marker='o', s=50, label='Dane')\n", - " ax.scatter(X1tp, X2tp, c='k', marker='o', s=50, label='Dane')\n", - " ax.scatter(X1fp, X2fp, c='k', marker='x', s=50, label='Dane')\n", - " elif highlight == 'fn':\n", - " ax.scatter(X1tn, X2tn, c='k', marker='x', s=50, label='Dane')\n", - " ax.scatter(X1fn, X2fn, c='g', marker='o', s=100, label='Dane')\n", - " ax.scatter(X1tp, X2tp, c='k', marker='o', s=50, label='Dane')\n", - " ax.scatter(X1fp, X2fp, c='k', marker='x', s=50, label='Dane')\n", - " elif highlight == 'tp':\n", - " ax.scatter(X1tn, X2tn, c='k', marker='x', s=50, label='Dane')\n", - " ax.scatter(X1fn, X2fn, c='k', marker='o', s=50, label='Dane')\n", - " ax.scatter(X1tp, X2tp, c='g', marker='o', s=100, label='Dane')\n", - " ax.scatter(X1fp, X2fp, c='k', marker='x', s=50, label='Dane')\n", - " elif highlight == 'fp':\n", - " ax.scatter(X1tn, X2tn, c='k', marker='x', s=50, label='Dane')\n", - " ax.scatter(X1fn, X2fn, c='k', marker='o', s=50, label='Dane')\n", - " ax.scatter(X1tp, X2tp, c='k', marker='o', s=50, label='Dane')\n", - " ax.scatter(X1fp, X2fp, c='r', marker='x', s=100, label='Dane')\n", - " else:\n", - " ax.scatter(X1tn, X2tn, c='r', marker='x', s=50, label='Dane')\n", - " ax.scatter(X1fn, X2fn, c='g', marker='o', s=50, label='Dane')\n", - " ax.scatter(X1tp, X2tp, c='g', marker='o', s=50, label='Dane')\n", - " ax.scatter(X1fp, X2fp, c='r', marker='x', s=50, label='Dane')\n", - "\n", - " else:\n", - " ax.scatter(X1n, X2n, c='r', marker='x', s=50, label='Dane')\n", - " ax.scatter(X1p, X2p, c='g', marker='o', s=50, label='Dane')\n", - " \n", - " if xlabel:\n", - " ax.set_xlabel(xlabel)\n", - " if ylabel:\n", - " ax.set_ylabel(ylabel)\n", - " \n", - " ax.margins(.05, .05)\n", - " return fig" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "# Wczytanie danych\n", - "import pandas\n", - "import numpy as np\n", - "\n", - "alldata = pandas.read_csv('data-metrics.tsv', sep='\\t')\n", - "data = np.matrix(alldata)\n", - "\n", - "m, n_plus_1 = data.shape\n", - "n = n_plus_1 - 1\n", - "\n", - "X2 = powerme(data[:, 1], data[:, 2], n)\n", - "Y2 = np.matrix(data[:, 0]).reshape(m, 1)" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm0AAAFmCAYAAAA/JK3gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfXRc9X3n8c9XYDkbWS22cVLH4EBqNQ04u0BVmm18KkIDIfoDDw6JTMiWtM5ySJPiAu3inHSbHNKcQrInirKlaalDQ3d9YCiRhbtV6uWxPd4NKYLlwYZDpJAtuGKDY5N0rDSSyHz3j3uvfTWakUb2zH2Yeb/OmaO5v3tn/Js7dzyfuff3YO4uAAAAZFtH2hUAAADA4ghtAAAAOUBoAwAAyAFCGwAAQA4Q2gAAAHKA0AYAAJADp6ZdgTScfvrpftZZZ6VdDQAAgDmeeOKJH7j7mmrr2jK0nXXWWRobG0u7GgAAAHOY2T/VWsflUQAAgBwgtAEAAOQAoQ0AACAHCG0AAAA5kInQZmZ3mtmrZra/xnozs6+Y2YSZPWNmF8TWXWNm4+HtmuRqDQAAkJxMhDZJX5d02QLr3y+pJ7xdK+mrkmRmqyR9RtKvSLpQ0mfMbGVTawoAAJCCTIQ2d/8HSUcW2GSzpL/ywGOSTjOztZLeJ+kBdz/i7q9JekALhz8AAIBcykRoq8M6SS/Hlg+GZbXKAQAAWkpeQptVKfMFyuc/gdm1ZjZmZmOHDh1qaOUAAACaLS+h7aCkM2PLZ0iaXKB8Hne/w9173b13zZqqs0MAAABkVl5C2x5JvxH2In2XpB+5+yuS9kq61MxWhh0QLg3LAMS5S7t3B3/rKQcAZE4mQpuZ3S3pW5LebmYHzWybmV1nZteFm4xKelHShKS/kPTbkuTuRyR9TtLj4e2WsAxA3MiItGWLdMMNxwOae7C8ZUuwHgCQaZmYMN7dr1pkvUv6RI11d0q6sxn1AlpGoSBt3y4NDQXLg4NBYBsaCsoLhXTrBwBYVCZCG4AmMwuCmhQEtSi8bd8elFu1Pj0AgCwxb8O2LL29vT42NpZ2NYDkuUsdsVYR5TKBDQAyxMyecPfeausy0aYNQAKiNmxx8TZuAIBMI7QB7SAKbFEbtnL5eBs3ghsA5AJt2oB2MDJyPLBFbdjibdz6+qQrrki3jgCABRHagHZQKEjDw8HfqA1bFNz6+ug9CgA5QGgD2oFZ9TNptcoBAJlDmzYAAIAcILQBAADkAKENAAAgBwhtAAAAOUBoAwAAyAFCGwAAQA4Q2gAAAHKAcdoAAKhQmi6peKCo8cPj6lndo4FzB9S9vDvtaqHNEdoAAIjZ99I+9e/qV9nLmpqdUteyLt2490aNXj2qTes3pV09tDEujwIAECpNl9S/q1+lmZKmZqckSVOzUyrNBOVHZ46mXEO0M0IbAACh4oGiyl6uuq7sZRX3FxOuEXAcoQ0AgND44fFjZ9gqTc1OaeLIRMI1Ao4jtAEAEOpZtUFdtrzqui5brg2rfj7hGgHHEdoAAAgNfK9LHT+Zrrqu4yfTGnjxjQnXCDiO0AYAQKj7A1dpdPpKdU9LXR4MsNDlp6p7WhqdvlIrPnBVyjVEO2PIDwAAImba9F/u1eQNn1Dx776qiVXShiOva+DXPq4Vg7dLZmnXEG3M3D3tOiSut7fXx8bG0q4GACCr3KWO2MWocpnAhkSY2RPu3lttHZdHAQCIc5duuGFu2Q03BOVAightAABEosA2NCRt3x6cYdu+PVgmuCFltGkDACAyMnI8sA0OBpdEBweDdUNDUl+fdMUV6dYRbSsToc3MLpM0JOkUSTvd/daK9YOS3hMuvlHSm9z9tHDdTyU9G657yd0vT6bWAICWUyhIw8PB36gNWxTc+vqCciAlqXdEMLNTJH1H0iWSDkp6XNJV7v5cje1/R9L57v5b4fJRd1+xlH+TjggAACCLst4R4UJJE+7+orvPSLpH0uYFtr9K0t2J1AwAACAjshDa1kl6ObZ8MCybx8zeKulsSQ/Hit9gZmNm9piZcd4aQHLcpd275zdOr1UOACchC6Gt2sA3tf6n2yrpPnf/aaxsfXga8cOSvmxmVSeGM7Nrw3A3dujQoZOrMQBIQaP1LVvm9iqMeh9u2RKsB4AGyUJoOyjpzNjyGZIma2y7VRWXRt19Mvz7oqRHJZ1f7YHufoe797p775o1a062zgAQNEqvHA4iPlwEjdYBNFAWeo8+LqnHzM6W9M8KgtmHKzcys7dLWinpW7GylZJ+7O7TZna6pHdL+kIitQaAyuEghoaC+/HhIgCgQVI/0+bur0v6pKS9kp6XdK+7HzCzW8wsPnzHVZLu8bndXd8haczMnpb0iKRba/U6BYCmiAe3CIENQBNk4Uyb3H1U0mhF2R9WLH+2yuP+t6R3NrVyALCQWlMeEdwANFjqZ9oAILeY8ghAgjJxpg0AcokpjwAkiNAGACeKKY8AJIjQBgAnyqz6mbRa5QBwEmjTBgAAkAOENgAAgBwgtAEAAOQAoQ0AACAHCG0AAAA5QGgDAADIAUIbAABADhDaAAAAcoDQBgAAkAOENgAAgBwgtAEAAOQAoQ0AACAHCG0AAAA5QGgDAADIAUIbAABADhDaAAAAcoDQBgAAkAOENgAAgBwgtAEAAOQAoQ0AACAHCG0AAAA5QGgDAADIAUIbAABADmQitJnZZWb2gplNmNmOKus/amaHzOyp8Pax2LprzGw8vF2TbM0BAACScWraFTCzUyTdLukSSQclPW5me9z9uYpNi+7+yYrHrpL0GUm9klzSE+FjX0ug6gAAAInJwpm2CyVNuPuL7j4j6R5Jm+t87PskPeDuR8Kg9oCky5pUTwAAgNRkIbStk/RybPlgWFbpA2b2jJndZ2ZnLvGxMrNrzWzMzMYOHTrUiHoDAAAkJguhzaqUecXy30g6y93/raQHJd21hMcGhe53uHuvu/euWbPmhCsLpK00XdLOJ3fq5gdu1s4nd6o0XUq7SgCABKTepk3B2bEzY8tnSJqMb+Duh2OLfyHptthjL6p47KMNryGQEfte2qf+Xf0qe1lTs1PqWtalG/feqNGrR7Vp/aa0qwcAaKIsnGl7XFKPmZ1tZp2StkraE9/AzNbGFi+X9Hx4f6+kS81spZmtlHRpWAa0nNJ0Sf27+lWaKWlqdkqSNDU7pdJMUH505mjKNQQANFPqoc3dX5f0SQVh63lJ97r7ATO7xcwuDze73swOmNnTkq6X9NHwsUckfU5B8Htc0i1hGdByigeKKnu56rqyl1XcX0y4RgCAJGXh8qjcfVTSaEXZH8buf0rSp2o89k5Jdza1gkAGjB8eP3aGrdLU7JQmjkwkXCMAQJJSP9MGoD49q3vUtayr6rquZV3asGpDwjUCACSJ0AbkxMC5A+qw6h/ZDuvQwMaBhGtUhbu0e3fwt55yAEDdCG1ATnQv79bo1aPq7uw+dsata1mXujuD8hWdK1KuoaSREWnLFumGG44HNPdgecuWYD0A4IRkok0bgPpsWr9JkzdNqri/qIkjE9qwaoMGNg5kI7BJUqEgbd8uDQ0Fy4ODQWAbGgrKC4V06wcAOWbehpcrent7fWxsLO1qAK0pOrMWBTcpCGyDg5JVGw8bABAxsyfcvbfqOkIbgIZzlzpirS/KZQIbANRhodBGm7ZWRqNwpCE60xYXb+MGADghhLZWRqNwJC1+aXT79uAMW9TGjeAGACeFjgitjEbhSNrIyPHjK2rDNjgYrBsakvr6pCuuSLeOAJBTtGlrdTQKR5Lcg+BWKMw9vmqVAwDmoCNChbYKbRKNwgEAyAk6IrQzGoUDANASCG2tjEbhAAC0DDoitDIahQMA0DIIba2sUJCGh+c2/o6CW18fvUcBAMgRLo+2MrPgTFplp4Na5WhvDMYMAJlGaAOg0nRJO7/2Cd38p1u08+ZLVPrJvwQrGIy55ZWmS9r55E7d/MDN2vnkTpWmS2lXCUANDPkBtLl9L+1T/65+lb2sqdkpdU1LHZ3LNPqbD2nT4Dfmt4tEy5j33i/rUod1aPTqUW1avynt6jUf4woigxjyA0BVpemS+nf1qzRT0tTslCRparlUsln1//mv6ehXCWytqup7Pzul0kxQfnTmaMo1TABT/SFnCG1AGyseKKrs5arrypKK54rA1qIWfO+9rOL+YsI1SkF8qr8ouDHVHzKM0Aa0sfHD48fOslSaWi5NrBJj+rWoBd/72SlNHJlIuEYpiHrTR8Gto4PmAMg0QhvQxnpW96hrWVfVdV3LurThgl9nMOYWteh7v2pDwjVKSXz8ygiBDRlFaAPa2MC5A+qw6v8NdFiHBj63+/hZCNr3tJRF3/uNAwnXKCVM9YccIbQBbax7ebdGrx5Vd2f3sbMuXcu61N0ZlK9Y3h2cdYgGaUbLWPS971yRcg0TwFR/yBmG/ACgozNHVdxf1MSRCW1YtUEDGwfa40sb7f3e794d9BKNt2GLB7nhYab6Q+IWGvKD0AYwVhPQnvjsI4MyP06bmV1mZi+Y2YSZ7aiy/kYze87MnjGzh8zsrbF1PzWzp8LbnmRrjpbAWE1Ae2KqP+RM6hPGm9kpkm6XdImkg5IeN7M97v5cbLP/I6nX3X9sZh+X9AVJUSvZf3X38xKtNFpLfKwmKbhMwlhNAICMST20SbpQ0oS7vyhJZnaPpM2SjoU2d38ktv1jkj6SaA3R2uJd/oeGjoc3xmoCAGRIFi6PrpP0cmz5YFhWyzZJ34wtv8HMxszsMTPjlAhODGM1AQAyLguhrdq3YtXeEWb2EUm9kr4YK14fNtj7sKQvm9nP13jstWG4Gzt06NDJ1hmthrGaAAAZl4XQdlDSmbHlMyRNVm5kZu+V9GlJl7v7dFTu7pPh3xclPSrp/Gr/iLvf4e697t67Zs2axtUe+cdYTQCAHMhCaHtcUo+ZnW1mnZK2SprTC9TMzpf05woC26ux8pVmtjy8f7qkdyvWFg6oy8jI/PkG4/MR0nsUAJABqYc2d39d0icl7ZX0vKR73f2Amd1iZpeHm31R0gpJf10xtMc7JI2Z2dOSHpF0a0WvUyzEPRhcsvJMUq3yVlUoBINoxtuwRcGNmQAAoP1k9PuRwXXbGaOBAwAwX4rfjwsNrpuFIT+QFsYnAwBgvox+P3Kmrd3FfzlEGJ8MqKk0XVLxQFHjh8fVs7pHA+cOqHt5d9rVAtBoKX0/MvdoBUJbBXepI9a8sVwmsAFV7Htpn/p39avsZU3NTqlrWZc6rEOjV49q0/pNaVcPQKOl8P2Y+blHkSLGJwPqUpouqX9Xv0ozJU3NTkmSpmanVJoJyo/OHE25hgAaKoPfj4S2dsb4ZEDdigeKKnu56rqyl1XcX0y4RgCaJqPfj3REaGe1xieTgvK+PnqPAqHxw+PHzrBVmpqd0sSRiYRrBKBpMvr9SGhrZ9H4ZIXC/PHJ+vroPQrE9KzuUdeyrqrBrWtZlzas2pBCrQA0RUa/H+mIAAB1KE2XtO5L61SaKc1b193ZrcmbJrWic0UKNQPQSuiIAAAnqXt5t0avHlV3Z7e6lnVJCs6wdXcG5QQ2AM3G5VEAqNOm9Zs0edOkivuLmjgyoQ2rNmhg4wCBDUAiCG0AsAQrOldo2wXb0q4GgDbE5VEAAIAcILQBAADkAKENAAAgBwhtSJ+7tHv3/BGma5UDANCGCG1I38iItGXL3KlBoilEtmwJ1gMA0OYIbUhfoTB/Trf4nG/MzAAAOBktckWH0Ib0RVODRMGto2P+nG8AAJyoFrmiQ2hDNsQn440Q2AAAjdAiV3QIbciG6AMUF/9FBADAiWqRKzqENqSv8hdPuTz/FxEAACejBa7oENqQvpGR+b944r+IctLWAACQYS1wRYfQhvQVCtLw8NxfPFFwGx7OTVsDAEBGtcgVHSaMR/rMpCuuqL8cAIClqHVFRwrK+/py8X1DaAMAAK0tuqJTKMy/otPXl5srOoQ2AADQ2lrkig5t2gAAAHIgE6HNzC4zsxfMbMLMdlRZv9zMiuH6b5vZWbF1nwrLXzCz9yVZbwAAgKSkHtrM7BRJt0t6v6RzJF1lZudUbLZN0mvuvkHSoKTbwseeI2mrpHMlXSbpT8PnAwAAaClZaNN2oaQJd39RkszsHkmbJT0X22azpM+G9++T9CdmZmH5Pe4+Lel7ZjYRPt+3Eqo7ALS10nRJxQNFjR8eV8/qHg2cO6Du5d1pVwtoSXWHNjO7RNKHJN3u7k+Z2bXufkcD6rBO0sux5YOSfqXWNu7+upn9SNLqsPyxiseua0CdAACL2PfSPvXv6lfZy5qanVLXsi7duPdGjV49qk3rN6VdPaDlLOXy6G9L+n1JHzGziyWd16A6VJs/onKUu1rb1PPY4AnMrjWzMTMbO3To0BKrCACIK02X1L+rX6WZkqZmpyRJU7NTKs0E5UdnjqZcQ6D1LCW0HXL3H7r770m6VNIvN6gOByWdGVs+Q9JkrW3M7FRJPyvpSJ2PlSS5+x3u3uvuvWvWrGlQ1QGgPRUPFFX2ctV1ZS+ruL+YcI2A1reU0Pa30R133yHprxpUh8cl9ZjZ2WbWqaBjwZ6KbfZIuia8f6Wkh93dw/KtYe/SsyX1SPrHBtULAFDD+OHxY2fYKk3NTmniyETCNQJa36Khzcy+bGbm7vfHy939vzaiAu7+uqRPStor6XlJ97r7ATO7xcwuDzf7mqTVYUeDGyXtCB97QNK9Cjot/J2kT7j7TxtRLwBAbT2re9S1rKvquq5lXdqwakPCNQJan/kik6Sa2R9J+neSBtz9x2Z2qaTPuPu7k6hgM/T29vrY2Fja1QCA3CpNl7TuS+tUminNW9fd2a3Jmya1onNFCjUD8s3MnnD33mrrFj3T5u5/IOluSX9vZvsk3aTwTBcAoD11L+/W6NWj6u7sPnbGrWtZl7o7g3ICG06au7R7d/C3nvI2sOiQH2b265L+o6QpSWslbXP3F5pdMQBAtm1av0mTN02quL+oiSMT2rBqgwY2DhDY0BgjI9KWLdL27cHE7mZBULvhBmloKJgAPkfzhjZCPeO0fVrSf3b3fWb2TklFM7vR3R9uct0AABm3onOFtl2wLe1qoBUVCkFgGxoKlgcHjwe27duD9W1m0dDm7hfH7j9rZu+X9A1Jv9rMigEAgDZmFgQ1KQhqUXiLn3lrM4t2RKj6ILN/4+7/2oT6JIKOCAAA5IS71BFrgl8ut3RgO6mOCNXkObABwMkoTZe088mduvmBm7XzyZ0qTc/vPQmgQaI2bHE33NCWnRCkbEwYDwC5wFybQILinQ6iS6LRstSWl0hP6EwbALQb5toEEjYyMjewRW3cos4JIyNp1zBxhDYAqANzbQIJKxSCYT3iZ9Si4DY8TO9RAEB1zLUJJMys+jhstcrbAGfaAKAOzLUJIG2ENgCow8C5A+qw6v9ldliHBjYOJFwjAO2G0JYE5k8Dco+5NpPBkCpAbSc0uG7eJT647u7dzJ8GtIijM0eZa7NJqg2p0mEdDKmCtrLQ4LqEtiQsNNZMG0/HAQCR0nRJ6760TqWZ+WfWuju7NXnTJOEYbaHhMyJgiSrHlunoILABQAxDqgCLI7QlJT7xbYTABgCSGFIFqAehLSnMnwYANTGkCrA4QlsSKtu0lcvHL5US3ACAIVWAOhDaksD8aQCwIIZUARZH79EkuAfBrFCY24atVnm7Y38BbYshVdDuGPKjQuKhDUvDuHYAgDbFkB/Il0Jhfpu/eJvAQiHtGgIAGomZg+pCaEP2MK4dALSXkZHgCku8c170g33LFtp+hwhtyCbGtQOA9sEVlroQ2pCsek+BM65dNnEJA0AzcIWlLoQ2JKueU+CMa5ddXMIA0CxcYVlUqqHNzFaZ2QNmNh7+XVllm/PM7FtmdsDMnjGzgdi6r5vZ98zsqfB2XrKvIGPycBaknlPgjGuXXVzCANAsXGFZnLundpP0BUk7wvs7JN1WZZtfkNQT3n+LpFcknRYuf13SlUv9d3/pl37JW9LwsLvkvn27e7kclJXLwbIUrM+CeJ2iW2Wdh4ePL8cfV60cyVrs/QOApYr/vxL9f1K53CYkjXmt3FRrRRI3SS9IWhveXyvphToe83QsxBHa4vJ00JfLc7/0s1S3E9FuQbPV3j8A6crLSYcELBTa0m7T9mZ3f0WSwr9vWmhjM7tQUqek78aKPx9eNh00s+XNq2oO5KUhp7fgKfB2auvViu8fgHQVCsHA6fHvqug7bXiYpheRWmmuUTdJD0raX+W2WdIPK7Z9bYHnWavgzNy7KspM0nJJd0n6wwUef62kMUlj69evb3gyzpQsnwXJ09nApWjV11WpXV4nAKREeb88KulnJD0p6YMLPNdFkv5HPf9uy14edc9+e6NWPgWe9X3fCK38/gFABiwU2lKde9TMvijpsLvfamY7JK1y9/9UsU2npG9K+ht3/3LFurXu/oqZmaRBST9x9x2L/bstO/eoV/TkGxycv5z2JVJv8cng3YPL0pFyOd+vp1Krv38AkLLMThhvZqsl3StpvaSXFJxJO2JmvZKuc/ePmdlHJP2lpAOxh37U3Z8ys4clrVFwifSp8DFHF/t3Wza0MdF6uuL7OpKVsAwAyIXMhra0tGxo4yxIevJwlhMAkHkLhbZTk64Mmsis+pm0WuVonFoDAktBeV8f7wEA4KSkPeQH0Brorg4gSzwHM+RgyQhtQCNEZzMrL4HWKgeAZmqnsSPbCJdHAQBoNfF5gqX57Ww5+59LhDYAAFpNZbvaKLzRMSrX6D0KAECravWxI1vQQr1HadMGAEAritqwxTFPcK4R2gAAaDWVY0eWy8fbuBHccos2bQAAtBrGjmxJhDa0pNJ0ScUDRY0fHlfP6h4NnDug7uXdaVcLAJIRjR0ZnwknCm59ffQezSk6IqDl7Htpn/p39avsZU3NTqlrWZc6rEOjV49q0/pNaVcPAICa6IiAtlGaLql/V79KMyVNzU5JkqZmp1SaCcqPzhxNuYYAAJwYQhtaSvFAUWUvV11X9rKK+4sJ1wgAgMYgtKGljB8eP3aGrdLU7JQmjkwkXCMAABqD0IaW0rO6R13Luqqu61rWpQ2rNiRcIwAAGoPQhpYycO6AOqz6Yd1hHRrYOJBwjQAAaAxCG1pK9/JujV49qu7O7mNn3LqWdam7Myhf0bki5RoCAHBiGKcNLWfT+k2avGlSxf1FTRyZ0IZVGzSwcYDABqClMT5l62OcNgAAco7xKVsH47ThOHdp9+75887VKgcAZBrjU7YPQlu7GRmRtmyZO2FwNLHwli3BegBAbjA+ZfugTVu7KRSCCYSHhoLlwcEgsEUTCzMfHQDkCuNTtg9CW7uJJgyWgqAWhbft24PyaGJhAEAuRONTVgtujE/ZWuiI0K7cpY7Y1fFymcAGADlUmi5p3ZfWqTRTmreuu7NbkzdN0ns+R+iIgLmiNmxx8TZuAIDcYHzK9sHl0XYTBbaoDVu8TZvEJVIAyCHGp2wPqYY2M1slqSjpLEn/V9KH3P21Ktv9VNKz4eJL7n55WH62pHskrZL0pKT/4O4zza95jo2MzA1slW3c+vqkK65Irj7uQZ0KhblhsVY5AKCqFZ0rtO2CbWlXA02U9uXRHZIecvceSQ+Fy9X8q7ufF94uj5XfJmkwfPxrkjhaF1MoSMPDc8+oRcFteDj53qMMQQIAQF3SDm2bJd0V3r9LUt2JwcxM0sWS7juRx7cts+BMWuXZq1rlzRYfgiQKbgxBAgDAPGm3aXuzu78iSe7+ipm9qcZ2bzCzMUmvS7rV3UckrZb0Q3d/PdzmoKR1Ta8xGoshSAAAqEvTQ5uZPSjp56qs+vQSnma9u0+a2dskPWxmz0r6lyrb1ez+aGbXSrpWktavX7+EfxpNFwW3KLBJBDYAACo0/fKou7/X3TdWud0v6ftmtlaSwr+v1niOyfDvi5IelXS+pB9IOs3MouB5hqTJBepxh7v3unvvmjVrGvb60AAMQQIAwKLSbtO2R9I14f1rJN1fuYGZrTSz5eH90yW9W9JzHowK/IikKxd6PDKusg1buTy/jRsAAEg9tN0q6RIzG5d0SbgsM+s1s53hNu+QNGZmTysIabe6+3Phupsl3WhmEwrauH0t0drj5NUagiQKbvQeBQBAEtNYIW2M0wYAyIoMfCcxjRWyK2tDkAAA2lfGxw5Ne8gPABlTmi6peKCo8cPj6lndo4FzB9S9vDvtagFA88XHDpXmTvWYgbFDuTwK4Jh9L+1T/65+lb2sqdkpdS3rUod1aPTqUW1avynt6gFA88U7yEUSHDt0ocujhDYAkoIzbOu+tE6lmdK8dd2d3Zq8aZLJpwG0B3epI9aCrFxOrLkObdoALKp4oKiyl6uuK3tZxf3FhGsEACnI8NihhDYAkqTxw+Oamp2qum5qdkoTRyYSrhEAJCzjY4cS2hrJXdq9e/6bWqscyJCe1T3qWtZVdV3Xsi5tWLUh4RoBQMIyPnYooa2RMt5VGFjIwLkD6rDq/yV0WIcGNg4kXCMASFihIA0Pz+10EAW34eHUe48S2hop3lU4Cm4Z6ioMLKR7ebdGrx5Vd2f3sTNuXcu61N0ZlNMJAUDLy/jYofQebbSUuwoDJ+vozFEV9xc1cWRCG1Zt0MDGAQJbI2VgxHUA2cWQHxWaPuRHil2FAWTc7t1Bc4n4j7n4j73h4eAXPbKJ0I0mY8iPJGW4qzCADKAZReOk0fmLtstIEaGtkTLeVRhABlT2RuvomN9bDfVJI0ARupEiLo82Epc9ANSLZhQnrzIwVc4T2awQTNtlNBFt2io0LbTR1gFAPfjSb5y09iWhG01Cm7akZLyrMIAMoBlFY0WXm+OSCGy0XUYKCG0AkKSMj7ieO0kHKEI3UkRoA4AkZXzE9VxJI0ARupEi2rQBAPIpjc5ftF1Gk9ERoQKhDQBaAAEKLWih0HZq0pUBAKAhok5e9ZYDOUebNgAAgBwgtAEAgGSkMfVYCyG0AQCAZDB360mhTRsAAEhGfO5Waf7UYwx5syBCGwAASEZ8BouhoePhjWnc6sKQHwAAIFnM3VpTZuceNbNVZvaAmY2Hf1dW2eY9ZvZU7PYTMyuE675uZt+LrTsv+VcBAADqxtytJyztjl7wUusAAA2HSURBVAg7JD3k7j2SHgqX53D3R9z9PHc/T9LFkn4s6X/GNvn9aL27P5VIrQEAwNIxd+tJSbtN22ZJF4X375L0qKSbF9j+SknfdPcfN7daAACg4WrN3SoF5X19DIy8gLTPtL3Z3V+RpPDvmxbZfqukuyvKPm9mz5jZoJktb0YlAQBAAxQKwZyw8U4HUXAbHqb36CKaHtrM7EEz21/ltnmJz7NW0jsl7Y0Vf0rSL0r6ZUmrtMBZOjO71szGzGzs0KFDJ/BKgCViEElgLj4TiKYYq+x0UKscczQ9tLn7e919Y5Xb/ZK+H4axKJS9usBTfUjSbnefjT33Kx6YlvSXki5coB53uHuvu/euWbOmMS8OWAiDSAKBKJTt3j33M+EenF353d/lMwHUIe02bXskXSPp1vDv/Qtse5WCM2vHmNlad3/FzExSQdL+ZlUUWDIGkQQC0Q+Y668PbkNDx3/IfOUrwV8+E8CiUh2nzcxWS7pX0npJL0n6oLsfMbNeSde5+8fC7c6S9L8knenu5djjH5a0RpJJeip8zNHF/l3GaUNi4j2lIgwiiXYT/xxcf31QFoU1KSj78pf5TABaeJw2BtcFmo1BJIHqP2AifCaAYzI7uC7Q8hhEEgjEh3aoxGcCqAuhDWgWBpEEjnMPOhzExdu48ZkAFkVoA5ql1iCSUXCjpxzaRfQDJmrHFoW1+DKfCWBRafceBVpXNIhkoTB/EMm+PnrKoX1EP2Cuv37uiPdmQfk3viFddBGfCWARdEQAADSXexDc4j9gFioH2thCHRE40wYAaK5otPt6ywFURZs2AACAHCC0AQAA5AChDQAAIAcIbQDQaNEE6ZUdvWqVA0AdCG0A0GjRBOnxAWOjscq2bGE8MgAnhN6jANBohcLxQZSlYGy++OwYjEcG4AQQ2gCg0eLzbA4NHQ9v8dkxAGCJuDwKAM1QbYL0vAQ22uQBmURoA4BmiNqwxeVlUnTa5AGZRGgDgEaLAk7Uhq1cPt7GLQ/BLd4mL6ovbfKA1NGmDQAaLZogPd6GLd7GLT5pehbRJg/IJCaMB4BGa5UJ0t2ljtgFmXI5H/VGc7XK8Z1RC00Yz+VRAGi0aCL0yi+uWuVZlOc2eWiuZrZ5pBPMgghtAIC58t4mD83VzDaPdIJZEG3aAABz5b1NHpqrmW0eGZh6QbRpAwDMRZsl1KNZbR7jZ+4ibdQJhjZtANDultJWqBXa5KG5mtnmMc8DUzcZoQ0A2gFthdAozW7zSCeYmghtANAOGDAXjVKrzWN0fJ1s71E6wdREmzYAaBdt3lYIDdLMNo+7dwdnfuPHZfy4HR5u+U4wC7VpI7QBQBYk1fifAXORZXSCyW5HBDP7oJkdMLOymVWtYLjdZWb2gplNmNmOWPnZZvZtMxs3s6KZdSZTcwBosCTanNFWCFlHJ5gFpd2mbb+kLZL+odYGZnaKpNslvV/SOZKuMrNzwtW3SRp09x5Jr0na1tzqAkCTNLvNGW2FgNxLdXBdd39ekmzh5HyhpAl3fzHc9h5Jm83seUkXS/pwuN1dkj4r6avNqi8ANE2zJ2lnwFwg99I+01aPdZJeji0fDMtWS/qhu79eUQ4A+dTM8akKhaARd/z5on9veJjeo0AOND20mdmDZra/ym1zvU9RpcwXKK9Vj2vNbMzMxg4dOlTnPw0ACWr2gKW0FQJyremhzd3f6+4bq9zur/MpDko6M7Z8hqRJST+QdJqZnVpRXqsed7h7r7v3rlmz5kReCgA0D23OACwiD5dHH5fUE/YU7ZS0VdIeD8YqeUTSleF210iqNwgCQLY0c8BSAC0h7SE/rjCzg5L+vaS/NbO9YflbzGxUksI2a5+UtFfS85LudfcD4VPcLOlGM5tQ0Mbta0m/BgBoCNqcAVgEg+sCAABkRGYH1wUAAEB9CG0AAAA5QGgDAADIAUIbAABADhDaAAAAcoDQBgAAkAOENgAAgBwgtAEAAOQAoQ0AACAHCG0AAAA50JbTWJnZIUn/1OCnPV3SDxr8nHnDPmAfSOwDiX0QYT+wDyT2gbS0ffBWd19TbUVbhrZmMLOxWnOFtQv2AftAYh9I7IMI+4F9ILEPpMbtAy6PAgAA5AChDQAAIAcIbY1zR9oVyAD2AftAYh9I7IMI+4F9ILEPpAbtA9q0AQAA5ABn2gAAAHKA0LYEZvZBMztgZmUzq9kLxMwuM7MXzGzCzHbEys82s2+b2biZFc2sM5maN46ZrTKzB8LX8ICZrayyzXvM7KnY7SdmVgjXfd3Mvhdbd17yr+Lk1LMPwu1+Gnude2Ll7XIcnGdm3wo/M8+Y2UBsXW6Pg1qf79j65eH7OhG+z2fF1n0qLH/BzN6XZL0bqY59cKOZPRe+7w+Z2Vtj66p+LvKmjn3wUTM7FHutH4utuyb87Iyb2TXJ1rxx6tgHg7HX/x0z+2FsXascB3ea2atmtr/GejOzr4T76BkzuyC2bunHgbtzq/Mm6R2S3i7pUUm9NbY5RdJ3Jb1NUqekpyWdE667V9LW8P6fSfp42q/pBPbBFyTtCO/vkHTbItuvknRE0hvD5a9LujLt15HEPpB0tEZ5WxwHkn5BUk94/y2SXpF0Wp6Pg4U+37FtflvSn4X3t0oqhvfPCbdfLuns8HlOSfs1NWkfvCf2mf94tA/C5aqfizzd6twHH5X0J1Ueu0rSi+HfleH9lWm/pmbsg4rtf0fSna10HISv49ckXSBpf431/ZK+KckkvUvSt0/mOOBM2xK4+/Pu/sIim10oacLdX3T3GUn3SNpsZibpYkn3hdvdJanQvNo2zWYFdZfqew1XSvqmu/+4qbVK1lL3wTHtdBy4+3fcfTy8PynpVUlVB4zMkaqf74pt4vvmPkm/Hr7vmyXd4+7T7v49SRPh8+XNovvA3R+JfeYfk3RGwnVstnqOg1reJ+kBdz/i7q9JekDSZU2qZzMtdR9cJenuRGqWIHf/BwUnJmrZLOmvPPCYpNPMbK1O8DggtDXeOkkvx5YPhmWrJf3Q3V+vKM+bN7v7K5IU/n3TIttv1fwP6ufD08SDZra8GZVssnr3wRvMbMzMHosuD6tNjwMzu1DBr/HvxorzeBzU+nxX3SZ8n3+k4H2v57F5sNTXsU3BmYZItc9F3tS7Dz4QHuP3mdmZS3xs1tX9OsLL42dLejhW3ArHQT1q7acTOg5ObWjVWoCZPSjp56qs+rS731/PU1Qp8wXKM2ehfbDE51kr6Z2S9saKPyXp/yn4Ar9D0s2SbjmxmjZPg/bBenefNLO3SXrYzJ6V9C9VtmuH4+C/SbrG3cthcS6Ogyrq+Rzn/v+ARdT9OszsI5J6JfXFiud9Ltz9u9Uen2H17IO/kXS3u0+b2XUKzr5eXOdj82Apr2OrpPvc/aexslY4DurR0P8PCG0V3P29J/kUByWdGVs+Q9KkgjnHTjOzU8Nf31F55iy0D8zs+2a21t1fCb+MX13gqT4kabe7z8ae+5Xw7rSZ/aWk32tIpRusEfsgvCQod3/RzB6VdL6kb6iNjgMz+xlJfyvpD8JLA9Fz5+I4qKLW57vaNgfN7FRJP6vg8kk9j82Dul6Hmb1XQcDvc/fpqLzG5yJvX9aL7gN3Pxxb/AtJt8Uee1HFYx9teA2bbynH81ZJn4gXtMhxUI9a++mEjgMujzbe45J6LOgh2KngYN3jQcvDRxS08ZKkayTVc+Yua/YoqLu0+GuY14Yh/IKP2nYVJFXtcZNxi+4DM1sZXfIzs9MlvVvSc+10HITH/24F7Tn+umJdXo+Dqp/vim3i++ZKSQ+H7/seSVst6F16tqQeSf+YUL0badF9YGbnS/pzSZe7+6ux8qqfi8Rq3jj17IO1scXLJT0f3t8r6dJwX6yUdKnmXo3Ii3o+CzKztytoaP+tWFmrHAf12CPpN8JepO+S9KPwR+uJHQdp97zI003SFQrS8bSk70vaG5a/RdJobLt+Sd9R8Kvh07Hytyn4T3pC0l9LWp72azqBfbBa0kOSxsO/q8LyXkk7Y9udJemfJXVUPP5hSc8q+JL+75JWpP2amrEPJP1q+DqfDv9ua7fjQNJHJM1Keip2Oy/vx0G1z7eCS7uXh/ffEL6vE+H7/LbYYz8dPu4FSe9P+7U0cR88GP4fGb3ve8Lymp+LvN3q2Ad/LOlA+FofkfSLscf+Vnh8TEj6zbRfS7P2Qbj8WUm3VjyulY6DuxX0jJ9VkA+2SbpO0nXhepN0e7iPnlVs5IkTOQ6YEQEAACAHuDwKAACQA4Q2AACAHCC0AQAA5AChDQAAIAcIbQAAADlAaAMAAMgBQhsAAEAOENoAYAnM7BEzuyS8/0dm9pW06wSgPTD3KAAszWck3WJmb1IwX+LlKdcHQJtgRgQAWCIz+3tJKyRd5O4lM3ubgimqftbdr1z40QBwYrg8CgBLYGbvlLRW0rS7lyTJ3V90923p1gxAqyO0AUCdzGytpF2SNkuaMrP3pVwlAG2E0AYAdTCzN0oalnSTuz8v6XOSPptqpQC0Fdq0AcBJMrPVkj4v6RJJO939j1OuEoAWRGgDAADIAS6PAgAA5AChDQAAIAcIbQAAADlAaAMAAMgBQhsAAEAOENoAAABygNAGAACQA4Q2AACAHCC0AQAA5MD/B4P4jzzet+KNAAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "fig = plot_data_for_classification(X2, Y2, xlabel=r'$x_1$', ylabel=r'$x_2$')" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "def safeSigmoid(x, eps=0):\n", - " \"\"\"Funkcja sigmoidalna zmodyfikowana w taki sposób, \n", - " żeby wartości zawsze były odległe od asymptot o co najmniej eps\n", - " \"\"\"\n", - " y = 1.0/(1.0 + np.exp(-x))\n", - " if eps > 0:\n", - " y[y < eps] = eps\n", - " y[y > 1 - eps] = 1 - eps\n", - " return y\n", - "\n", - "def h(theta, X, eps=0.0):\n", - " \"\"\"Funkcja hipotezy (regresja logistyczna)\"\"\"\n", - " return safeSigmoid(X*theta, eps)\n", - "\n", - "def J(h,theta,X,y, lamb=0):\n", - " \"\"\"Funkcja kosztu dla regresji logistycznej\"\"\"\n", - " m = len(y)\n", - " f = h(theta, X, eps=10**-7)\n", - " j = -np.sum(np.multiply(y, np.log(f)) + \n", - " np.multiply(1 - y, np.log(1 - f)), axis=0)/m\n", - " if lamb > 0:\n", - " j += lamb/(2*m) * np.sum(np.power(theta[1:],2))\n", - " return j\n", - "\n", - "def dJ(h,theta,X,y,lamb=0):\n", - " \"\"\"Gradient funkcji kosztu\"\"\"\n", - " g = 1.0/y.shape[0]*(X.T*(h(theta,X)-y))\n", - " if lamb > 0:\n", - " g[1:] += lamb/float(y.shape[0]) * theta[1:] \n", - " return g\n", - "\n", - "def classifyBi(theta, X):\n", - " \"\"\"Funkcja predykcji - klasyfikacja dwuklasowa\"\"\"\n", - " prob = h(theta, X)\n", - " return prob" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "def GD(h, fJ, fdJ, theta, X, y, alpha=0.01, eps=10**-3, maxSteps=10000):\n", - " \"\"\"Metoda gradientu prostego dla regresji logistycznej\"\"\"\n", - " errorCurr = fJ(h, theta, X, y)\n", - " errors = [[errorCurr, theta]]\n", - " while True:\n", - " # oblicz nowe theta\n", - " theta = theta - alpha * fdJ(h, theta, X, y)\n", - " # raportuj poziom błędu\n", - " errorCurr, errorPrev = fJ(h, theta, X, y), errorCurr\n", - " # kryteria stopu\n", - " if abs(errorPrev - errorCurr) <= eps:\n", - " break\n", - " if len(errors) > maxSteps:\n", - " break\n", - " errors.append([errorCurr, theta]) \n", - " return theta, errors" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "theta = [[ 1.37136167]\n", - " [ 0.90128948]\n", - " [ 0.54708112]\n", - " [-5.9929264 ]\n", - " [ 2.64435168]\n", - " [-4.27978238]]\n" - ] - } - ], - "source": [ - "# Uruchomienie metody gradientu prostego dla regresji logistycznej\n", - "theta_start = np.matrix(np.zeros(X2.shape[1])).reshape(X2.shape[1],1)\n", - "theta, errors = GD(h, J, dJ, theta_start, X2, Y2, \n", - " alpha=0.1, eps=10**-7, maxSteps=10000)\n", - "print('theta = {}'.format(theta))" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "def plot_decision_boundary(fig, theta, X):\n", - " \"\"\"Wykres granicy klas\"\"\"\n", - " ax = fig.axes[0]\n", - " xx, yy = np.meshgrid(np.arange(-1.0, 1.0, 0.02),\n", - " np.arange(-1.0, 1.0, 0.02))\n", - " l = len(xx.ravel())\n", - " C = powerme(xx.reshape(l, 1), yy.reshape(l, 1), n)\n", - " z = classifyBi(theta, C).reshape(int(np.sqrt(l)), int(np.sqrt(l)))\n", - "\n", - " plt.contour(xx, yy, z, levels=[0.5], lw=3);" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [], - "source": [ - "Y_expected = Y2.astype(int)\n", - "Y_predicted = (classifyBi(theta, X2) > 0.5).astype(int)" - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "# Przygotowanie interaktywnego wykresu\n", - "\n", - "dropdown_highlight = widgets.Dropdown(options=['all', 'tp', 'fp', 'tn', 'fn'], value='all', description='highlight')\n", - "\n", - "def interactive_classification(highlight):\n", - " fig = plot_data_for_classification(X2, Y2, xlabel=r'$x_1$', ylabel=r'$x_2$',\n", - " Y_predicted=Y_predicted, highlight=highlight)\n", - " plot_decision_boundary(fig, theta, X2)" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/home/pawel/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:10: UserWarning: The following kwargs were not used by contour: 'lw'\n", - " # Remove the CWD from sys.path while we load stuff.\n" - ] - }, - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmwAAAFmCAYAAADQ5sbeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3iV5f3H8fedkARIwkgIKxBWwgYRoqCCoODCFVAMiuun1Wq1ZWgLdc+KODDaalXqaKUaqwFpBReKypCNbEjYECBAgAwg69y/P06Ch5DISvKc8Xld17nOOc/znOR7IOOTexprLSIiIiLivYKcLkBEREREfp0Cm4iIiIiXU2ATERER8XIKbCIiIiJeToFNRERExMspsImIiIh4uVpOF+CERo0a2datWztdhkjVOHAANmyAxo2hZctfjm/bBllZ0K4dNGjgXH0iInLSFi9evNdaG1P+eEAGttatW7No0SKnyxCpGtbC6NGQkgI33ggTJ/7yfORI93NjnK5SREROgjFmS0XHAzKwifgVY9yhDNwhLSXF/VhhTUTEb5hA3OkgMTHRqoVN/I61EOQxLNXlUlgTEfExxpjF1trE8sc16UDEH5R1i3oaPdp9XEREfJ4Cm4iv8xzDNnKku2Vt5Ej3c4U2ERG/oDFsIr5u6tTjJxh4jmnr3x+GDHG2RhEROSMKbCK+LikJ0tLc92Vj1spCW//+7uMiIuLTFNhEfJ0xFbegVXZcRER8jsawiYiIiHg5BTYRERERL6fAJiIiIuLlFNhEREREvJwCm4iIiIiXU2ATERER8XIKbCIiIiJeToFNRERExMspsImIiIh4Oe10ICIicpJyC3JJXZVK+r50EqITSO6STGRYpNNlSQBQYBMRETkJs7fOZvDkwbisi/yifMJDwhnz5Rimj5hO37i+Tpcnfk5doiIiIieQW5DL4MmDyS3MJb8oH4D8onxyC93H8wrzHK5Q/J0Cm4iIyAmkrkrFZV0VnnNZF6krU2u4Igk0CmwiIiInkL4v/WjLWnn5RflkZGfUcEUSaBTYRERETiAhKp5wE1bhuXATRnxUuxquSAKNApuIiMgJJG8KJ+hIQYXngo4UkLyxbg1XJIFGgU1EROQEIq+7kekF1xNZAOHWvcBCuK1FZAFML7ieiOtudLhC8Xda1kNEROREjKHvix+TOfo+Ur94g4woiM8uJvnCe4mY+DcwxukKxc8Za63TNdS4xMREu2jRIqfLEBERX2MtBHl0TrlcCmtSpYwxi621ieWPq0tURETkZFgLo0cfe2z0aPdxkWqmwCYiInIiZWEtJQVGjnS3rI0c6X6u0CY1wCvGsBljLgdSgGBgkrV2fLnzE4GLSp/WBRpbaxuUnisBVpSe22qtvaZmqhYRkYAxdeovYW3iRHc36MSJ7nMpKdC/PwwZ4myN4tccH8NmjAkG1gOXANuBhcCN1trVlVz/e+Bsa+0dpc/zrLURp/I5NYZNREROibXu0JaUdOyYtcqOi5wmbx7Ddi6QYa3daK0tBD4Crv2V628EPqyRykRERMAdxoYMOT6UVXZcpIp5Q2CLBbZ5PN9eeuw4xphWQBvgW4/DtY0xi4wxPxljkqqvTBERERFneENgq+jPksr6aYcDn1hrSzyOxZU2Hd4EvGKMqXB/EGPM3aXBbtGePXvOrGIRCVzWwpQpxw8yr+y4iEgV8IbAth1o6fG8BZBZybXDKdcdaq3NLL3fCMwCzq7ohdbat6y1idbaxJiYmDOtWUQC1dSpMHTosTMDy2YQDh3qPi8iUsW8IbAtBBKMMW2MMaG4Q9m08hcZYzoADYF5HscaGuPejdcY0wi4AKhwsoKISJVISjp+OQfP5R6SNDJDRKqe48t6WGuLjTH3A1/iXtbjHWvtKmPMU8Aia21ZeLsR+MgeO621E/CmMcaFO3yOr2x2qYhIlSi/nENKivux53IPIiJVzPFlPZygZT1E5IxpiyIRqQbevKyHiIhv0RZFIlLDFNhERE6FtigSEQc4PoZNRMSnaIsiEXGAApuIyKlISoK0tGO3IioLbf37a5aoiFQLBTYRkVNRthXRyR4XEakCGsMmIiIi4uUU2ERERES8nAKbiIiIiJdTYBMRERHxcgpsIiIiIl5OgU1ERETEyymwiYiIiHg5BTYRERERL6fAJiIiIuLlFNhEREREvJwCm4iIiIiX016iIiJCSUkJh3IOk3cgn/yDhziSd4TDeUc4kl/A4dLHhYcLKTxSROGRX+6LCoopKSmhpLiEkmIXJcUluIpLcLlshZ8nKDiIWiHBBNcqvQUHUSu0FqG1QwmtHUJo7VBCaocQVieU2uFh1I2sQ53IOtSNrE3tiNqE16tLZFQEdevVIShIbQ4SOBTYRET8jLWW3P157N99kAO7D5K96wAH9+RwcG8OB/fmkrOv9H5vLnkH8sk7kM+hnMMn/fGDgow7YNUJpVZoLY8AFkRwrWCCgoMICj4+TFlrcZW4jgl2JcUuigqKKCwoovBwIcVFJSddQ0TDCCKj3LcGMfXctyYNaNikPg0a1yeqaQMaxUYRHRtFnfDaJ/3+RLyRApuIiA8pKixi7/ZssrbuZc/2fezdkc2+Hdnszcx2P87M5sDugxQVFh/3WmMMkVER1IuOoF6jejRu1Yh2PVoT0SD86C28QV3q1qtL3cja1ImoTe3wsvswwuqGEVo7hOBawRhjquX9lZSUUFRQTOHhwqMte4dyD3Ok9D7vwCHy9ueRtz+fnOw8cvfnkbMvlz3b95G+ZCMHsnIoKT4+9EU2DKdRi2gaxUbRuGUjmrZpfPTWpHVjGsTUq7b3JFIVFNhERLxISXEJWdv2snNjFrs27mbnxt3s2pxF1ta97N6yh+ydB7D22O7GuvXq0Cg2ikaxUbS8uCtRTRoQ1bQhDZq4W5kaNqlP/Zh6REZFEBwc7NA7OznBwcEE1w2mdt0w6kVHnvLrXS4Xefvz2Z91kP27DrB3RzZ7t+87Gm73bN/H+kUbOLg395jX1Q4PIzahGS3aN6NFQnNi2zejRfvmxHVsTnj98Kp6eyKnzZT/xg8EiYmJdtGiRU6XISIBylpL9q4DbF+Xyfb1mWxbl8mO9J1sW5fJrk1Zx7QQBdcKpkmrRjRuFUPjuEY0iXPfN45rRExLd4tRnYg6Dr4b33Q47zC7Nu9h16Ysdm3KYufG3ezI2Mn29TvZtSkLV4nr6LVNWsXQpnscbbu1ok23ONp0b0WL9s28PvyKbzLGLLbWJh53XIFNRKR6WGvJ2rqXLau3s2X1drau3saWNe7HnmPGQmuH0KK9u1Untl1TmrVrSrO2jWnerimNWkQpGNSwosIidm3KYtu6TLas2s6mlVvYtHwrW9fuOBrkatcNI75nG9r3akf7xHa0T2xLbEIzTYSQM6bA5kGBTUSqWt6BfDb8vJlNK7ayecVWNq3cyuaV2ziU+0swa9ikPq06tyCuUwtadoylZYfmtOzQnEYtovWL3gcUFhSxdc12Ni3fSvqSjaxfvIGMJZsoOFwIuLumO/ZOoHOf9nQ+vwOdz2tPeL26DlctvkaBzYMCm4iciYN7c0hfsomMJRtJX7qJ9MUb2blx99HzkQ3Dad0tjjZd42jTrRWtu7hD2umMyRLvVlJcwta1O1i/aAPrFmSw+qf1bFq+BZfLEhRkaNejNd37d6HnoO50u7CTZqvKCSmweVBgE5GTdSj3MOmLN7JuYQbrSn8p796y5+j5Zm2bEN+zDQlnt6Vdj9a07R5HdPMozTgMYIdyD7N2fjorflzDih/XsHreeooKiqgVEkzn8zvQc2B3zh7UjQ7ntFN3txxHgc2DApuIVMTlcrFt7Q5Wz1vP6nnrWTs/nS2rtx+dldm0TWM6nNOODonxJPRyB7TIhhEOVy3e7sihAlbNWcuSb1awdOZyMpZuxlpLZFQE51zeg96De5J4eQ/qRakFVhTYjqHAJiLgHpO0fmEGy39Yw8o5a1kzbz15B/IBiIyKoFOfBDqem0CHc+LpcE476jeq53DF4g8O7s1h6cwVLJixlIUzlnJgTw5BwUF079+ZvkN60++63kQ1beh0meIQBTYPCmwigenIoQJWz13H8h9Ws+LHNaz5KZ2igiIAWnVuQZfzOxwdLN6ifXN1a0q1c7lcrFu4gbmfLWTO1AVsW7uDoCBD9wFdGHDD+fS7ro/GPgYYBTYPCmwigaHwSCGr561n2Xcr+XnWKtbOT6e4qISgIEN8z7Z069eJbv060bVvR7WeiVfYvGobs1LnMCt1LjvSdxJcK5ieg7oxcMSF9B16LmF1wpwuUaqZVwc2Y8zlQAoQDEyy1o4vd/524AVgR+mhv1prJ5Weuw14pPT4M9ba90/0+RTYRPyTtZbNq7ax6MufWfz1z6z4YTWFR4oICjIk9GrLWQO60uOiLnQ+v4OWWxCvZq1lw7LNR8Pb7i17iGgQzqCbL2TwXQNp062V0yVKNfHawGaMCQbWA5cA24GFwI3W2tUe19wOJFpr7y/32ihgEZAIWGAx0Mtau//XPqcCm4j/yNmXy6KvfmbRV8tY/NVysne6v/1bdW5Bz0HdOXtgN7pf2EnbC4nPcrlcrPhhDdMnfcOPn/xEUWExnc9rz+C7BnHR8AsIrR3qdIlShbw5sJ0HPGGtvaz0+Z8BrLXPeVxzOxUHthuBAdba35Y+fxOYZa398Nc+pwKbiO+y1pKxdBPzP1/Cwi+WsnZ+Oi6Xe8Zdz0HdSLy0B70uPYuYFtFOlypS5Q7uzeGbf/3A529/w7a1O6jfKJLBdw3imvsup1HzKKfLkypQWWDzhs3fY4FtHs+3A70ruO46Y8yFuFvjRltrt1Xy2tiKPokx5m7gboC4uLgqKFtEakrhkUKWzlzB3M8WMn/6EvZl7scYQ4dz2jHikes554qzaZ/YVmtaid+r36ge142+iqGjrmTZdyuZ+toMPho/lf+8OI1Bt/Tnhj9eQ8sOFf4aFB/nDYGtomlY5Zv9/gt8aK0tMMbcA7wPXHySr3UftPYt4C1wt7Cdfrki1SO3IJfUVamk70snITqB5C7JRIYF7uyw3P15zP98CXM/W8DCL5ZxJL+AOhG1Sby8B+ddlUji5T1o2Li+02WKOMIYw9kXd+Psi7uxc+NuPnn5v3zxzrd8+e539Lu+Dzc/ej1tuqpxwp/4RJdoueuDgWxrbX11iYq/mL11NoMnD8ZlXeQX5RMeEk6QCWL6iOn0jevrdHk1Zv/uA8yesoDZaT+x7LtVuEpcRDVryHlXJ3JB0jmcdVFXQsNCnC5TxCvtzzrI1FenM/W1GRzOO8KFw/pwy2PDaNW5pdOlySnw5jFstXB3cw7EPQt0IXCTtXaVxzXNrLU7Sx8PAcZaa/uUTjpYDPQsvXQJ7kkH2b/2ORXYxJvkFuQS+3IsuYW5x52LDI0k84FMIkL9dzX9/VkHmZ02nx/+M5fl36/G5bK0aN+MvkN6c37SuXQ4p502Rhc5BTnZuXz68v+Y8up0Cg4VcNn/XcytT96gMW4+wmvHsFlri40x9wNf4l7W4x1r7SpjzFPAImvtNOAPxphrgGIgG7i99LXZxpincYc8gKdOFNZEvE3qqlRc1lXhOZd1kboylTt73lnDVVWvvAP5zE6bz3epc1g2cwUul6Vlx1huevg6Lhx2Hq27tNSitSKnqV5UJP/3zI1cN/oqJj/zKdNe/4LvPpzN8HFDGPbg1ZpV6qMcb2FzglrYxJuM/XosE+ZOqPT8uAvG8dygCkcI+JSCwwX89N/FfPvhbBbOWEpRYTHN2jbhouEXMCD5fFp3jVNIE6kGOzfu5u2x/+LHT+fTtE1j7nnpNi5IOtfpsqQSXtvCJhLoEqITCA8JJ78o/7hz4SHhxEfFO1BV1bDWsnreer567ztmfTyXQzmHiWrWkKvvvYyLbryADufEK6SJVLNmbZvw2H8eZOm3K3h95Ls8MfQFLhhyLr//62+IbqY9S32FWthEHOaPY9iytu7h63/9wNf//J4d6TupXTeMfsP6cMkt/enev7OW3xBxSElxCZ+8/D/++UQqIWEh3PPy7Vx2+wD94eRFvHbSgRMU2MTb+NQsUWth6lRISgKPH/JFBUXMffhvzFiRx5JvlmOt5awBXbjk1v70u64PdSPrOFi0iHjavj6Tl+/+Oyt+WMN51yQy+q17tEyOl1Bg86DAJt4orzCP1JWpZGRnEB8VT3LXZO9sWZsyBYYOhZEjYeJEdmzYxfS3vuGrv/2PA4ddNG5Ul8t+dyWX3NafZm2aOF2tiFTC5XIx9dUZTPrzZOpFR/DIR6Pp2reT02UFPAU2DwpsImfAWkpGjmLea5/w37iLWLKtgCAD57l2cOW13en5yd8IrqUuTxFfsXH5Fp4a9hI7N+7mN+Nv5voxV6mL1EGadCAiZ2z/7gNMf3smn08tYI85n5it+7mNTVxuN9Fo5G9h4sRjuklFxPu17d6Kvy14jhfvfIO3/vhPVs1dyx/fvY/wenWdLk08qIVNRE4ofclG0lI+5/vUuRQVFtPr0rO45p5L6T2kD8Flu8G5XAprIj7MWkvaK5/z9tgPaNWlBc/NeJiopppFWtMqa2HT8uHeyFr3OKHyYbqy4yLVwOVyMXfaQh646HF+lziWOVMWcOXdl/DOmlcYP+Nhzp/171/CGsDo0fraFPFhxhiuG30VT08bS2b6Lkb1fZTMDbucLktKKbB5o6lT3YO6PX8BWut+PnSo+7xINTlyqIBpr3/JHZ1G8XjSBHZtyuK3L97Kh9v+zn2v3kHL9s3dX4spKe6JBy6X+z4lRaFNxA+cc/nZTJj5OPkHDzGq7yNkLNvkdEmCukS9U1k4K/uFOHHi8c/V9SRVLHd/HtP+9iVTXv2cg3tz6XhuPNePuZq+Q3sfO4mg3CxRjDn2azYtDYYMce6NiEiV2Lp2B+Mue5ojeUd48bsnadu9ldMlBQTNEvXg9YENjv0FWEZhTarB/t0H+HTi//jvG19xKPcwva/sSfKfkujat2PFM8UqWYet0uMi4rN2btrNmAsfo7iohJdmPUlcx1inS/J7CmwefCKwgfsXYJBHr7UGdUsVytq2l48nfMaMf8ykqKCY/jecR/LYJOJ7tHG6NBHxItvW7WBM/8epFRLMK7OfoUmrGKdL8muadOBrylrYPGl8kFSBvZnZvHb/JG5P+D2fv/U1F9/Uj3fWvMLDH45WWBOR47TsEMvzXz3K4bwjPD5kAkcOFThdUkBSYPNG5cewaVC3VIH9WQf5+wPvc1v8/Xz+1jdccmt/3k9/jQcm3UuL9s2dLk9EvFjb7q146N+j2PjzFlLufYtA7J1zmhbO9UZTpx4/wWDiRPe5lBTo31+DuuWk5R3IJ3XCZ0x9bTqFhwsZeMuF3PLoMJq11bZRInLyzr3ibG594gbefzyVDufEk3T/FU6XFFA0hs0baVC3VIHCgiL+98ZXfPDMJ+Ttz6d/8vnc+vgwWnbQoGEROT0ul4vHkyaw+KufeX3xBFp3ael0SX5HY9h8iTHuFrTyoayy4yIerLXMSp3DnZ1H8caY90jo1ZbXFz/Pw5NH0nL1Ai3ILCKnLSgoiDGT7qVuvTq8eMffKCkucbqkgKHAJuJHVs5ewx/Oe4hnb3yFOhG1+cuMh3n+y0dp0qkRk/5xH2NfH8qksZeQeyTH/QItyCxAbkEuk5ZMYuzXY5m0ZBK5BblOlyRerGHj+vz+r79h3cIN/OfFaU6XEzDUJSriB/ZmZjNp7AfMnPwj0c0bcvvTN3LJrRcSHBzM7K2zGTx5MC7rIr8on/ACCAoNYfr/zaTvxE+1IHOAO+7rIyScIBPE9BHT6RvX1+nyap6GpJwUay1PDXuJhTOW8t76V2kUG+10SX5DXaIifqiosIiPX/iMOzqO5If/zOOmh4by7rpXufz/LiI4OJjcglwGTx5MbmEu+UX5AOSHQa4pYvCbF5L3hsJaIKvw66Mon9xC9/G8wjyHK3SAtgY8KcYY7n7hFlwlLt5//GOnywkICmwiPmrJN8v57VkP8vbYDzhrQBfeXvky//fMjdQJr330mtRVqbisq8LXu4DULiisBbBf/fqwLlJXptZwRV4gKen4ZZQ8l1lKSnK6Qq/RrE0TrvztJXz1/iy2p+90uhy/p8Am4mMO7s1h/K2vMvbSpykpLuGZ/47j6WnjiI1vdty16fvSj7aclJcfBhlRaG2/AParXx9F+WRkZ9RwRV6gbBmlstAWFKRhA7/ipoeGUiskmE80lq3aKbCJ+AhrLd988AN3dBrF96lzGfHIdby94mV6X9mr0tckRCcQHhJe4bnwkHDiew7UgswB7IRfH1HxNVyRl/Bc+7KMwlqFGjZpwEXD+zLz3z+Sf7Di8C9VQ4FNxAfs3rKHhwY/y/O3vkZsQlPeWPICtz81nNDaob/6uuQuyQSZir/Ng0wQyU9P+aUlQWNzAs4Jvz66JtdwRV5CWwOekqt/dxlH8gv4+l8/OF2KX1NgE/Fi1lqmv/0Nd3Ubw6o567jv1TuY+OPTJ71YZWRYJNNHTCcyNPJoS0p4SDiRoe7jEWGR7paDtDSNzQlAJ/z6CI1wuEIHaGvAU9YhsR0JPdsw8wMFtuqkZT1EvNS+nft5+a43WDB9KT0u7sqD//gdTVrFnNbHyivMI3VlKhnZGcRHxZPcNTkwfxlLhfT14WHKFPdsUM8xa54hLi1NWwNW4N9/SePdRz7kox1vEd2sodPl+LTKlvVQYBPxQj9++hOv3PMWBYcK+M3zN3PN7y4jKEgN4iLVTuuwnZZNK7dyd/cHGPX3u7ny7kucLsenaR02CQyVbbPkI9svHTlUwMS7/85Tw16iWdvGvLFkAkn3X6GwJlJTtDXgaWndpSWN4xqxZOYKp0vxW17xW8AYc7kxZp0xJsMYM66C82OMMauNMcuNMTONMa08zpUYY5aV3jSvOND58KKXG5dv4b5zxjLjH98yfGwSr8x+Rhu1i4hPMMbQsXcC6Ys3Ol2K33I8sBljgoG/AVcAnYEbjTGdy122FEi01nYHPgEmeJw7bK3tUXq7pkaKFu/lg4telk0s+H2fP5O3P5/xXz7Cnc+NoFZILadLExE5ae17tWPnxt3kZGsv2urgDb8RzgUyrLUbAYwxHwHXAqvLLrDWfudx/U/AzTVaofgOz/WTUlLcN/DaRS8LjxTy2v3/4It3vqXnJd0Z968/0LBxfafLEhE5ZW3Pcnd+bVm1nW79Ojlcjf9xvIUNiAW2eTzfXnqsMncCMzye1zbGLDLG/GSM8b7mE6l5PrLoZdbWPYy+8DG+eOdbRjx8HX+Z/pDCmoj4rJgW7g3g9+7IdrgS/+QNLWwV/RatcGS4MeZmIBHo73E4zlqbaYxpC3xrjFlhrd1QwWvvBu4GiIuLO/OqxXtVtuilF4W2n2et4ukbXqKooJgn0v7IBUnnOl2SiMgZiW7uXs5Dga16eEML23bAcxXQFkBm+YuMMYOAh4FrrLUFZcettZml9xuBWcDZFX0Sa+1b1tpEa21iTMzprWUlPsAHFr388r3vGHvp09RrVI/X5j+nsCYifiGiQThBwUHkagxbtfCGwLYQSDDGtDHGhALDgWNmexpjzgbexB3WsjyONzTGhJU+bgRcgMfYN7/l40tXVKupU4/fqNlzI2cHZ4laa3nv0Y948Y7XOWtAZ16b9yxxHTULVET8gzGGoCCDdQXw76Bq5Hhgs9YWA/cDXwJrgI+ttauMMU8ZY8pmfb4ARAD/Kbd8RydgkTHmZ+A7YLy11v8Dmw8vXVHtkpLcK5F7dn+WhTYHt18qKizi+dteY/Kzn3L5HRfz7OcPEV6/4k23RUR8lQkKwhXoga2aGlW8YQwb1trpwPRyxx7zeDyoktfNBbpVb3VeyHPpCnCHES9fuqLGlC1uebLHa8Dh/CM8df2LLPryZ25/ejg3PTQU4yVj6UREqorL5aKkqJjgWo63BTmrrFGlirc384rAJqfIx5auCGQ52bk8evV41s5PZ8zb93DFnQOdLklEpFrk7c/H5bI0CPTZ7tXUqKLA5qvKQlvZFwQorHmZ/bsPMPbSp9m+LpNHPn6AfkN7O12SnKbcglxSV6WSvi+dhOgEkrskExkW6XRZIl5lf9ZBAAW2ampUCfB2Sx9W2dIVgTzhwIsc2HOQPw16ip0bdvPM//6ssObDZm+dTezLsYz6YhQT5k5g1BejiH05ltlbZztdmohXydqyB/hlPbaAVg3rgSqw+SIfWLoikOVk5zL20qfJ3LCLp6aNpeeg7k6XJKcptyCXwZMHk1uYS35RPgD5RfnkFrqP5xXmOVyhiPfYtGIrAK27tjzBlQGgGhpVFNh8kRcvXRHo8g/mM+7Sp9m2ZgdPTh3L2RcH3pwYf5K6KhWXdVV4zmVdpK5MreGKRLzXppVbaRQbRWTDCKdLcVY1NapoDJsvKlu6Iinp+KUr+vcP7FmiDio8UshjSRPYuHwrT075I4mXnuV0SXKG0velH21ZKy+/KJ+M7IwarkjEe635KZ34nm2cLsN5lTWqgPt4//6nNUtULWy+qGyJivJ94ZUdl2pXUlLC+FteZfn3q/nTe/fR+8peTpckVSAhOoHwkIrXywsPCSc+Kr6GKxLxTllb97AjfSdnX6RehepaD1SBTaQK/H3M+/z46Xzueek2Lr6pn9PlSBVJ7pJMkKn4x2SQCSK5a3INVyTinZbMXAnA2QO7OlyJF6imRhUFNpEzNH3STKa+NoPrRl3JdaOvcrocqUKRYZFMHzGdyNDIoy1t4SHhRIa6j0eEBvhYHZFScz9bQHTzhrTuGud0KX5LY9hEzsDqeev46/2T6HXpWdz1wi1OlyPVoG9cXzIfyCR1ZSoZ2RnER8WT3DVZYU2kVO7+PBbOWMq1912uXVyqkQKbyGnat3M/T17/Eo1aRPPQv0cSHBzsdElSTSJCI7iz551OlyHileZMWUBxUQkX3djX6VL8mgKbyGlwuVw8f+tr5B/IZ/wXD1MvSqvei0hg+vztb2jZoTntE9s5XYpf0xg2kdPw8YTPWDpzBb9LuYM23Vo5XY6IiCPWzE9n7fx0rlF3aLVTYBM5RWsXpPPuox9x4bDzuOLOi50uR0TEMVNfm07dyDpcetsAp0vxewpsIqegsKCIF+94nUBtRLQAACAASURBVOjmDRn95m/1F6WIBKxt63Yw66M5XPGbgdSNrON0OX5PgU2qhrUwZcrxW25UdtxHfTzhM7as3s7I1+8iokHFC6qKiASC9x5LJbROKMljtbtOTVBgk6oxdSoMHXrsPmll+6kNHeoX+5tuX5/Jv5/9lAHJ52snAxEJaOsXb+CH/8zjutFX0bBxfafLCQgKbFI1kpKO39zWc/NbP9jf9K0//YuQsBDunXi706WIiDjG5XLxtz+8Q4OYegx74Gqny6leXtR7pMAmVaNsn7Sy0BYUdPzmtz7s5+9XMW/aIoaPG0JU04ZOlyMi4piv3pvF6nnruWvCLYTX9/OhIV7Ue6TAJlWnLLR58oOwZq3l7T/9i5gW0QwdNdjpckREHJOzL5dJ4z6gywUdGHTLhU6XU/28qPdIgU2qTtkXsifPv0p81IIZS1m3cAO3PnEDYXXCnC5HRMQxr90/ifyDh/jD3+4iKCgAIoQX9R4FwL+21Ijyf3W4XMf/VeKjPho/hZiW0Qy8uZ/TpYiIOGZW6hxmpc7l5seG0bZ7AC0Y7iW9RwpsUjWmTj3+rw7Pv0p8dJboyjlrWTl7LcMeuIaQ0BCnyxERccTezGxevW8SHc+NZ3igLePhJb1HCmxSNZKSIC3t2L86ykJbWprPzhL939+/Irx+XS7XjgYiEqCKi4p5Jvllio4U8cf37ie4VrDTJdUcL+o90ubvUjWMgSFDTv64D8jdn8cPn/zEFXdeTJ3w2k6XIyLiiDcf/Cer5qzjoX+PIq5jrNPl1KzKeo/Afbx//xr7HafAJlKJb/89m6KCIq74zUCnSxERccTMyT8y9bUZDB15JRcNv8DpcmpeWe9RUtLxvUf9+9do75ECm0glZqf9ROsuLYnv0cbpUkREatzKOWt56c7X6XZhJ+6acLPT5TjDi3qPNIZNpAL5B/NZ8eNa+lylLahEJPBkbtjFE0Mm0LhVDE98+kdqhah9x2leEdiMMZcbY9YZYzKMMeMqOB9mjEktPT/fGNPa49yfS4+vM8ZcVpN1i/9a9NVySopL6K3AJiIB5uDeHB656jlcLssz//sz9aIjnS5J8ILAZowJBv4GXAF0Bm40xnQud9mdwH5rbTwwEXi+9LWdgeFAF+By4PXSjydyRlb+uIba4WF06p3gdCkiIjUmP+cQDw3+C7u37OHJKX+iRUIzp0uSUt7QxnkukGGt3QhgjPkIuBZY7XHNtcATpY8/Af5qjDGlxz+y1hYAm4wxGaUfb14N1S5+at2iDBJ6tQ2s6esiZyC3IJfUVamk70snITqB5C7JRIapZcaXHM4/wiNXPceGZZt5Iu2PdOvXyemSxIM3BLZYYJvH8+1A78qusdYWG2MOAtGlx38q99oAm3MsVa24qJiMpZu59r7LnS5FxCfM3jqbwZMH47Iu8ovyCQ8JZ8yXY5g+Yjp94/o6XZ6chCOHCnhiyARWz3Uv36Hxu97H8S5RoKK9HcqvRFfZNSfzWvcHMOZuY8wiY8yiPXv2nGKJEkh2bd5DUUERbbrFOV2KiNfLLchl8OTB5Bbmkl+UD0B+UT65he7jeYV5DlcoJ3I47zCPXPUcS2eu5IF//I7+N5zvdElSAW8IbNuBlh7PWwCZlV1jjKkF1AeyT/K1AFhr37LWJlprE2NiYqqodPFHWVvcgb5JK32diJxI6qpUXNZV4TmXdZG6MrWGK5JTkX8wn3GXP8uKH9cw7l+/59LbBjhdklTCGwLbQiDBGNPGGBOKexLBtHLXTANuK318PfCttdaWHh9eOou0DZAALKihusVP7S4LbK0V2EROJH1f+tGWtfLyi/LJyM6o4YrkZB3cm8PYS59m3YIMHv5wFBff1M/pkuRXOD6GrXRM2v3Al0Aw8I61dpUx5ilgkbV2GvAP4F+lkwqycYc6Sq/7GPcEhWLgPmttiSNvRPxGbra7C0dT2UVOLCE6gfCQ8ApDW3hIOPFR8Q5UJSeSuWEXDw3+C3u27eXxTx/kvKsTnS5JTsDxwAZgrZ0OTC937DGPx0eAYZW89lng2WotUAJK4ZEiAMLqhDpciYj3S+6SzJgvx1R4LsgEkdw1uYYrkhNZuyCdR68eT0mJi+e/foyuF3R0uiQ5Cd7QJSriVYoKiggKMlrSQ+QkRIZFMn3EdCJDIwkPCQfcLWuRoe7jEaERDlconmZPmc+DFz1B7YjapMx5xv/CmrUwZYr7/mSO+5CTDmzGmEuMMW8bY3qUPr+7+soScU6tkFq4XJaSEvWui5yMvnF9yXwgk5TLUxh3wThSLk8h84FMLenhRay1TH72U5687kXadIvj1bnP0rKDH66CNXUqDB0Ko0f/Es6sdT8fOtR93kedSpfo74D/Ax4xxkQBPaqnJBFnhTeoC8ChnMNENlTrgMjJiAiN4M6edzpdhlSg4HABL975BrM+msPAm/sx5q17CK3tp0M+kpJg5EhISXE/nzjRHdZSUtzHk5Kcre8MnEpg22OtPQA8aIwZD5xTTTWJOCqigbtbJ29/vgKbiPi0zA27ePqGl9mwbDN3PjeC5D9di3ujID9ljDukgTuklQW3kSPdx334vZ/KGLbPyx5Ya8cB/6z6ckSc16hFNAC7Nmc5XImIyOn7MW0+9/b6E7s3Z/H0tLEMH5vk32GtjGdoK+PjYQ1OIrAZY14xxhhr7Weex621r1VfWSLOievkHtexdc0OhysRETl1RYVFvDH6PZ66/kVadmjO64sn0PvKANpqqmzMmifPMW0+6mRa2PKAacaYugDGmEuNMXOqtywR50Q3a0jdyDpsWb3d6VLES+QW5DJpySTGfj2WSUsmkVuQ63RJIhXauXE3Y/o/TlrK5yTdfwUv//A0TVs3drqsmlMW1srGrLlcv4xp8/HQdsIxbNbaR4wxNwHfG2MKgHxgXLVXJuIQYwztE9uyet46p0sRL6CNzcUXWGv58r1ZvD7yHYKCg3gkdQz9h53ndFk1b+rUX8JaWTeo55i2/v1hyBBnazxNJ9MlOhC4C3dQiwH+YK39sboLE3FSj4u7sWHZZg7uzXG6FHGQNjYXX3Bgz0GevP5FXrrzdRJ6teXNZS8GZlgD9yzQtLRjx6yVhba0NJ+eJXoyXaIPA49aawfg3scz1RhzcbVWJeKwswd2A2DZtysdrkScpI3NxdvN/Wwhd3d/gAWfL+HuCbfwwszHadIqgPdBNsbdglZ+gkFlx33IyXSJXuzxeIUx5grgU+D86ixMxEkdEtvRsEl9Zn08l/436Es9UGljc/FWB/fm8Pqod/n237Np270V4798lLbdWzldllSjU95L1Fq7s7SbVMRvBdcK5qLhffnvG1+Ssy9XG8EHKG1sLt7GWst3H83hjVHvkrs/n1sfv4Hhf04iJDTE6dKkmp3WXqLW2sNVXYiIt7nktv4UFRbzzQc/OF2KOCS5SzJBpuIfk9rYXGrazk27efSa8Tw3IoUmrWN4fdHz3PL4MIW1AHHKLWwigSK+Rxs6n9+BT17+L1ffe6l+KAagso3Ny88SDTJB2ti8BuQW5JK6KpX0fekkRCeQ3CWZyLDAa+0uKizik5f+x+RnPsEEGX774q0MGTmY4OBgp0uTGmSsD69JcroSExPtokWLKj5prXtacFLSsYMTKzsufm3hF0t5aPBfGP3mbxl81yCnyxGH5BXmkboylYzsDOKj4knumqywVs0qWk6lLCgH0nIqy75byWv3T2Lrmh30HdqbeyfeTuOWjZwuS6qRMWaxtTbxuOMKbOVMmQJDhx67hovnQnxpaT67houcOmstv+/zZw7uyeEfq1/x3w2TRbxIbkEusS/Hklt4/ALFkaGRZD6Q6feBedfmLN4e+wE//GceTds05r6UO+hzVQDtVhDAKgtspzWGza8lJR2/KrLnqsk+vIaLnDpjDHc8exO7Nu/h4xemOV2OSEAI5OVUDuUe5p2H/80dnUYx/3+LufXxG5i08mWFNdEYtuOUXxU5JcX92LPFTQJKz0Hd6X/DeXz4XBoDR/SjWdsmTpck4tcCcTmVkpISvn7/e9595EOydx1g4M39uPMvI4hpEe10aeIl1MJWEc/QVkZhLaD99sXbCAoOIuV3b+NyVfyXv4hUjbLlVCrib8upWGuZ//li7u35J176zRs0aR1DytxnGffPPyisyTEU2CpS1g3qycc3jZUzE9Mimt+Mv5nFX/3M1FdnOF2OiF8LlOVUVs5ew5j+j/HI1eMpOFzIIx+NJmXOs3Tu097p0sQLKbCVV37Mmst1/Jg2CUjX/O4yzrsmkUnjPiBj6SanyxHxW2XLqUSGRh5taQsPCScyNNIvllPJWLaJR68Zz+gLHyNzw25GvnE3/1g1kf43nI9RT45UQrNEy9MsUfkVB/fm8NseDxJWJ5SUuc/SIKa+0yWJ+C1/W05lw8+b+eDpT5idNp+IBuEk/+lakv4wmNp1w5wuTbyIlvXwoHXYaoAf/zuunreOPw58krZnteaFmY/rh62I/KqMpZv411P/Ye5nC6lbrw5D/jCY68dcTUSDisfpSWDTsh4nyxh3C1r5MFHZcanY1KnulkrPbuSylsqhQ93nfVTn8zrw58kjWbcgg7/c9AolxSVOlyQiXmjN/HQevXY89/b6E8u/X82tj9/A5M1vcPtTwxXWqoK17l6x8g1PlR33cQpsUj38fD27vkN6c9+rdzBv2iJevvvvlJQotImIe9bnwi+W8seBT/CH8x5i1Zx13PZkMh9s+hu3PD5MQa0q+XHDQEW0DptUjwBYz+7a+y4nZ28u/3zyY1wlLh5853fa208kQBUXFfP9x/P4+IXP2Lh8C41io7j7hVu56reDqBNRx+ny/JNnwwC4f7f4UcNAeRrDJtXLWgjyaMh1ufwirHma/MynvPfYR1x04wX86b37qRWiv4NEAkXegXymv/0NU1+bwZ7t+2jdpSXDHryGi268gJDQEKfL83+evTdlfLxhQGPYpOqc7LiBAFnPbsQj13HncyP47sM5PHbt8+QfrHiFdp8RYONCRE7HtnU7+Ovv/8FNcffw9tgPiG3fjGf+O443f36RS28boLBWUwJooXtHA5sxJsoY87UxJr30vmEF1/Qwxswzxqwyxiw3xiR7nHvPGLPJGLOs9NajZt9BgDqZcQMBtp7d8LFJjH7rHpZ8s4Lf93mI7esznS7p9AXYuBCRk1VSUsK8/y7iz1c8wx2dRjH97W/oO7Q3byyZwAvfPE7vK3sRFKR2kBoVIA0DgHuApFM3YAIwrvTxOOD5Cq5pDySUPm4O7AQalD5/D7j+VD9vr169rNdwuaxNS3Pfn8xxb+ByWTtypLXgvq/oeVrasc/Lvy4tzdn3UE1+/n6VvS7m/2xSw9vswi+XOV3O6TmZ/1+RALI/64D9cPwUe3Obe+0gc71Njr3L/uup/9js3QecLi2w+enPKmCRrSgzVXSwpm7AOqBZ6eNmwLqTeM3PHgHO9wObrwYbzxrLbuXfg68F0Sqyc9Nue1f3MfbS4GH2g2c+scXFxU6XdOpO9P8r4udcLpdd9t1K+9zNKfaK2jfaQeZ6++DFj9sfPplniwqLnC5PrPXd358nUFlgc3TSgTHmgLW2gcfz/dba47pFPc6fC7wPdLHWuowx7wHnAQXATNytdQUn+rxeNenAlus6LD/LxZv74q2fTyiwp7/47+G8w0z87Zt89+EcelzUhT++ex+N42JqqPAq4u//vyIV2JuZzdfvf88X735LZsYuwuvXZdDNF3L1vZfSqnNLp8sTT2fwM9qbVTbpoCZa0b4BVlZwuxY4UO7a/b/ycZrhbpHrU+6YAcJwB7nHfuX1dwOLgEVxcXFVG4fPlC+2ZvhizafqDP96c7lcdsY739qrI2+219S/xX71/izr8pV/n0D4/xUpVVRYZOdMXWAfueY5e2mtG+wgc70dM+Ax+9U/Z9nD+UecLk8CDL7cJQrUA5YAw37lYw0A/ncyn9erukTLuFzH/nL05l+Mfjpu4DhV9D4zN+yyo/o9YgeZ6+0jVz9nd23OqubCz1Cg/P9KQHO5XHbN/PX2r7//h72+8R12kLne3tDsN3bSuA/stvWZTpcnAcxbA9sLHDvpYEIF14Ti7u4cVcG5srBngFeA8Sfzeb0usPlaa4afjhuoUBX93xQXF9v/vDTNXhU+wl4VPsJ+/OI07x0HE0j/vxJwMjfssv966j/29g6/t4PM9faK2jfaJ4e9aOd8tsAWF/ngeFPxO5UFNqfHsEUDHwNxwFbcLWjZxphE4B5r7W+MMTcD7wKrPF56u7V2mTHmWyAGd2BbVvqavBN9Xo1hO0PWP8cNVMpW3Viu3Vv28Nr9k5j/+RJad2nJva/8Hz0HdquiQqtIoP3/it/bt3M/3388l+8+msPa+ekAnDWgCwNH9KPfdX20XZR4lcrGsGmnA6dNmeJe28oznHmGuLQ096bz4gzP/4syZxikrbXM/Wwhf3/gfXZtyqLPVb2449kbadOtVRUVLSLZu/YzZ8oCfvhkHsu/X43LZWnXozUXDe/LRcPP971JQBIwFNg8eFVgU2uG96rm1s/CI4WkvfI5Hz0/lUM5hxl4cz9ufeIGmrVpUoVvQiRwZG3by+y0+cxOm8/K2Wux1tKyQ3P633A+A4ZfQKtOLZwuUeSEFNg8eFVgE+9VQ62fOdm5pD7/GVNfm05JsYvLbh/AjQ8NpWnrxlXwJkT827Z1O5gzdSGz035i3cINALTu2pJ+Q/vQ77retO4ah9EfveJDFNg8KLDJSanh1s892/fx0fgpzJg0E5fLMiD5fIY9eA3tzmpdZZ9DxNcVFxWzas465v13EfM/X8z29TsBaJ/Yjn5De9N3aG9atG/ucJU+Qj08XkmBzYMCm3izPdv38clL/2X6pG84kl9Az0u6c92oq0i87CztUygBaX/WQRZ9sYwFM5aw6MufyTuQT0hoLboP6EKfq3px/rXn0LhlI6fL9D0aQ+2VFNg8KLCJL8jdn8fnb37NlFenk73rAM3jm3Lt7y7n0tsHaFab+LWSkhLWL9rIwhlLWTBjCesXbcRaS8Mm9TnnirPpc1UivS7pTt3IOk6X6tt8cZWCAKDA5kGBTXxJUWERP346n8/+OoPV89ZTu24YF95wHlfccTFdLuio8Tni86y17EjfyZJvVrBk5nJ+/m4VeQfyCQoydOzTnnMvP5tzB59Nux6t1cpc1aphJrycGQU2Dwps4qvWL97A529+zXcfzeFw3hGatW3CwBH9GHjzhbRIaOZ0eSInLWvrHpZ9t4pls1ay7NuV7Nm2D4DGcY3oOag7PQd1p9cl3akXHelwpQGgCtealDOnwOZBgU183eG8w/zwyU/MnPwjy75dibWWjr0TGHTzhfS/4TwaxNR3ukSRY2Rt28uKH9aw7LuV/DxrFTs37gagXnQkZw3ozNkDu9NzUDeat2uqVuOapBY2r6PA5kGBTfzJ3h37+Pbfs5k5+Uc2Lt9CUJChS9+OnH/NOVyQdC7N2mpdN6lZJSUlbF65jVVz1rFyzhpWzl57tAUtokE43ft35qwBXehxUVdad22pbk6naAybV1Jg86DAJv5q4/It/PDJPOZNW8TG5VsAaNMtjvOvOYdzB59Nh3PjCQ4OdrhK8TcH9hxk7fwM1vy0njXz01m3IINDuYcBiG7ekK59O9Ll/I507duRtme10tegt9AsUa+kwOZBgU0Cwc6Nu5n72ULmTlvIyh/X4HJZIhuG0+vSs0i8rAe9Lj2LRs2jnC5TfMyRQwVkLN3EugUZrFuUwboFGWRucHdvBgUH0e6sVnTs3Z7O57Wna9+ONGkVoy5Ob6V12LySApsHBTbfkVuQS+qqVNL3pZMQnUByl2QiwzQI+VTlZOey9JsVLPxiGQu/XEb2zv0AtOwYS/cLO9OtXye6XdhJa1nJMXKyc9n48xY2/ryFjGWbSF+8ka1rtuNyuX9vNIqNosM57ejUpwOd+iTQPrEdteuGOVy1iG9TYPOgwOYbZm+dzeDJg3FZF/lF+YSHhBNkgpg+Yjp94/o6XZ7PstaycfkWFn35Mz9/v4pVc9ZyKMfdfdW0dQzdLuxMpz7t6dQ7gTbd4giupe4rf1dSXML29J1sXrGVTSu2snHFFjYs20zW1r1Hr4lq2oD4nm1o36sdCb3a0uGceKKbNXSwahH/pMDmQYHN++UW5BL7ciy5hbnHnYsMjSTzgUwiQiMcqMz/lJSUsGn5Vlb8uIYVP65mxY9rOZB1EICwOqEk9GpLx3MT6Ng7gfizW9OsbRMNEvdRxUXF7Ny4my2rt7N1zQ62rt3OphVb2bZmB0WFxQAEBRli2zcn/uzWtDurDe16tKbdWa1o2KSBw9WLBAYFNg8KbN5v0pJJjPpiFPlF+cedCw8JJ+XyFO7seacDlfk/ay27NmcdHUS+dkE6GUs3U1RQBECdiNq06RZHm26taHdWK1p3jaNlx+ZaSsRLuFwu9u7IZkf6TjIzdrEjfSc7MnaxfX0mmRm7KC4qOXptTItoWneLo02Xlu77bnHEdYwltHaog+9AJLBVFthqOVGMyImk70uvMKwB5Bflk5GdUcMVBQ5jDM3aNKFZmyZcNPwCwL3bwsblW9mwbDMbf97MxhVbmJU6h8/f+vro6+pFRxLXKZa4jrHEdWpB8/imNI9vSrM2jRUAqpC1lpx9uWRt3cuuzXvYtXE3OzdlsWtzFrs2ZbF7cxaFR4qOXh8SFkLzdk1o0b45519zDi07xtKqcwtadozV1k4iPkSBTbxSQnQC4SHhlbawxUfFO1BV4AoJDaFDYjs6JLY7esxaS9bWvaXda+4utm3rdjB7ygJyJs08ep0xhkaxUTSPb0rT1o1p0iqGmLhGNC69xbSIIqyOBqqDu3v6QFYO+zKz2Ze5v/SWzd7t+8jatpc92/aRtXUvBYcLj3ldRINwmrZpTKvOLehzZU+axzcjNqEpsfFNadQiWl3YIn5AXaLilTSGzbcd3JvDjoxd7Nywm8wNu0pvu9m9OYvsnQco/3MnokE4DZvUp2HTBkQ1bUDDJg1o0Lg+9RtFUi+67BZBZHQkkQ3DCa0d6tVLRVhrKSooIj/nMPkHD5GbnXfc7eDeHA7sOciBrBwO7snhQNZBDu7NPe7fJijI0KBJg6MBt3HLRsS0jCamZSOato6haZvGRDbU94KIv9AYNg8KbL5Bs0T9U2FBEft2ZJO1dS+7t+xhX+Z+snfuJ3v3AfbvOkD2Lvd92cKrFQmuFUzdenUIr1+XuvXqUDeyDrXDwwirG0btumGE1QmldnhtQsJqUSu0FrVCyu6DqRVSi6DgIPfyUsYQFGSOhr+SEhfWZXGVuNw3l4viwmKKCoopLCii6OitmCOHjlBwqJAj+Uc4Unp/OPcIh3IOkX/w0DFjxSoS0SCcBo3r0aBxffctph71Y+oR3awhUc0a0ig2iujmDWnYpIFm6ooEEAU2DwpsviOvMI/UlalkZGcQHxVPctdktawFiILDBeTsyyNnX27pzf04/0D+0ZarQ7mHOJRzmEM5hyk4VMCRQwUUHCp0P84vcIer0tmPZyq4VjAhYbUICa1FSFgItcPDqB1e+2hQDKsbSt3IOoTXq1saJsOPhsrIqAgioyKoV3of0SBcIUwcobUtvZ8CmwcFNpHAYa27xay4qJjiwmKKi0pwuSzW5cJa93lbuhBsUHAQQUHGfV/6uFaou3VO2ymJr1OvhW/QLFFfpy1ERE6LMYbgWsEE1wrW5AYJWLkFuQyePPiYccFlk7oGTx6sccE+QFOHfMXUqe5NekePdoc0+GWT3qFD3edFREQqkLoqFZd1VXjOZV2krkyt4YrkVKmFzVckJcHIkZCS4n4+caI7rKWkuI8nJTlbn4iIeC2tben7FNh8hTHukAbukFYW3EaOdB9Xd6iIiFRCa1v6Pk068DXWgucimC6XwpqIiPwqrW3pOyqbdKAxbL6kbMyaJ88xbSIiIhWIDItk+ojpRIZGEh4SDrhb1iJD3ccV1ryfukR9RVlYKxuz5jmGDdQtKiIiv6pvXF8yH8jU2pY+SoHNV0ydemxYKz+mrX9/GDLEmdq05IiIiE+ICI3gzp53Ol2GnAZHu0SNMVHGmK+NMeml9w0rua7EGLOs9DbN43gbY8z80tenGmNCa676GpaUBGlpx7aklYW2tDRnZ4lqyREREZFq5fQYtnHATGttAjCz9HlFDltre5TervE4/jwwsfT1+wH//bPBGHcLWvmWqsqO1yTPJUfKQpuWHBEREakyjs4SNcasAwZYa3caY5oBs6y1HSq4Ls9aG1HumAH2AE2ttcXGmPOAJ6y1l53o8/r0LFFv5RnSymjJERERkVPirbNEm1hrdwKU3jeu5LraxphFxpifjDFlzTXRwAFrbdnOztuB2Mo+kTHm7tKPsWjPnj1VVb+U8RxTV0ZhTUREpEpUe2AzxnxjjFlZwe3aU/gwcaVp8ybgFWNMO6CiJFBpc6G19i1rbaK1NjEmJuYU34WckJYcERERqTbVHtistYOstV0ruH0G7C7tCqX0PquSj5FZer8RmAWcDewFGhhjyma6tgAyq/ntSEXKj1lzuY4f0yYiIiKnzeku0WnAbaWPbwM+K3+BMaahMSas9HEj4AJgtXUPvvsOuP7XXi81oLIlR8pCm2aJioiInBGnJx1EAx8DccBWYJi1NtsYkwjcY639jTHmfOBNwIU7YL5irf1H6evbAh8BUcBS4GZrbcGJPq8mHVQxrcMmIiJSJSqbdKC9REVERETORBU2XHjrLFERqWG5BblMWjKJsV+PZdKSSeQWHL8ZtIiInIIaWEBeW1OJBJDZW2czePJgXNZFflE+4SHhjPlyDNNHTKdvXF+nyxMR8U2eC8jDsft9V9EC8uoSFQkQuQW5xL4cS27h8S1qkaGRZD6QqU2gRUROVxUtIK8uUZEAl7oqFZd1VXjOZV2krkyt4YpERPxINS8gr8AmEiDS96WTX5Rf6NkKmQAAD/9JREFU4bn8onwysjNquCIRET9SzQvIK7CJBIiE6ATCQ8IrPBceEk58VHwNVyQi4idqYAH5wA5s1sKUKcf/Q1Z2XMSHJXdJJshU/C0fZIJI7ppcwxWJiPiJGlhAPrADWw1MwxXxFpFhkUwfMZ3I0MijLW3hIeFEhrqPa8KBiMhpSkqCtLRjx6yVhba0NM0SPV1HZ4mWb8IsPw23CgcLiniLvMI8UlemkpGdQXxUPMldkxXWRES8hHY68HDMsh5VNA1XRORXaQs3ETkJWtajMtU8DVdEBNAQjECgcdFSjRTYqnkarogIcOxK6GU/Y6p4JfSA53RgUiiXahTYga0GpuGKiADHzxoLCtJ42armdGBSKJdqFNhj2KZMcX8Te/7A9PwGS0uDIUOcLldE/Im17rBWxuVSWKsq3jCRTOOi5Qxp0oGHY2aJahCwiNQU/TKvft7wb6xQLmdAkw4qYoy7Ba38N1Jlx0VETpeGYNQMpyeSaVy0VJPADmwiIjWlBlZCF5wNTArlUo0U2EREakINrIQe8JwOTArlUo0CewybiIj4D6cnkmlctFQBTTrwoMAmIuKHFJjED1QW2Go5UYyIiEiVK5swdrLHRXyIxrCJiIiIeDkFNhEREREvp8AmIiIi3svpPWK9hAKbiIiIeC+n94j1Epp0ICIiIt4rKemXtezg+D1iA2QNQwU2ERER8V6e242lpPwS3AJsH16twyYiIiLez1oI8hjJ5XL5ZVjzys3fjTFRxpivjTHppfcNK7jmImPMMo/bEWNMUum594wxmzzO9aj5dyEiIiLVysk9Yr2E05MOxgEzrbUJwMzS58ew1n5nre1hre0BXAwcAr7yuOSPZeettctqpGoRERGpGU7vEeslnB7Ddi0woPTx+8AsYOyvXH89MMNae6h6yxIRERGvMHXqL2GtbMya55i2/v0DYicLp1vYmlhrdwKU3jc+wfXDgQ/LHXvWGLPcGDPRGBNWHUWKiIiIQ5KSIC3t2AkGZaEtLS1gZolWe2AzxnxjjFlZwe3aU/w4zYBuwJceh/8MdATOAaL4ldY5Y8zdxphFxphFe/bsOY13IoIWcBSpKvpekpNVthds+QkGlR33U9Ue2Ky1g6y1XSu4fQbsLg1iZYEs61c+1A3AFGttkcfH3mndCoB3gXN/pY63rLWJ1trEmJiYqnlzEni0gKPImSkLZFOmHPu9ZK27tWTUKH0viVTA6TFs04DbgPGl95/9yrU34m5RO8oY08xau9MYY4AkYGV1FSoCaAFHkTNV9kfPH/7gvqWk/PLHz6uvuu/1vSRyHEfXYTPGRAMfA3HAVmCYtTbbGJMI/9/e3cbKcZUHHP8/SeQghEqcYFqHpMSWXCACKUhXkdp+cEgDCf0Q37imNRWSKUEV9CUfaCWCUqkRLaLpl0DVqjSlLfRFJJDYwRUgK8FOK1UEMFIoCZbjG/MBNykJhbShVV3ATz/sLIzvzt47197defv/pNXdPTuzPvP4zOyzc86c4V2Z+c5iuauAfwGuzMwzpfUPA1uAAB4r1vneev+u87DpvJSvWBob2ASO0jkr7z+33TYqGydqMCr70IfclzRY0+Zhc+Jc6VwMZAJHaS6qfvSMuS9p4Fo5ca7USU7gKJ2f8rQMq7kvSZVM2KSNcAJH6fxlji4uKCuPaXNfkiaYsEkbMW0Cx3HS5pVt0trGP3rG49bGiVr5tfuSNKHpq0SlbhlP4Li8PDmB486dXtkmrWf8o+e2286eoT5iVP7AA3Ddde5L0ipedCBJWpzMUdJW/tGzVrk0MNMuOvAMmyRpccaz09ctlwQ4hk2SJKn1TNgkSZJazoRNkiSp5UzYJEmSWs6ETZJmJRMOHJic9HVauSTVZMImSbPy4IOwe/fZM/WPJ4rdvdvJYCWdM6f1kKRZWV7+8V0vYDShcvlWZk4GK+kceYZNkmZl9a3KLrhg8lZmXWH3rtQqJmySNEvjpK2sa8ka2L0rtYwJmyTN0jipKSsnPV1R7t4d19/uXakxJmySNCurk5ozZyaTnq7oU/eu1AMmbJI0Kw8+OJnUlJOernUj9qV7V81xLOTMmLBJ0qwsL8P+/WcnNeOkZ//+7nUj9qV7V81xLOTMmLBJ0qxEwC23TJ6BmlbeZn3q3lVz5j0WckBn8EzYJEmT+ta9q2bMeyzkgM7gRfYo+6xraWkpjx492nQ1JKm9MkdfdsvLZ3+pTiuX1pI5StbGzpyZTftZfcZu9WTVHRxzGRFfycyl1eXe6UCSNGncjVu3XJpm2ljIWSRT5QtjPvzhH99lpKPJ2lrsEpWkoRjQeB+1xCLGQg7kamYTNkkaigGN91FLLGIs5ECuZjZhk6Sh8O4FWrR5T3UzoKuZvehAkoak/AU31sPxPhqIAwdGZ4fLbbjcxvfv79yYy2kXHZiwSVLTFn1F5ryu2JMWrYdXM09L2BrtEo2It0TEExFxJiImKlda7qaIOB4RKxFxe6l8W0R8MSJORMR9EbFpMTWXpBla5NiygYz30UD0abLqdTQ9hu1xYDfwz9MWiIgLgT8D3gxcDbw1Iq4u3r4LuDszdwDfBW6db3UlaQ4WNbZsQON9pL5pdB62zDwGEGtnwNcCK5l5slj2XmBXRBwDrgd+tVju48CdwJ/Pq76SNBeLmktq2hV74393587OjfeRhqLpM2x1vAL4Zun1qaLsMuD5zPzBqnJJ6p5FzCXVt5vTSwMy94QtIh6OiMcrHrvqfkRFWa5RPq0evx4RRyPi6HPPPVfzn5akBVnE2LIBjfeR+mbuCVtm3pCZr614fLrmR5wCriy9vgJ4Gvg2cElEXLSqfFo97snMpcxc2rJly7lsiiTNh2PLJK2jC12iXwZ2FFeEbgL2AgdzNB/JEWBPsdw+oG4SKEntsYjZ4CV1WtPTetwSEaeAnwU+ExGHivLLI+KzAMUYtd8CDgHHgE9m5hPFR7wXeE9ErDAa0/ZXi94GSTpvji2TtA4nzpUkSWqJVk6cK0mSpPWZsEmSJLWcCZskSVLLmbBJkiS1nAmbJElSy5mwSZIktZwJmyRJUsuZsEmSJLWcCZskSVLLmbBJkiS1nAmbJElSyw3yXqIR8QJwvOl6tMzLgG83XYkWMi7VjEs141LNuFQzLpOMCbwyM7esLryoiZq0wPGqG6sOWUQcNSaTjEs141LNuFQzLtWMyyRjMp1dopIkSS1nwiZJktRyQ03Y7mm6Ai1kTKoZl2rGpZpxqWZcqhmXScZkikFedCBJktQlQz3DJkmS1Bm9TNgi4i0R8UREnImIqVebRMRNEXE8IlYi4vZS+baI+GJEnIiI+yJi02JqPl8RcWlEPFRs10MRsblimTdExGOlx/9GxHLx3sci4hul965Z/FbMXp24FMv9sLTtB0vlQ24v10TEF4r97V8j4ldK7/WqvUw7XpTev7j4/18p2sNVpffeV5Qfj4gbF1nveaoRk/dExNeLtvH5iHhl6b3K/akPasTl7RHxXGn731l6b1+xz52IiH2Lrfl81YjL3aWYPBkRz5fe6217qS0ze/cAXgO8CngEWJqyzIXAU8B2YBPwVeDq4r1PAnuL5x8B3t30Ns0oLn8M3F48vx24a53lLwW+A7y4eP0xYE/T29FUXIDvTSkfbHsBfgbYUTy/HHgGuKRv7WWt40Vpmd8APlI83wvcVzy/ulj+YmBb8TkXNr1NC4rJG0rHj3ePY1K8rtyfuv6oGZe3A39ase6lwMni7+bi+eamt2lRcVm1/G8Df9339rKRRy/PsGXmscxcb2Lca4GVzDyZmf8H3AvsiogArgfuL5b7OLA8v9ou1C5G2wP1tmsP8LnM/J+51qp5G43Ljwy9vWTmk5l5onj+NPAsMDHhYw9UHi9WLVOO1/3ALxTtYxdwb2aezsxvACvF53XdujHJzCOl48ejwBULrmMT6rSVaW4EHsrM72Tmd4GHgJvmVM9F22hc3gp8YiE164heJmw1vQL4Zun1qaLsMuD5zPzBqvI++MnMfAag+PvydZbfy+QO84Gie+PuiLh4HpVsQN24vCgijkbEo+NuYmwvPxIR1zL65fxUqbgv7WXa8aJymaI9/Cej9lFn3S7a6HbdCnyu9Lpqf+qDunH5pWLfuD8irtzgul1Ue9uKrvNtwOFScV/bS22dvdNBRDwM/FTFW3dk5qfrfERFWa5R3glrxWWDn7MVeB1wqFT8PuDfGX0p3wO8F3j/udV0sWYUl5/OzKcjYjtwOCK+BvxXxXJDbS9/B+zLzDNFcWfbS4U6x4VeHlPWUHu7IuJtwBKws1Q8sT9l5lNV63dMnbj8I/CJzDwdEe9idGb2+prrdtVGtm0vcH9m/rBU1tf2UltnE7bMvOE8P+IUcGXp9RXA04zuYXZJRFxU/Eoel3fCWnGJiG9FxNbMfKb4gn12jY/6ZeBAZn6/9NnPFE9PR8TfAL87k0ovwCziUnT5kZknI+IR4PXAAwy8vUTETwCfAX4vMx8tfXZn20uFaceLqmVORcRFwEsZjQGts24X1dquiLiB0Q+AnZl5elw+ZX/qwxfwunHJzP8ovfxL4K7SutetWveRmdewGRvZD/YCv1ku6HF7qW3IXaJfBnbE6Aq/TYwayMHMTOAIo/FbAPuAOmfsuuAgo+2B9bdrYvxA8aU9Hre1DDw+hzo2Yd24RMTmcZdeRLwM+Hng60NvL8W+cwD428z81Kr3+tReKo8Xq5Ypx2sPcLhoHweBvcVVpNuAHcCXFlTveVo3JhHxeuAvgJsz89lSeeX+tLCaz1eduGwtvbwZOFY8PwS8qYjPZuBNnN3L0WV19iEi4lWMLrj4Qqmsz+2lvqavepjHA7iFUTZ/GvgWcKgovxz4bGm5XwSeZJSl31Eq387ogLoCfAq4uOltmlFcLgM+D5wo/l5alC8BHy0tdxXwb8AFq9Y/DHyN0Rfv3wMvaXqbFhUX4OeKbf9q8fdW20sCvA34PvBY6XFNH9tL1fGCURfvzcXzFxX//ytFe9heWveOYr3jwJub3pYFxuTh4hg8bhsHi/Kp+1MfHjXi8kHgiWL7jwCvLq37jqINrQC/1vS2LDIuxes7gT9atV6v20vdh3c6kCRJarkhd4lKkiR1ggmbJElSy5mwSZIktZwJmyRJUsuZsEmSJLWcCZskSVLLmbBJkiS1nAmbJG1ARByJiDcWz/8wIv6k6TpJ6r/O3ktUkhry+8D7I+LljO5neHPD9ZE0AN7pQJI2KCL+CXgJcF1mvhAR2xndfuqlmbln7bUlaePsEpWkDYiI1wFbgdOZ+QJAZp7MzFubrZmkPjNhk6SaImIr8A/ALuC/I+LGhqskaSBM2CSphoh4MbAf+J3MPAb8AXBno5WSNBiOYZOk8xQRlwEfAN4IfDQzP9hwlST1jAmbJElSy9klKkmS1HImbJIkSS1nwiZJktRyJmySJEktZ8ImSZLUciZskiRJLWfCJkmS1HImbJIkSS1nwiZJktRy/w8iX1JMsAtbFwAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - }, - { - "data": { - "text/plain": [ - "" - ] - }, - "execution_count": 12, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "widgets.interact(interactive_classification, highlight=dropdown_highlight)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "Zadanie klasyfikacyjne z powyższego przykładu polega na przypisaniu punktów do jednej z dwóch kategorii:\n", - " 0. czerwone krzyżyki\n", - " 1. zielone kółka\n", - "\n", - "W tym celu zastosowano regresję logistyczną." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "W rezultacie otrzymano model, który dzieli płaszczyznę na dwa obszary:\n", - " 0. na zewnątrz granatowej krzywej\n", - " 1. wewnątrz granatowej krzywej\n", - " \n", - "Model przewiduje klasę 0 („czerwoną”) dla punktów znajdujący się w obszarze na zewnątrz krzywej, natomiast klasę 1 („zieloną”) dla punktów znajdujących sie w obszarze wewnąrz krzywej." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "Wszysktie obserwacje możemy podzielić zatem na cztery grupy:\n", - " * **true positives (TP)** – prawidłowo sklasyfikowane pozytywne przykłady (zielone kółka w wewnętrznym obszarze)\n", - " * **true negatives (TN)** – prawidłowo sklasyfikowane negatywne przykłady (czerwone krzyżyki w zewnętrznym obszarze)\n", - " * **false positives (FP)** – negatywne przykłady sklasyfikowane jako pozytywne (czerwone krzyżyki w wewnętrznym obszarze)\n", - " * **false negatives (FN)** – pozytywne przykłady sklasyfikowane jako negatywne (zielone kółka w zewnętrznym obszarze)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "Innymi słowy:\n", - "\n", - "" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "slideshow": { - "slide_type": "skip" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "TP = 5\n", - "TN = 35\n", - "FP = 3\n", - "FN = 6\n" - ] - } - ], - "source": [ - "# Obliczmy TP, TN, FP i FN\n", - "\n", - "tp = 0\n", - "tn = 0\n", - "fp = 0\n", - "fn = 0\n", - "\n", - "for i in range(len(Y_expected)):\n", - " if Y_expected[i] == 1 and Y_predicted[i] == 1:\n", - " tp += 1\n", - " elif Y_expected[i] == 0 and Y_predicted[i] == 0:\n", - " tn += 1\n", - " elif Y_expected[i] == 0 and Y_predicted[i] == 1:\n", - " fp += 1\n", - " elif Y_expected[i] == 1 and Y_predicted[i] == 0:\n", - " fn += 1\n", - " \n", - "print('TP =', tp)\n", - "print('TN =', tn)\n", - "print('FP =', fp)\n", - "print('FN =', fn)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "skip" - } - }, - "source": [ - "Możemy teraz zdefiniować następujące metryki:" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "#### Dokładność (*accuracy*)\n", - "$$ \\mbox{accuracy} = \\frac{\\mbox{przypadki poprawnie sklasyfikowane}}{\\mbox{wszystkie przypadki}} = \\frac{TP + TN}{TP + TN + FP + FN} $$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "Dokładność otrzymujemy przez podzielenie liczby przypadków poprawnie sklasyfikowanych przez liczbę wszystkich przypadków:" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Accuracy: 0.8163265306122449\n" - ] - } - ], - "source": [ - "accuracy = (tp + tn) / (tp + tn + fp + fn)\n", - "print('Accuracy:', accuracy)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "**Uwaga:** Nie zawsze dokładność będzie dobrą miarą, zwłaszcza gdy klasy są bardzo asymetryczne!\n", - "\n", - "*Przykład:* Wyobraźmy sobie test na koronawirusa, który **zawsze** zwraca wynik negatywny. Jaką przydatność będzie miał taki test w praktyce? Żadną. A jaka będzie jego *dokładność*? Policzmy:\n", - "$$ \\mbox{accuracy} \\, = \\, \\frac{\\mbox{szacowana liczba osób zdrowych na świecie}}{\\mbox{populacja Ziemi}} \\, \\approx \\, \\frac{7\\,700\\,000\\,000 - 600\\,000}{7\\,700\\,000\\,000} \\, \\approx \\, 0.99992 $$\n", - "(zaokrąglone dane z 27 marca 2020)\n", - "\n", - "Powyższy wynik jest tak wysoki, ponieważ zdecydowana większość osób na świecie nie jest zakażona, więc biorąc losowego Ziemianina możemy w ciemno strzelać, że nie ma koronawirusa.\n", - "\n", - "W tym przypadku duża różnica w liczności obu zbiorów (zakażeni/niezakażeni) powoduje, że *accuracy* nie jest dobrą metryką.\n", - "\n", - "Dlatego dysponujemy również innymi metrykami:" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "#### Precyzja (*precision*)\n", - "$$ \\mbox{precision} = \\frac{TP}{TP + FP} $$" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Precision: 0.625\n" - ] - } - ], - "source": [ - "precision = tp / (tp + fp)\n", - "print('Precision:', precision)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "Precyzja określa, jaka część przykładów sklasyfikowanych jako pozytywne to faktycznie przykłady pozytywne." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "#### Pokrycie (czułość, *recall*)\n", - "$$ \\mbox{recall} = \\frac{TP}{TP + FN} $$" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Recall: 0.45454545454545453\n" - ] - } - ], - "source": [ - "recall = tp / (tp + fn)\n", - "print('Recall:', recall)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "Pokrycie mówi nam, jaka część przykładów pozytywnych została poprawnie sklasyfikowana." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "#### *$F$-measure* (*$F$-score*)\n", - "$$ F = \\frac{2 \\cdot \\mbox{precision} \\cdot \\mbox{recall}}{\\mbox{precision} + \\mbox{recall}} $$" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "F-score: 0.5263157894736842\n" - ] - } - ], - "source": [ - "fscore = (2 * precision * recall) / (precision + recall)\n", - "print('F-score:', fscore)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "$F$-_measure_ jest kompromisem między precyzją a pokryciem (a ściślej: jest średnią harmoniczną precyzji i pokrycia)." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "$F$-_measure_ jest szczególnym przypadkiem ogólniejszej miary:" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "*$F_\\beta$-measure*:\n", - "$$ F_\\beta = \\frac{(1 + \\beta) \\cdot \\mbox{precision} \\cdot \\mbox{recall}}{\\beta^2 \\cdot \\mbox{precision} + \\mbox{recall}} $$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - "Dla $\\beta = 1$ otrzymujemy:\n", - "$$ F_1 \\, = \\, \\frac{(1 + 1) \\cdot \\mbox{precision} \\cdot \\mbox{recall}}{1^2 \\cdot \\mbox{precision} + \\mbox{recall}} \\, = \\, \\frac{2 \\cdot \\mbox{precision} \\cdot \\mbox{recall}}{\\mbox{precision} + \\mbox{recall}} \\, = \\, F $$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "## 3.3. Obserwacje odstające" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "**Obserwacje odstające** (*outliers*) – to wszelkie obserwacje posiadające nietypową wartość.\n", - "\n", - "Mogą być na przykład rezultatem błędnego pomiaru albo pomyłki przy wprowadzaniu danych do bazy, ale nie tylko.\n", - "\n", - "Obserwacje odstające mogą niekiedy znacząco wpłynąć na parametry modelu, dlatego ważne jest, żeby takie obserwacje odrzucić zanim przystąpi się do tworzenia modelu." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "W poniższym przykładzie można zobaczyć wpływ obserwacji odstających na wynik modelowania na przykładzie danych dotyczących cen mieszkań zebranych z ogłoszeń na portalu Gratka.pl: tutaj przykładem obserwacji odstającej może być ogłoszenie, w którym podano cenę w tys. zł zamiast ceny w zł." - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "# Przydatne funkcje\n", - "\n", - "def h_linear(Theta, x):\n", - " \"\"\"Funkcja regresji liniowej\"\"\"\n", - " return x * Theta\n", - "\n", - "def linear_regression(theta):\n", - " \"\"\"Ta funkcja zwraca funkcję regresji liniowej dla danego wektora parametrów theta\"\"\"\n", - " return lambda x: h_linear(theta, x)\n", - "\n", - "def cost(theta, X, y):\n", - " \"\"\"Wersja macierzowa funkcji kosztu\"\"\"\n", - " m = len(y)\n", - " J = 1.0 / (2.0 * m) * ((X * theta - y).T * (X * theta - y))\n", - " return J.item()\n", - "\n", - "def gradient(theta, X, y):\n", - " \"\"\"Wersja macierzowa gradientu funkcji kosztu\"\"\"\n", - " return 1.0 / len(y) * (X.T * (X * theta - y)) \n", - "\n", - "def gradient_descent(fJ, fdJ, theta, X, y, alpha=0.1, eps=10**-5):\n", - " \"\"\"Algorytm gradientu prostego (wersja macierzowa)\"\"\"\n", - " current_cost = fJ(theta, X, y)\n", - " logs = [[current_cost, theta]]\n", - " while True:\n", - " theta = theta - alpha * fdJ(theta, X, y)\n", - " current_cost, prev_cost = fJ(theta, X, y), current_cost\n", - " if abs(prev_cost - current_cost) > 10**15:\n", - " print('Algorithm does not converge!')\n", - " break\n", - " if abs(prev_cost - current_cost) <= eps:\n", - " break\n", - " logs.append([current_cost, theta]) \n", - " return theta, logs\n", - "\n", - "def plot_data(X, y, xlabel, ylabel):\n", - " \"\"\"Wykres danych (wersja macierzowa)\"\"\"\n", - " fig = plt.figure(figsize=(16*.6, 9*.6))\n", - " ax = fig.add_subplot(111)\n", - " fig.subplots_adjust(left=0.1, right=0.9, bottom=0.1, top=0.9)\n", - " ax.scatter([X[:, 1]], [y], c='r', s=50, label='Dane')\n", - " \n", - " ax.set_xlabel(xlabel)\n", - " ax.set_ylabel(ylabel)\n", - " ax.margins(.05, .05)\n", - " plt.ylim(y.min() - 1, y.max() + 1)\n", - " plt.xlim(np.min(X[:, 1]) - 1, np.max(X[:, 1]) + 1)\n", - " return fig\n", - "\n", - "def plot_regression(fig, fun, theta, X):\n", - " \"\"\"Wykres krzywej regresji (wersja macierzowa)\"\"\"\n", - " ax = fig.axes[0]\n", - " x0 = np.min(X[:, 1]) - 1.0\n", - " x1 = np.max(X[:, 1]) + 1.0\n", - " L = [x0, x1]\n", - " LX = np.matrix([1, x0, 1, x1]).reshape(2, 2)\n", - " ax.plot(L, fun(theta, LX), linewidth='2',\n", - " label=(r'$y={theta0:.2}{op}{theta1:.2}x$'.format(\n", - " theta0=float(theta[0][0]),\n", - " theta1=(float(theta[1][0]) if theta[1][0] >= 0 else float(-theta[1][0])),\n", - " op='+' if theta[1][0] >= 0 else '-')))" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "# Wczytanie danych (mieszkania) przy pomocy biblioteki pandas\n", - "\n", - "alldata = pandas.read_csv('data_flats_with_outliers.tsv', sep='\\t',\n", - " names=['price', 'isNew', 'rooms', 'floor', 'location', 'sqrMetres'])\n", - "data = np.matrix(alldata[['price', 'sqrMetres']])\n", - "\n", - "m, n_plus_1 = data.shape\n", - "n = n_plus_1 - 1\n", - "Xn = data[:, 0:n]\n", - "\n", - "Xo = np.matrix(np.concatenate((np.ones((m, 1)), Xn), axis=1)).reshape(m, n + 1)\n", - "yo = np.matrix(data[:, -1]).reshape(m, 1)\n", - "\n", - "Xo /= np.amax(Xo, axis=0)\n", - "yo /= np.amax(yo, axis=0)" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmwAAAFoCAYAAADq7KeuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcvElEQVR4nO3de7CkZ10n8O9vkpDozBQBEgyEZNHKLIgXAnskQVK7iFILs2zCrlgT19KAwYhcRLmsQSxBtMq4LiJCECNyiWvhWGhhdAOIXMQUBphgAiQxO2NcYZwRwmXjyZALk3n2j+5xTk56Jj3n9Ol+Ts/nU3Wqu9/3ed/3d97uPvOd57081VoLAAD92jDrAgAAODKBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6N7PAVlVnVNVHqurmqrqxql46ok1V1W9V1a6q+kxVPXEWtQIAzNLxM9z2/iQvb619uqo2J7muqj7YWrtpSZtnJtky/DknyW8PHwEAjhkz62Frre1trX16+Hwxyc1JTl/W7IIkV7aBa5OcXFWPmHKpAAAz1cU5bFX16CRPSPKJZbNOT/KFJa935/6hDgBgrs3ykGiSpKo2JfnjJD/TWvuX5bNHLHK/sbSq6pIklyTJxo0b/91jH/vYidcJALAa11133Zdba6euZNmZBraqOiGDsPYHrbU/GdFkd5Izlrx+VJI9yxu11q5IckWSLCwstB07dqxBtQAAK1dV/7jSZWd5lWgl+b0kN7fWfuMwza5K8mPDq0XPTXJ7a23v1IoEAOjALHvYnpLkR5N8tqquH077+SRnJklr7a1Jrk6yNcmuJF9P8rwZ1AkAMFMzC2yttWsy+hy1pW1akhdNpyIAgD51cZUoAACHJ7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOjcTANbVb29qr5UVZ87zPynVtXtVXX98OcXp10jAMCsHT/j7b8zyZuTXHmENn/dWnvWdMoBAOjPTHvYWmsfS/LVWdYAANC79XAO25Or6oaqel9VfcesiwEAmLZZHxJ9IJ9O8m9aa3dU1dYk702yZXmjqrokySVJcuaZZ063QgCANdZ1D1tr7V9aa3cMn1+d5ISqOmVEuytaawuttYVTTz116nUCAKylrgNbVZ1WVTV8/qQM6v3KbKsCAJiumR4Srap3J3lqklOqaneS1yQ5IUlaa29N8pwkP1VV+5PcmeTC1lqbUbkAADMx08DWWvvhB5j/5gxu+wEAcMzq+pAoAAACGwBA9wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzs00sFXV26vqS1X1ucPMr6r6raraVVWfqaonTrtG6NbiYvK2tyU/93ODx8XFWVcEwBo5fsbbf2eSNye58jDzn5lky/DnnCS/PXyEY9s11yRbtyYHDiT79iUbNyYve1ly9dXJeefNujoAJmymPWyttY8l+eoRmlyQ5Mo2cG2Sk6vqEdOpDjq1uDgIa4uLg7CWDB4PTr/jjtnWB8DE9X4O2+lJvrDk9e7hNDh2bd8+6Fkb5cCBwXwA5krvga1GTGv3a1R1SVXtqKodt9122xTKghnaufNQz9py+/Ylu3ZNtx4A1lzvgW13kjOWvH5Ukj3LG7XWrmitLbTWFk499dSpFQczsWXL4Jy1UTZuTM46a7r1ALDmeg9sVyX5seHVoucmub21tnfWRcFMbduWbDjMV3fDhsF8AObKTK8Srap3J3lqklOqaneS1yQ5IUlaa29NcnWSrUl2Jfl6kufNplLoyObNg6tBl18lumHDYPqmTbOuEIAJm2lga6398APMb0leNKVyYP0477xkz57BBQa7dg0Og27bJqwBzKlZ34cNWKlNm5KLL551FQBMQe/nsAEAHPMENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6d/ysCwBWaXEx2b492bkz2bIl2bYt2bx51lUBMEECG6xn11yTbN2aHDiQ7NuXbNyYvOxlydVXJ+edN+vqAJgQh0RhvVpcHIS1xcVBWEsGjwen33HHbOsDYGIENlivtm8f9KyNcuDAYD4Ac0Fgg/Vq585DPWvL7duX7No13XoAWDMCG6xXW7YMzlkbZePG5KyzplsPAGtGYIP1atu2ZMNhvsIbNgzmAzAXBDZYrzZvHlwNunnzoZ62jRsPTd+0abb1ATAxbusB69l55yV79gwuMNi1a3AYdNs2YQ1gzghssN5t2pRcfPGh14uLydve5ka6AHNEYIN54ka6AHPJOWwwL9xIF2BuCWwwL9xIF2BuCWwwL9xIF2BuCWwwL9xIF2BuCWwwL9xIF2BujX2VaFU9JMmWJCcdnNZa+9haFAWswMEb5i6/SnTDBjfSBVjnxgpsVfX8JC9N8qgk1yc5N8nfJHna2pUGHDU30gWYS+P2sL00yfckuba19n1V9dgkv7R2ZQErtvxGugCse+Oew3ZXa+2uJKmqE1trf5fkMWtXFgAAB43bw7a7qk5O8t4kH6yqryXZs3ZlASu2uDg4JGpoKoC5Ua21o1ug6j8keXCS97fW7lmTqlZhYWGh7dixY9ZlwGyMGprq4EUHhqYCmKmquq61trCSZce+rUdVHVdVj0zyDxlceHDaSja4bJ3PqKpbqmpXVV06Yv5zq+q2qrp++PP81W4T5pahqQDm1rhXib4kyWuSfDHJwbFvWpLvXumGq+q4JJcneXqS3Uk+VVVXtdZuWtZ0e2vtxSvdDsy1pYc///mfk298Y3S7b3xj0M7FCADr0tFcJfqY1tpXJrjtJyXZ1Vq7NUmq6g+TXJBkeWADRll++PP445P9+0e3veuu5CZfLYD1atxDol9IcvuEt336cL0H7R5OW+4Hq+ozVfWeqjpj1Iqq6pKq2lFVO2677bYJlwkdGnX483Bh7aCvTPL/WwBM07g9bLcm+WhV/e8kdx+c2Fr7jVVsu0ZMW34FxJ8leXdr7e6qekGSd2XEzXpba1ckuSIZXHSwippgfdi+fdCzdjQe9rC1qQWANTduYPv88OdBw59J2J1kaY/Zo7LsViHLDsH+bpJfm9C2YX3bufNQz9o4Tjopedzj1q4eANbUWIGttfZLSVJVG1trR/GvxBF9KsmWqvrWJP+U5MIk/21pg6p6RGtt7/Dl+UluntC2YX3bsiU57rjk3nvHa3/CCQZ/B1jHxjqHraqeXFU3ZRiYqurxVfWW1Wy4tbY/yYuTfGC43j9qrd1YVa+rqvOHzX66qm6sqhuS/HSS565mmzA3tm4dL6xt3HhoUHjjiQKsW+MeEv3NJP8xyVVJ0lq7oar+/Wo33lq7OsnVy6b94pLnr0ryqtVuB+bOG9/4wG2qkksuSV73OmENYJ0b+8a5rbUvLJs05rEYYKIWF5Pf/M0Hbtda8oY3JB//+NrXBMCaGvu2HlX1vUlaVT2oql4R55PBbGzfPv65a0ly/vlGOQBY58YNbC9I8qIM7pO2O8nZSV64VkUBR/C3f3t0gW3//kHIA2DdGjewvT7Ji1tr39Jae3iSlyT5n2tXFnBYH/nI0bW/997kk59cm1oAmIpxA9t3t9a+dvDF8PkT1qYk4LAWF5ObV3A2whVXJG9Z1YXdAMzQuIFtQ1U95OCLqnpoxr/CFJiU3/mdlS/7ohcNBogHYN05mkOiH6+qX66q1yX5eJL/sXZlASO96U2rW/7SSydTBwBTNe5IB1dW1Y4MxvGsJP+1tXbTmlYG3NfiYvL5z69uHbfcMplaAJiqsQ9rDgOakAazMokrPR/zmNWvA4CpG/vGucCM7dy5+nVcdtnq1wHA1AlssF6cccbqlv+RH0lOO20ytQAwVQIbrAfXXLO6CwaOOy5561snVw8AUyWwQe8WF5OtW5N9+1a+jle8wgDwAOuYwAa92749OXBg5ctv2pT8wi9Mrh4Apk5gg97t3Lny3rWNG5P3vU/vGsA6J7BB77ZsGQSvlbjgguS88yZbDwBTJ7BB77ZtSzas8Kt6662TrQWAmRDYoHebNydXX72yZU8+ebK1ADATAhusB49//MqW+6u/Su64Y7K1ADB1AhusB89+9sqW+8Y3JjOkFQAzJbDBevDhD69suf37k1277j99z57koouSc84ZPO7Zs7r6AFhTAhvMu9bu+/otb0lOPz258srkk58cPJ5++mA6AF2qtvyP+Tq3sLDQduzYMesyYLKqVr7spk3J3r2Dxz17BuHscPbuNd4owBqpqutaawsrWVYPG8y7e+45dB7bq1515LarGa8UgDVz/KwLAB7Aas8vu+ee5PrrB8//7u+O3PaWW1a3LVipxcXBfyx27hzcLHrbtsEtbYAkAhv076UvXf06Pv3pweNjHzs4b+1wHvOY1W8LjtY11yRbtw7GzN23bzCyx8teNrj/oJE6IIlDotC/lV4hutRddw0ef/VXj9zusstWvy04GouLg7C2uHhozNx9+w5Ndx9BSCKwQf/uvnv16zjppMHjIx+ZXH756DaXX+6CA6Zv+/ZBz9ooBw64jyAMCWzQuyNd1TmuJz7x0PMXvnBwNehFFyXnnjt43Lt3MB2mbefOQz1ry+3bN/o+gnAMcg4b9O4nfiJ55StXvnxVcvbZ95122mnJO9+5qrJgIrZsGZyzNiq0bdyYnHXW9GuCDulhg9795E+ubvnjjhtccQc92rYt2XCYf4o2bPDZhSGBDXq3uLi65d/znsFNc6FHmzcPrgbdvHnQo5YMHg9O99mFJA6JQv9e8YqVL/v61ycXXDC5WmAtnHfe4H6D27cPzlk766xBz5qwBv9KYIPe/c3frHzZJz1pcnXAGmobNyY//uOHXrckB+47dOKogRRHDa84ut2IaSNajjta4/J2465r0r/DqIYrrWXS+3LMSSt+b8belyt8T8et42hqWY25C2z/98v78tx3HLox6KR37Kre0DGWHbfeSX5JB+1W9kUdXdva/wGd5Ps6jS/p+O/hiGn/6TXJPd849HrEsKIt95/YqpI/+D/JNXcnG2pY26h6RxU3qt3k3tfx9+XISsbc5qj1rfBzPun3dI2/Iz3/zQBWZu4C2+Ld+/PRW26bdRkwOZtOXd3y/++uydQBU1Yj/nMyYlJqRMPR7Uatb7yNjLO+Ueua9O8wfm0rq2U1+3JUy/F//1HtHnh94/7uo4zc5grrGLeWfxyrstHmLrA9+mHfnLc+93vuO3FGX/px39DlEyf+pV/FF2acL+Cs9uUoK/7ST/xLOrK6la3vmc9M3XrrfduM6LaoEf0b1Vryghckr/zvyY4dycU/nrr33uTOO5Nv+qbBVXjveEeysDBebRN8X1f6/Ri1rqNZ30p/h7H/Ye/o780k1zXx32HcLzXMkVrFHZrmLrBtPumEfN9jHz7rMmByvrI7+doKB4A/8cTkhk8k73578qpX3XeYn9uHj8951uCEbyd4A3TLbT2gd09+8sqXvfvu5P3vT17+8sOPyWj4H4DuCWzQu9e+dvXruOeew88z/A9A9wQ26N1f/MXart/wPwDdE9igd3/+52u7fsP/AHRv7i46AB7AiScOzm3buHEQ1gz/A9C9mfawVdUzquqWqtpVVZeOmH9iVW0fzv9EVT16+lXCjD3rWZNb16ZNg+GqLr00eeMbB1eHnnfe5NYPwJqYWQ9bVR2X5PIkT0+yO8mnquqq1tpNS5pdnORrrbWzqurCJL+WxLEbji0XXZS85CVHv9yGDYN7re3bd9/eNAENYN2Z5SHRJyXZ1Vq7NUmq6g+TXJBkaWC7IMlrh8/fk+TNVVVt0gN0Qc82b17ZchdemDztaQbTBpgDswxspyf5wpLXu5Occ7g2rbX9VXV7kocl+fLSRlV1SZJLkuTMM89cq3phfXn965PTTpt1FQBMwCzPYRs1LsnynrNx2qS1dkVrbaG1tnDqqascdxHmwTnnCGsAc2SWgW13kjOWvH5UkuXj7/xrm6o6PsmDk3x1KtVBT37lV46u/XvfuzZ1ADATswxsn0qypaq+taoelOTCJFcta3NVkouGz5+T5MPOX+OY9OpXDy4gGMfll+tdA5gzMwtsrbX9SV6c5ANJbk7yR621G6vqdVV1/rDZ7yV5WFXtSvKyJPe79QccM77+9eTXf/3+0086KfnO7xxcTbp3b/LCF06/NgDWVM1bh9XCwkLbsWPHrMsAALiPqrqutbawkmUNTQUA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA52YS2KrqoVX1waraOXx8yGHa3VtV1w9/rpp2nQAAPZhVD9ulST7UWtuS5EPD16Pc2Vo7e/hz/vTKAwDox6wC2wVJ3jV8/q4kz55RHQAA3ZtVYPuW1treJBk+Pvww7U6qqh1VdW1VHTbUVdUlw3Y7brvttrWoFwBgZo5fqxVX1V8mOW3ErFcfxWrObK3tqapvS/Lhqvpsa+3vlzdqrV2R5IokWVhYaCsqGACgU2sW2FprP3C4eVX1xap6RGttb1U9IsmXDrOOPcPHW6vqo0mekOR+gQ0AYJ7N6pDoVUkuGj6/KMmfLm9QVQ+pqhOHz09J8pQkN02tQgCATswqsF2W5OlVtTPJ04evU1ULVfW2YZtvT7Kjqm5I8pEkl7XWBDYA4JizZodEj6S19pUk3z9i+o4kzx8+/3iS75pyaQAA3THSAQBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHMCGwBA5wQ2AIDOCWwAAJ0T2AAAOiewAQB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDYAgM4JbAAAnRPYAAA6J7ABAHROYAMA6JzABgDQOYENAKBzAhsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDoXLXWZl3DRFXVbUn+cdZ1HGNOSfLlWRdxDLG/p8v+ni77e7rs7+l6TGtt80oWPH7Slcxaa+3UWddwrKmqHa21hVnXcaywv6fL/p4u+3u67O/pqqodK13WIVEAgM4JbAAAnRPYmIQrZl3AMcb+ni77e7rs7+myv6drxft77i46AACYN3rYAAA6J7Bx1KrqoVX1waraOXx8yGHa3VtV1w9/rpp2netdVT2jqm6pql1VdemI+SdW1fbh/E9U1aOnX+X8GGN/P7eqblvymX7+LOqcB1X19qr6UlV97jDzq6p+a/hefKaqnjjtGufJGPv7qVV1+5LP9i9Ou8Z5UlVnVNVHqurmqrqxql46os1Rf8YFNlbi0iQfaq1tSfKh4etR7mytnT38OX965a1/VXVcksuTPDPJ45L8cFU9blmzi5N8rbV2VpI3JPm16VY5P8bc30myfcln+m1TLXK+vDPJM44w/5lJtgx/Lkny21OoaZ69M0fe30ny10s+26+bQk3zbH+Sl7fWvj3JuUleNOLvyVF/xgU2VuKCJO8aPn9XkmfPsJZ59aQku1prt7bW7knyhxns96WWvg/vSfL9VVVTrHGejLO/mZDW2seSfPUITS5IcmUbuDbJyVX1iOlUN3/G2N9MUGttb2vt08Pni0luTnL6smZH/RkX2FiJb2mt7U0GH8wkDz9Mu5OqakdVXVtVQt3ROT3JF5a83p37f+H/tU1rbX+S25M8bCrVzZ9x9neS/ODw8MV7quqM6ZR2TBr3/WBynlxVN1TV+6rqO2ZdzLwYnqryhCSfWDbrqD/jczfSAZNRVX+Z5LQRs159FKs5s7W2p6q+LcmHq+qzrbW/n0yFc29UT9nyS7rHacN4xtmXf5bk3a21u6vqBRn0bj5tzSs7NvlsT9enk/yb1todVbU1yXszOFTHKlTVpiR/nORnWmv/snz2iEWO+BkX2BiptfYDh5tXVV+sqke01vYOu3C/dJh17Bk+3lpVH83gfxkC23h2J1nag/OoJHsO02Z3VR2f5MFx2GOlHnB/t9a+suTl78Y5g2tpnM8/E7I0TLTWrq6qt1TVKa01Y4yuUFWdkEFY+4PW2p+MaHLUn3GHRFmJq5JcNHx+UZI/Xd6gqh5SVScOn5+S5ClJbppahevfp5JsqapvraoHJbkwg/2+1NL34TlJPtzcWHGlHnB/Lzu/5PwMzkthbVyV5MeGV9Kdm+T2g6dhMHlVddrB81+r6kkZZIOvHHkpDme4L38vyc2ttd84TLOj/ozrYWMlLkvyR1V1cZLPJ/mhJKmqhSQvaK09P8m3J/mdqjqQwZf/staawDam1tr+qnpxkg8kOS7J21trN1bV65LsaK1dlcEfhN+vql0Z9KxdOLuK17cx9/dPV9X5GVwB9tUkz51ZwetcVb07yVOTnFJVu5O8JskJSdJae2uSq5NsTbIrydeTPG82lc6HMfb3c5L8VFXtT3Jnkgv9529VnpLkR5N8tqquH077+SRnJiv/jBvpAACgcw6JAgB0TmADAOicwAYA0DmBDQCgcwIbAEDnBDaAEarq7OFd31e6/McnWQ9wbBPYAEY7O4P7JN3PcGSJI2qtfe/EKwKOWe7DBsyt4cDL709yTZJzk9yQ5B1JfinJw5P8SJIbk7wpyXdlcDPx1yZ5XwY3tPymJP+U5FczuBn0I5M8OsmXM7gR5u8n2Tjc3Itbax8f3mz3/OG0hya5rrX2X9bslwSOCQIbMLeGgW1XBuPY3pjBEFQ3JLk4g1D1vAyGTLuptfa/qurkJJ8ctv+hJAuttRcP1/XaJP85yXmttTur6puTHGit3VVVWzIYGH5hybY3JvlYkp9trX1sCr8uMMcMTQXMu39orX02SarqxiQfaq21qvpsBr1lj0pyflW9Ytj+pAyHkBnhqtbancPnJyR5c1WdneTeJP92Wdt3JHmnsAZMgsAGzLu7lzw/sOT1gQz+Bt6b5Adba7csXaiqzhmxrn1Lnv9ski8meXwG5wPftWTZVyf5emvtTauuHiAuOgD4QJKXVFUlSVU9YTh9McnmIyz34CR7W2sHMhjo+bjh8lszONz6gjWrGDjmCGzAse6XMzi8+Zmq+tzwdZJ8JMnjqur6qto2Yrm3JLmoqq7N4HDowd63VyY5Lcm1w2XfsLblA8cCFx0AAHRODxsAQOcENgCAzglsAACdE9gAADonsAEAdE5gAwDonMAGANA5gQ0AoHP/H8NZqxKSv1A/AAAAAElFTkSuQmCC\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "fig = plot_data(Xo, yo, xlabel=u'metraż', ylabel=u'cena')\n", - "theta_start = np.matrix([0.0, 0.0]).reshape(2, 1)\n", - "theta, logs = gradient_descent(cost, gradient, theta_start, Xo, yo, alpha=0.01)\n", - "plot_regression(fig, h_linear, theta, Xo)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "Na powyższym przykładzie obserwacja odstająca jawi sie jako pojedynczy punkt po prawej stronie wykresu. Widzimy, że otrzymana krzywa regresji zamiast odwzorowywać ogólny trend, próbuje „dopasować się” do tej pojedynczej obserwacji.\n", - "\n", - "Dlatego taką obserwację należy usunąć ze zbioru danych (zobacz ponizej)." - ] - }, - { - "cell_type": "code", - "execution_count": 21, - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [], - "source": [ - "# Odrzućmy obserwacje odstające\n", - "alldata_no_outliers = [\n", - " (index, item) for index, item in alldata.iterrows() \n", - " if item.price > 100 and item.sqrMetres > 10]\n", - "\n", - "alldata_no_outliers = alldata.loc[(alldata['price'] > 100) & (alldata['sqrMetres'] > 100)]" - ] - }, - { - "cell_type": "code", - "execution_count": 22, - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "outputs": [], - "source": [ - "data = np.matrix(alldata_no_outliers[['price', 'sqrMetres']])\n", - "\n", - "m, n_plus_1 = data.shape\n", - "n = n_plus_1 - 1\n", - "Xn = data[:, 0:n]\n", - "\n", - "Xo = np.matrix(np.concatenate((np.ones((m, 1)), Xn), axis=1)).reshape(m, n + 1)\n", - "yo = np.matrix(data[:, -1]).reshape(m, 1)\n", - "\n", - "Xo /= np.amax(Xo, axis=0)\n", - "yo /= np.amax(yo, axis=0)" - ] - }, - { - "cell_type": "code", - "execution_count": 23, - "metadata": { - "scrolled": true, - "slideshow": { - "slide_type": "subslide" - } - }, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmwAAAFoCAYAAADq7KeuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfZDkVX3v8c93nndnengIq7ssbJBiFbEUMBOBuOXlqiSwZSBX0cWbiosXiktYooJaQrzXx1hiKvgMmBXJAjG4CaZ0E1cpHy/ZKOhAQFxgYUJKWWfFFXCnZ3aeeuZ7//j9eqen59c93TPd/TvT/X5VdfXD79fdZ6anlw/nfM855u4CAABAuNrSbgAAAADKI7ABAAAEjsAGAAAQOAIbAABA4AhsAAAAgSOwAQAABC61wGZmJ5rZ983sMTPba2bvTDjHzOyzZjZkZj81s1em0VYAAIA0daT43jlJ73b3B80sI+kBM/u2uz9acM4FkjbGl7Mk3RJfAwAAtIzUetjc/YC7Pxjfzkp6TNL6otMuknSHR+6TdLSZrWtwUwEAAFIVRA2bmZ0k6UxJ9xcdWi/p6YL7+7Uw1AEAADS1NIdEJUlm1ifpq5Le5e4jxYcTnrJgLy0zu0LSFZLU29v7e6eeemrN2wkAALAcDzzwwG/cfc1SnptqYDOzTkVh7cvu/s8Jp+yXdGLB/RMkDRef5O7bJW2XpIGBAR8cHKxDawEAAJbOzH6+1OemOUvUJH1J0mPu/skSp+2S9LZ4tujZkg65+4GGNRIAACAAafawvVrSn0l6xMweih/7S0kbJMndvyBpt6TNkoYkHZb09hTaCQAAkKrUApu771FyjVrhOS5pW2NaBAAAEKYgZokCAACgNAIbAABA4AhsAAAAgSOwAQAABI7ABgAAEDgCGwAAQOAIbAAAAIEjsAEAAASOwAYAABA4AhsAAEDgCGwAAACBI7ABAAAEjsAGAAAQOAIbAABA4AhsAAAAgSOwAQAABI7ABgAAEDgCGwAAQOAIbAAAAIEjsAEAAASOwAYAABA4AhsAAEDgCGwAAACBI7ABAAAEjsAGAAAQOAIbAABA4AhsAAAAgSOwAQAABI7ABgAAEDgCGwAAQOAIbAAAAIEjsAEAAASOwAYAABA4AhsAAEDgCGwAAACBI7ABAAAEjsAGAAAQOAIbAABA4AhsAAAAgSOwAQAABI7ABgAAEDgCGwAAQOAIbAAAAIEjsAEAAASOwAYAABA4AhsAAEDgCGwAAACBI7ABAAAELtXAZma3mdmvzexnJY6fa2aHzOyh+PKBRrcRAAAgbR0pv/8OSZ+XdEeZc/7N3d/QmOYAAACEJ9UeNne/V9JzabYBAAAgdCuhhu0cM3vYzL5pZi9LOsHMrjCzQTMbPHjwYKPbBwAAUFehB7YHJf2uu58u6XOSvpZ0krtvd/cBdx9Ys2ZNQxsIAABQb0EHNncfcffR+PZuSZ1mdlzKzQIAAGiooAObma01M4tvv0pRe59Nt1UAAACNleosUTO7S9K5ko4zs/2SPiipU5Lc/QuSLpb052aWkzQu6RJ395SaCwAAkIpUA5u7v3WR459XtOwHAABAywp6SBQAAAAENgAAgOAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAApdqYDOz28zs12b2sxLHzcw+a2ZDZvZTM3tlo9sIIGXZrHTrrdL73hddZ7NptwgAGq4j5fffIenzku4ocfwCSRvjy1mSbomvAbSCPXukzZul2VlpbEzq7ZWuvVbavVvatCnt1gFAw6Taw+bu90p6rswpF0m6wyP3STrazNY1pnUAUpXNRmEtm43CmhRd5x8fHU23fQDQQKHXsK2X9HTB/f3xYwCa3c6dUc9aktnZ6DgAtIjQA5slPOYLTjK7wswGzWzw4MGDDWgWgLp78sm5nrViY2PS0FBj2wMAKQo9sO2XdGLB/RMkDRef5O7b3X3A3QfWrFnTsMYBqKONG6OatSS9vdIppzS2PQCQotAD2y5Jb4tni54t6ZC7H0i7UQAaYMsWqa3EP1FtbdFxAGgRqc4SNbO7JJ0r6Tgz2y/pg5I6JcndvyBpt6TNkoYkHZb09nRaCqDhMploNmjxLNG2tujxvr60WwgADZNqYHP3ty5y3CVta1BzAIRm0yZpeDiaYDA0FA2DbtlCWAPQctJehw0Ayuvrky67LO1WAECqQq9hAwAAaHkENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAteRdgMAYEXIZqWdO6Unn5Q2bpS2bJEymbRbBaBFENgAYDF79kibN0uzs9LYmNTbK117rbR7t7RpU9qtA9ACGBIFgHKy2SisZbNRWJOi6/zjo6Pptg9ASyCwAUA5O3dGPWtJZmej4wBQZxUPiZrZMZI2SurJP+bu99ajUQAQjCefnOtZKzY2Jg0NNbY9AFpSRYHNzC6X9E5JJ0h6SNLZkn4k6bX1axoABGDjxqhmLSm09fZKp5zS+DYBaDmVDom+U9LvS/q5u/93SWdKOli3VgFAKLZskdpK/FPZ1hYdB4A6qzSwTbj7hCSZWbe7Py7pJfVrFgAEIpOJZoNmMlGPmhRd5x/v60u3fQBaQqU1bPvN7GhJX5P0bTN7XtJw/ZoFAAHZtEkaHo4mGAwNRcOgW7YQ1gA0jLl7dU8w+2+SjpL0LXefqkurlmFgYMAHBwfTbgYAAMA8ZvaAuw8s5bnVzBJtl/RCSf8VP7RW0i+W8qYAAACoXKWzRP9C0gclPSMpvyCRS3pFndoFAAgF23IBqau0h+2dkl7i7s/WszEAgMCwLRcQhEpniT4t6VA9GwIACAzbcgHBqLSH7SlJPzCzb0iazD/o7p+sS6sAAOmrZFuuyy5rbJuAFlVpYPtFfOmKLwCAZse2XEAwKgps7v5hSTKzXncv8e0FADQVtuUCglFRDZuZnWNmj0p6LL5/upndXNeWAQDSxbZcQDAqnXTwaUl/JOlZSXL3hyW9pl6NAgAEgG25gGBUvHCuuz9tZoUPzdS+OQCAoLAtFxCESgPb02b2B5LczLokvUPx8CgAoMn19TEbFEhZpUOiV0raJmm9pP2SzpB0Vb0aBQAAgDmV9rDdKOlqd39ekszsmPix/1WvhgEAAsL2VECqKg1sr8iHNUly9+fN7Mw6tQkAEBK2pwJSV+mQaFvcqyZJMrNjVcWEhVLM7Hwz22dmQ2Z2XcLxS83soJk9FF8uX+57AgCqwPZUQBCqGRL9oZndLcklvUXSx5bzxmbWLukmSecpqov7iZntcvdHi07d6e5XL+e9AABLxPZUQBAq3engDjMblPRaSSbpjQnBqlqvkjTk7k9Jkpl9RdJFkpb7ugBaGbVWtcX2VEAQqlmH7VHVNkytl/R0wf39ks5KOO9NZvYaSU9Iusbdn044BwDqW2s1PCxdf730+OPSqadKH/+4dPzxtWl3yE48sfzxE05oTDuAFldpDVs9WMJjXnT/XySd5O6vkPQdSbcnvpDZFWY2aGaDBw8erHEzAawI9ay1uvlmaf166Y47pB//OLpevz56HAAaIM3Atl9S4f+6nSBpuPAEd3/W3Sfju1+U9HtJL+Tu2919wN0H1qxZU5fGAghcJbVWSzE8LG3blnxs2zbpV79a2uuuFE8vMqixf39j2gG0uGXP9FyGn0jaaGYvkvRLSZdI+p+FJ5jZOnc/EN+9UOyuACCvuFZt79761Fpdf33549ddJ+3YsbTXXgk2boyGlpN+t7290VZVQIubzM1oZDyn7MS0RiZyGhmf1sjEtLIFt0fGc8t6j9QCm7vnzOxqSfdIapd0m7vvNbOPSBp0912S3mFmF0rKSXpO0qVptRdAQJJq1XI5qadHmphYeP5ygsXjj5c/vm/f0l53pdiyJaoDTNLWFh0HVjB31/h0YeCKwtVIUfgqF8gmcyV692sozR42uftuSbuLHvtAwe3rJS3yv7cAglbrWZuFtWp5pXrW8pYTLE49NapbK+UlL1na6y5Vo2fBZjLRpI3igNzWFj3OJvBI2eysa3QqN9ebNT4XqhIDVkEgyz8nN1tcQl+djjZT/6pO9fd0xNedyvR0qL+nU/2rOo7cf/snlv4e5r68RoZmYGDABwcH024GACm5Jyz/H/qlztq89VbpXe9KDmk9PdF1e3vt3m94OJpgUMqBA9LatUt77WqV+32efnp9g9zoaPT6Q0NRb+WWLYQ11ERuZlbZiVxBmJou6t3Kh6/C43Fv1/i0spM5LTfK9HS2KdOTELji2/2rOhYcL7zd09kms6S5lPOZ2QPuPrCUNhLYANRHNhsFncKesLx8j9ff/E31S2O8733SX/916ePXXiuddlptg8XNNydPPLjpJumqq5b32pUq9/tcvTr6nbrXLqgCFZrMzRTUapWu3yo1nDg2NbPsNvR2tc8LV6UD18LjmZ4OdXe01+A3sbjlBLZUh0QBNLHFZm3edVd0qTb0LFYEf9pp0cr7+aHDj350rscp365qe6Guukp64xujCQb79kXDoDfc0LieNan87/Pw4fn387+b886TrrxSetnLWEAYifL1W8Xhqrh+qziQFYav5dZvmUmZ7srCVeFj+dt93R3qaE9z0YvGoIcNQOWqqZ9arCesUCXDivn33rtXuuUWaXJy4TmZTDSE+dBDC4cO3aNLW9vK7IWq5vdZbKX9rKjY7KxrbCo3F66KhgsXDCcm9HY1qn5rLnAV3u9Qb1eH2toWH05sBvSwAai/ancROPFEqatLmppa/LUXWxqj+L3ztWr5WaGFgcS9skkJ+cc2b45CXuj1WOV6Fhez0n7WFpKbmdXoZG6uVysfshbUas09VljLVYv6re6OtnmBq3QtV8F1HLyqqd/C8hDYACyu3MzM171OuvFGaevWud62PXui9csqCWuS9GiZXe+S3rtw6Y58zVq+Vu3WW0sPHSZZKRuYl1teo1KN/llbYDuv4vqt4nCVPJw4d7we9VvVFM83sn4Ly0NgA7C4cvVTU1PSu98tvec90pveJJ1zTvQf6Wq2gvqP/4hCXlJPXbn3bm+fq1nLK7dZeZKVsoF54fIauZw0Ph4V/7S3R5ekIeJijfxZiydq5Lf0auREjUW4uyamZ5c0M7Fe9VvVzExspfotENgAFCpVo7ZYCMr3pP3DP0h33115z1peLie99rXSJz85v6dOKv/eSQGk2qHDlbRa/6ZN0oc+FAVkKRr+zeWiS1eX1NlZ/ueu98+a//t58MGozjDJtm3RBI4aTNgort+aXzif0OOV0NtV6/qtI7Vbi/V2tWD9FpaHSQcAIkk1amZRb8jDD0s/+EFlvTjL0d0dBY/Curhy66719kqf+cz8HrZyy18kyU9UWAl1XYutCXfjjdIvfxn1YpWblFGPn7X476ecrVulHTsW1m+VrNXKL3I6//joZE7LzFvz6rcyPZ3J4Wte3VbHvML5VZ3t1G+hYqzDVoDABixBtSGn3gqDxfCwdPLJ1QWQpPC50meJSlHQueOO8sd37Fh8weIa7ZYwlYuHE3/zW2Vf/0camTGNdPdqpKdPI929ynav1kh3n0Z6euP7vdHtzLEa6T+m5vVbxcXxmaLi+KTj1G+hkZglCmB5ytWJpWFmRvrDP5R++9soVLQX/Ue1pyca/iu1NdKmTVGQK16ZX1rZq/VXuq9pqZ+/r+9ImPPZWU1M5jRy7BqNfPRGjXz+Fo28+LSKZibmj09MF/zNXPyx6n6WqZl59VvVzkzM9HQo00P9FloHgQ1A9YX69Xb4sPSjH83dz+UWnvPEE+XroPI9arOzc7czmfBng5ZTsK/prExjXT0a6e6LerJ6+jTyinM18uD+uYC19hyNHPX70XDiV/ZqZGxCI489qZFLv6Bs92pNt3fOvfa/j0n//pOqmpOv38qMHlL/r/arf3JM/ZNjykyOqX8iut0/MbrwsT3/T5kN69RH/RZQMQIb0OyyWen226V//dfo/hvesLCwfzlrfKWhvV36xjfmwlfxEN+GDdGM1enpaCi1u1u65hrpm98MZvhzZtYXFMQXb0g9f5HTaY28fKtG/vdrlI2HG2fbEobz/vHh8m989LojN7unJ+fC1PSE+k/eoP5TTopruUrXb+WPH6nfuvVW6ab/U9nfz003SS/eUOVvCwA1bEAz27MnGlocH5//+OrV0j33zIWX0GrYKnHWWdLll0fh7OKL5+q1Vq9euFVT3urV0jPP1GQYNF+/lbSlT+nhxLlANjqZ0GtYpd7Jw8pMHY56rk54ofpftGHBCvPzarm+9Lfq/+ItykweVmZyTD0z0/Nf8LrrorXSqlXu76ejQzrzzGj5lUZv5wUEhho2AAtls9IFFywMa1IUaC64INoSqq8v6m0rXC6iHjo6koc2l+r++6VHHlkYzkqFtfyx22+XX3XVkfW3shPTOrRIuCpcKiJ/fF791hLk67eWMjOxf+R5ZT78f9Wx7/Hq9jXduFaaOiQdLjHjdqlLfhSuEVdqogOAZaGHDWhGTzwRBbKnnip/3sknS//0T9Kdd0qf/nR929TdHYW2Ggy7uqTRrlXRrMOCWYkjPb0FjxXNTOzujeq9evs10rVa07a8YvWONlu4yGl30VDi4H3K3LlD/ZOjR+q3MpNj6v/Yh9V31ZWNr98q1xNWiyU/RkdX9qQOoM5Y1qMAgQ0t79prpU99Ku1WzOnqii5f/Wo0dJnNasbalO1erWx3rw4dCVT54vneI0tBHFkWoiiMlazfqkJ3bioaTlxzrPqP7S/q3covfpq06nxR/VYpi62ZVsmG9/Ww2JIfAOqGwFaAwIaW9sQT0RBZnU21dczNTCzuzVrVp5Gu1RpZlVG2c5VGVmeic9Ycr2xnj0amZjXavXrZbVg9NR7PQhxT/+SoMpOHj9ye15tVMDsxP1NxXv1WvRaTrXTNtDTQEwakgho2AJFLL130FJc00dF9pDfr0JHerPywYdLQYtz7Fd+f6OxZehu7JfNZ9U0eTgxT0dIQo+ovCmCF4atv6rA6Z5e/6Kqk+m2IXumaaWno61vZy5sALYjABqww7q6xqZn5MxPHp5WdnNZI90kaOWdD6Vqu+P689beWoH125sj6WlGYOhyHrKQANjr/nIkx9U2Nq02B9O7Xa0P0gjXTEjWgJxRA8yCwAQ02M+sajWcaHhpP3pB6/qry82cuZiemS++feNafVtSGrtzUXJiajMPUxPzAtSB8FQSwVdOTaprlTuu1IfrHP15+SPSGG2r/ngCaFoENqNJUbjZeyDQ3P1AtWOg06Xht1t9a3dVeVAgfF8dPjKn/tu0LerwKA1fi+lutrK1tbtuqWjr++GiR2G3bFh676SbWIwNQFQIbWoq7azI3eyRIHVokXEW9XfOPL3f9LUnz1tqqZmZif0+n+no61Flu/8R3XyQ999yy29i08uvBFc6OrFfB/VVXSW98Y7Qg7b591a2ZBgAFCGxYUZLqt45s23Oklqt8b9f0zPJqp9rbbOEipyXC1YJAtqqz/vsnXn+99N731u/1i5lF+3SuBL290dIi69Y1bnbk2rXpzQYF0DRY1gMNVVy/Nb9WK1d6S59K6rcq1NXeFvVgFQ8nzrudtOp8dH911yLrb6Utm5WOO06amlp4rKsr2oczafeDpdq6Neqx2rlz8Z0M8r1aN9wQ9Trl1wJbtar6Nq1aFf0s7nPrieU3e5+YSH5OvZbwAIAKsA5bAQJbfRXXbyUWx5csnq9t/VZmkXCVFMgyPR3q6Vzegqsrwp490vnnR8FlZiYKNj090re+FR1//eujTdGXqzAAHTgQ7ZxQKiy95S3Rvqb5Xq3itcAOHUreGuuaa6LNxZMWej3jjIXriUnSX/1VtHiwWfRzsjgsgAAQ2AoQ2Eorrt8aWSRcZQtu54+PTy9/7at8sCre1qe4fmvBtj/xY2XrtzCn3OKoo6PS9u1R8Xvh9lVmUY3Xy18eBbCJCWl6OgpK7nPDn6UCUPEq+p2dUVj8+tejsLaYX/0qud5rKQu9sjgsgMAQ2Ao0c2ArrN9arGerVG/X1MzyCubz9VuZeSErOXwlBa6+7g61N3r/RJRXabCp9XkA0GIIbAVCDmyF9VsjpWq14sfm1XIVhK+a1W/1dCizqqhm68hwYlEgKxhuDL5+CwCAQLE1VYPk67fKhqsyxfPZGtVvZRYJV+Xqu1qifgsAgCbTMoFtfv1WhTMTi47XrX6rgpmJ+cep3wIAoPU0XWD75W/Hte3LDyYGrlrUbxUvclrpzMT+VdRvAQCApWm6wPbc2JS+8ciBxGNR/VYctObVbZVedb5wuJH6LQAAkIamC2zrj16lz7z1zMT6Luq3AADAStR0ge3Y3i798enHp90MAACAmqGCHQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHCpBjYzO9/M9pnZkJldl3C828x2xsfvN7OTGt9KAACAdKUW2MysXdJNki6QdJqkt5rZaUWnXSbpeXc/RdKnJH2isa0EAABIX5o9bK+SNOTuT7n7lKSvSLqo6JyLJN0e375b0uvMzBrYRgAAgNSlGdjWS3q64P7++LHEc9w9J+mQpN8pfiEzu8LMBs1s8ODBg3VqLgAAQDrSDGxJPWW+hHPk7tvdfcDdB9asWVOTxgEAAIQizcC2X9KJBfdPkDRc6hwz65B0lKTnGtI6AACAQKQZ2H4iaaOZvcjMuiRdImlX0Tm7JG2Nb18s6XvuvqCHDQAAoJl1pPXG7p4zs6sl3SOpXdJt7r7XzD4iadDdd0n6kqQ7zWxIUc/aJWm1FwAAIC2pBTZJcvfdknYXPfaBgtsTkt7c6HYBAACEhJ0OAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcKkENjM71sy+bWZPxtfHlDhvxsweii+7Gt1OAACAEKTVw3adpO+6+0ZJ343vJxl39zPiy4WNax4AAEA40gpsF0m6Pb59u6Q/SakdAAAAwUsrsL3Q3Q9IUnz9ghLn9ZjZoJndZ2aEOgAA0JI66vXCZvYdSWsTDr2/ipfZ4O7DZnaypO+Z2SPu/p8J73WFpCskacOGDUtqLwAAQKjqFtjc/fWljpnZM2a2zt0PmNk6Sb8u8RrD8fVTZvYDSWdKWhDY3H27pO2SNDAw4DVoPgAAQDDSGhLdJWlrfHurpK8Xn2Bmx5hZd3z7OEmvlvRow1oIAAAQiLQC2w2SzjOzJyWdF9+XmQ2Y2a3xOS+VNGhmD0v6vqQb3J3ABgAAWk7dhkTLcfdnJb0u4fFBSZfHt38o6eUNbhoAAEBw2OkAAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACR2ADAAAIHIENAAAgcAQ2AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBwBDYAAIDAEdgAAAACZ+6edhtqyswOSvp52u0IwHGSfpN2I1ocn0EY+BzSx2cQBj6H9L3E3TNLeWJHrVuSNndfk3YbQmBmg+4+kHY7WhmfQRj4HNLHZxAGPof0mdngUp/LkCgAAEDgCGwAAACBI7A1r+1pNwB8BoHgc0gfn0EY+BzSt+TPoOkmHQAAADQbetgAAAACR2BrEmZ2rJl928yejK+PKXHejJk9FF92NbqdzcjMzjezfWY2ZGbXJRzvNrOd8fH7zeykxreyuVXwGVxqZgcL/vYvT6Odzc7MbjOzX5vZz0ocNzP7bPw5/dTMXtnoNja7Cj6Dc83sUMF34QONbmOzM7MTzez7ZvaYme01s3cmnFP1d4HA1jyuk/Rdd98o6bvx/STj7n5GfLmwcc1rTmbWLukmSRdIOk3SW83stKLTLpP0vLufIulTkj7R2FY2two/A0naWfC3f2tDG9k6dkg6v8zxCyRtjC9XSLqlAW1qNTtU/jOQpH8r+C58pAFtajU5Se9295dKOlvStoR/k6r+LhDYmsdFkm6Pb98u6U9SbEsreZWkIXd/yt2nJH1F0WdRqPCzuVvS68zMGtjGZlfJZ4AGcPd7JT1X5pSLJN3hkfskHW1m6xrTutZQwWeAOnP3A+7+YHw7K+kxSeuLTqv6u0Bgax4vdPcDUvTHIukFJc7rMbNBM7vPzAh1y7de0tMF9/dr4RfzyDnunpN0SNLvNKR1raGSz0CS3hQPPdxtZic2pmkoUulnhfo6x8weNrNvmtnL0m5MM9vWxPoAAAQfSURBVItLYM6UdH/Roaq/C02300EzM7PvSFqbcOj9VbzMBncfNrOTJX3PzB5x9/+sTQtbUlJPWfHU60rOwdJV8vv9F0l3ufukmV2pqMfztXVvGYrxXUjfg5J+191HzWyzpK8pGpZDjZlZn6SvSnqXu48UH054StnvAoFtBXH315c6ZmbPmNk6dz8Qd6v+usRrDMfXT5nZDxQlfwLb0u2XVNhbc4Kk4RLn7DezDklHiSGLWlr0M3D3ZwvuflHUEaalku8L6qgwOLj7bjO72cyOc3f2GK0hM+tUFNa+7O7/nHBK1d8FhkSbxy5JW+PbWyV9vfgEMzvGzLrj28dJerWkRxvWwub0E0kbzexFZtYl6RJFn0Whws/mYknfcxZArKVFP4Oi2pALFdWUoPF2SXpbPEPubEmH8qUcaAwzW5uvoTWzVynKAc+WfxaqEf9+vyTpMXf/ZInTqv4u0MPWPG6Q9I9mdpmkX0h6sySZ2YCkK939ckkvlfS3Zjar6Et6g7sT2JbB3XNmdrWkeyS1S7rN3fea2UckDbr7LkVf3DvNbEhRz9ol6bW4+VT4GbzDzC5UNHvrOUmXptbgJmZmd0k6V9JxZrZf0gcldUqSu39B0m5JmyUNSTos6e3ptLR5VfAZXCzpz80sJ2lc0iX8D2TNvVrSn0l6xMweih/7S0kbpKV/F9jpAAAAIHAMiQIAAASOwAYAABA4AhsAAEDgCGwAAACBI7ABAAAEjsAGAAnM7Ix4JfilPv+HtWwPgNZGYAOAZGcoWidpgXjHirLc/Q9q3iIALYt12AA0rXjj5W9J2iPpbEkPS/o7SR+W9AJJfyppr6TPSXq5osXEPyTpm4oWtFwl6ZeSPq5o4enjJZ0k6TeKFsK8U1Jv/HZXu/sP4wV7L4wfO1bSA+7+P+r2QwJoCQQ2AE0rDmxDivbM3atoG6uHJV2mKFS9XdH2bI+6+9+b2dGSfhyf/2ZJA+5+dfxaH5L0x5I2ufu4ma2WNOvuE2a2UdHm8gMF790r6V5J17j7vQ34cQE0MbamAtDs/svdH5EkM9sr6bvu7mb2iKLeshMkXWhm74nP71G8hUyCXe4+Ht/ulPR5MztD0oykFxed+3eSdhDWANQCgQ1As5ssuD1bcH9W0b+BM5Le5O77Cp9kZmclvNZYwe1rJD0j6XRF9cATBc99v6TD7v65ZbceAMSkAwC4R9JfmJlJkpmdGT+elZQp87yjJB1w91lFGz23x8/frGi49cq6tRhAyyGwAWh1H1U0vPlTM/tZfF+Svi/pNDN7yMy2JDzvZklbzew+RcOh+d6390paK+m++Lmfqm/zAbQCJh0AAAAEjh42AACAwBHYAAAAAkdgAwAACByBDQAAIHAENgAAgMAR2AAAAAJHYAMAAAgcgQ0AACBw/x8FU+TbtSGH2wAAAABJRU5ErkJggg==\n", - "text/plain": [ - "
" - ] - }, - "metadata": { - "needs_background": "light" - }, - "output_type": "display_data" - } - ], - "source": [ - "fig = plot_data(Xo, yo, xlabel=u'metraż', ylabel=u'cena')\n", - "theta_start = np.matrix([0.0, 0.0]).reshape(2, 1)\n", - "theta, logs = gradient_descent(cost, gradient, theta_start, Xo, yo, alpha=0.01)\n", - "plot_regression(fig, h_linear, theta, Xo)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "Na powyższym wykresie widać, że po odrzuceniu obserwacji odstających otrzymujemy dużo bardziej „wiarygodną” krzywą regresji." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "## 3.4. Warianty metody gradientu prostego\n", - "\n", - "* Batch gradient descent\n", - "* Stochastic gradient descent\n", - "* Mini-batch gradient descent" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### _Batch gradient descent_\n", - "\n", - "* Klasyczna wersja metody gradientu prostego\n", - "* Obliczamy gradient funkcji kosztu względem całego zbioru treningowego:\n", - " $$ \\theta := \\theta - \\alpha \\cdot \\nabla_\\theta J(\\theta) $$\n", - "* Dlatego może działać bardzo powoli\n", - "* Nie można dodawać nowych przykładów na bieżąco w trakcie trenowania modelu (*online learning*)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### *Stochastic gradient descent* (SGD)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - "#### Algorytm\n", - "\n", - "Powtórz określoną liczbę razy (liczba epok):\n", - " 1. Randomizuj dane treningowe\n", - " 1. Powtórz dla każdego przykładu $i = 1, 2, \\ldots, m$:\n", - " $$ \\theta := \\theta - \\alpha \\cdot \\nabla_\\theta \\, J \\! \\left( \\theta, x^{(i)}, y^{(i)} \\right) $$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "**Randomizacja danych** to losowe potasowanie przykładów uczących (wraz z odpowiedziami)." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "#### SGD - zalety\n", - "\n", - "* Dużo szybszy niż _batch gradient descent_\n", - "* Można dodawać nowe przykłady na bieżąco w trakcie trenowania (*online learning*)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "#### SGD\n", - "\n", - "* Częsta aktualizacja parametrów z dużą wariancją:\n", - "\n", - "\n", - "\n", - "* Z jednej strony dzięki temu nie utyka w złych minimach lokalnych, ale z drugiej strony może „wyskoczyć” z dobrego minimum" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### _Mini-batch gradient descent_" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - "#### Algorytm\n", - "\n", - "1. Ustal rozmiar \"paczki/wsadu\" (*batch*) $b \\leq m$.\n", - "2. Powtórz określoną liczbę razy (liczba epok):\n", - " 1. Powtórz dla każdego batcha (czyli dla $i = 1, 1 + b, 1 + 2 b, \\ldots$):\n", - " $$ \\theta := \\theta - \\alpha \\cdot \\nabla_\\theta \\, J \\left( \\theta, x^{(i : i+b)}, y^{(i : i+b)} \\right) $$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "#### _Mini-batch gradient descent_\n", - "\n", - "* Kompromis między _batch gradient descent_ i SGD\n", - "* Stabilniejsza zbieżność dzięki redukcji wariancji aktualizacji parametrów\n", - "* Szybszy niż klasyczny _batch gradient descent_\n", - "* Typowa wielkość batcha: między kilka a kilkaset przykładów\n", - " * Im większy batch, tym bliżej do BGD; im mniejszy batch, tym bliżej do SGD\n", - " * BGD i SGD można traktować jako odmiany MBGD dla $b = m$ i $b = 1$" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "slideshow": { - "slide_type": "skip" - } - }, - "outputs": [], - "source": [ - "# Mini-batch gradient descent - przykładowa implementacja\n", - "\n", - "def MiniBatchSGD(h, fJ, fdJ, theta, X, y, \n", - " alpha=0.001, maxEpochs=1.0, batchSize=100, \n", - " logError=True):\n", - " errorsX, errorsY = [], []\n", - " \n", - " m, n = X.shape\n", - " start, end = 0, batchSize\n", - " \n", - " maxSteps = (m * float(maxEpochs)) / batchSize\n", - " for i in range(int(maxSteps)):\n", - " XBatch, yBatch = X[start:end,:], y[start:end,:]\n", - "\n", - " theta = theta - alpha * fdJ(h, theta, XBatch, yBatch)\n", - " \n", - " if logError:\n", - " errorsX.append(float(i*batchSize)/m)\n", - " errorsY.append(fJ(h, theta, XBatch, yBatch).item())\n", - " \n", - " if start + batchSize < m:\n", - " start += batchSize\n", - " else:\n", - " start = 0\n", - " end = min(start + batchSize, m)\n", - " \n", - " return theta, (errorsX, errorsY)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "Porównanie uśrednionych krzywych uczenia na przykładzie klasyfikacji dwuklasowej zbioru [MNIST](https://en.wikipedia.org/wiki/MNIST_database):\n", - "\n", - "" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Wady klasycznej metody gradientu prostego, czyli dlaczego potrzebujemy optymalizacji\n", - "\n", - "* Trudno dobrać właściwą szybkość uczenia (*learning rate*)\n", - "* Jedna ustalona wartość stałej uczenia się dla wszystkich parametrów\n", - "* Funkcja kosztu dla sieci neuronowych nie jest wypukła, więc uczenie może utknąć w złym minimum lokalnym lub punkcie siodłowym" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "## 3.5. Algorytmy optymalizacji metody gradientu\n", - "\n", - "* Momentum\n", - "* Nesterov Accelerated Gradient\n", - "* Adagrad\n", - "* Adadelta\n", - "* RMSprop\n", - "* Adam\n", - "* Nadam\n", - "* AMSGrad" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Momentum\n", - "\n", - "* SGD źle radzi sobie w „wąwozach” funkcji kosztu\n", - "* Momentum rozwiązuje ten problem przez dodanie współczynnika $\\gamma$, który można trakować jako „pęd” spadającej piłki:\n", - " $$ v_t := \\gamma \\, v_{t-1} + \\alpha \\, \\nabla_\\theta J(\\theta) $$\n", - " $$ \\theta := \\theta - v_t $$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Przyspiesony gradient Nesterova (*Nesterov Accelerated Gradient*, NAG)\n", - "\n", - "* Momentum czasami powoduje niekontrolowane rozpędzanie się piłki, przez co staje się „mniej sterowna”\n", - "* Nesterov do piłki posiadającej pęd dodaje „hamulec”, który spowalnia piłkę przed wzniesieniem:\n", - " $$ v_t := \\gamma \\, v_{t-1} + \\alpha \\, \\nabla_\\theta J(\\theta - \\gamma \\, v_{t-1}) $$\n", - " $$ \\theta := \\theta - v_t $$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Adagrad\n", - "\n", - "* “Adaptive gradient”\n", - "* Adagrad dostosowuje współczynnik uczenia (*learning rate*) do parametrów: zmniejsza go dla cech występujących częściej, a zwiększa dla występujących rzadziej:\n", - "* Świetny do trenowania na rzadkich (*sparse*) zbiorach danych\n", - "* Wada: współczynnik uczenia może czasami gwałtownie maleć\n", - "* Wyniki badań pokazują, że często **starannie** dobrane $\\alpha$ daje lepsze wyniki na zbiorze testowym" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Adadelta i RMSprop\n", - "* Warianty algorytmu Adagrad, które radzą sobie z problemem gwałtownych zmian współczynnika uczenia" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Adam\n", - "\n", - "* “Adaptive moment estimation”\n", - "* Łączy zalety algorytmów RMSprop i Momentum\n", - "* Można go porównać do piłki mającej ciężar i opór\n", - "* Obecnie jeden z najpopularniejszych algorytmów optymalizacji" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### Nadam\n", - "* “Nesterov-accelerated adaptive moment estimation”\n", - "* Łączy zalety algorytmów Adam i Nesterov Accelerated Gradient" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "### AMSGrad\n", - "* Wariant algorytmu Adam lepiej dostosowany do zadań takich jak rozpoznawanie obiektów czy tłumaczenie maszynowe" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "## 3.6. Metody zbiorcze" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - " * **Metody zbiorcze** (*ensemble methods*) używają połączonych sił wielu modeli uczenia maszynowego w celu uzyskania lepszej skuteczności niż mogłaby być osiągnięta przez każdy z tych modeli z osobna." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - " * Na metodę zbiorczą składa się:\n", - " * dobór modeli\n", - " * sposób agregacji wyników" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - " * Warto zastosować randomizację, czyli przetasować zbiór uczący przed trenowaniem każdego modelu." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "### Uśrednianie prawdopodobieństw" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - "#### Przykład\n", - "\n", - "Mamy 3 modele, które dla klas $c=1, 2, 3, 4, 5$ zwróciły prawdopodobieństwa:\n", - "\n", - "* $M_1$: [0.10, 0.40, **0.50**, 0.00, 0.00]\n", - "* $M_2$: [0.10, **0.60**, 0.20, 0.00, 0.10]\n", - "* $M_3$: [0.10, 0.30, **0.40**, 0.00, 0.20]\n", - "\n", - "Która klasa zostanie wybrana według średnich prawdopodobieństw dla każdej klasy?" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "Średnie prawdopodobieństwo: [0.10, **0.43**, 0.36, 0.00, 0.10]\n", - "\n", - "Została wybrana klasa $c = 2$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "### Głosowanie klas" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "fragment" - } - }, - "source": [ - "#### Przykład\n", - "\n", - "Mamy 3 modele, które dla klas $c=1, 2, 3, 4, 5$ zwróciły prawdopodobieństwa:\n", - "\n", - "* $M_1$: [0.10, 0.40, **0.50**, 0.00, 0.00]\n", - "* $M_2$: [0.10, **0.60**, 0.20, 0.00, 0.10]\n", - "* $M_3$: [0.10, 0.30, **0.40**, 0.00, 0.20]\n", - "\n", - "Która klasa zostanie wybrana według głosowania?" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "subslide" - } - }, - "source": [ - "Liczba głosów: [0, 1, **2**, 0, 0]\n", - "\n", - "Została wybrana klasa $c = 3$" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "### Inne metody zbiorcze\n", - "\n", - " * Bagging\n", - " * Boostng\n", - " * Stacking\n", - " \n", - "https://towardsdatascience.com/ensemble-methods-bagging-boosting-and-stacking-c9214a10a205" - ] - } - ], - "metadata": { - "celltoolbar": "Slideshow", - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.3" - }, - "livereveal": { - "start_slideshow_at": "selected", - "theme": "white" - } - }, - "nbformat": 4, - "nbformat_minor": 4 -}