{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Klasyfikacja w Pythonie"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**zad. 1** Które z poniższych problemów jest problemem regresji, a które klasyfikacji?\n",
    " 1. Sprawdzenie, czy wiadomość jest spamem.\n",
    " 1. Przewidzenie oceny (od 1 do 5 gwiazdek) na podstawie komentarza.\n",
    " 1. OCR cyfr: rozpoznanie cyfry z obrazka.\n",
    " \n",
    " Jeżeli problem jest klasyfikacyjny, to jakie mamy klasy?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. klasyfikacja\n",
    "2. można traktować jako klasyfikację lub regresję. Jeżeli jako regresję to należy sprowadzić liczbę rzeczywistą do jednej z {1,2,3,4,5}\n",
    "3. klasyfikacja"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Miary dla klasyfikacji"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Istnieje wieje miar (metryk), na podstawie których możemy ocenić jakość modelu. Podobnie jak w przypadku regresji liniowej potrzebne są dwie listy: lista poprawnych klas i lista predykcji z modelu. Najpopularniejszą z metryk jest trafność, którą definiuje się w następujący sposób:\n",
    " $$ACC = \\frac{k}{N}$$ \n",
    " \n",
    " gdzie: \n",
    "  * $k$  to liczba poprawnie zaklasyfikowanych przypadków,\n",
    "  * $N$ liczebność zbioru testującego."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**zadanie** Napisz funkcję, która jako parametry przyjmnie dwie listy (lista poprawnych klas i wyjście z klasyfikatora) i zwróci trafność."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ACC: 0.4\n"
     ]
    }
   ],
   "source": [
    "def accuracy_measure(true, predicted):\n",
    "    return sum([1 if t==p else 0 for t,p in zip(true, predicted)]) / len(true)\n",
    "\n",
    "true_label = [1, 1, 1, 0, 0]\n",
    "predicted = [0, 1, 0, 1, 0]\n",
    "print(\"ACC:\", accuracy_measure(true_label, predicted))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Klasyfikator $k$ najbliższych sąsiadów *(ang. k-nearest neighbors, KNN)*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Klasyfikator [KNN](https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm), który został wprowadzony na ostatnim wykładzie, jest bardzo intuicyjny. Pomysł, który stoi za tym klasyfikatorem jest bardzo prosty: Jeżeli mamy nowy obiekt do zaklasyfikowania, to szukamy wśród danych trenujących $k$ najbardziej podobnych do niego przykładów i na ich podstawie decydujemy (np. biorąc większość) do jakie klasy powinien należeć dany obiekt."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Przykład 1** Mamy za zadanie przydzielenie obiektów do dwóch klas: trójkątów lub kwadratów. Rozpatrywany obiekt jest zaznaczony zielonym kółkiem. Przyjmując $k=3$, mamy wśród sąsiadów 2 trójkąty i 1 kwadrat. Stąd obiekt powinienm zostać zaklasyfikowany jako trójkąt. Jak zmienia się sytuacja, gdy przyjmiemy $k=5$?\n",
    "\n",
    "![Przykład 1](./KnnClassification.svg.png)\n",
    "\n",
    "( Grafika pochodzi z https://pl.wikipedia.org/wiki/K_najbli%C5%BCszych_s%C4%85siad%C3%B3w )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Herbal Iris"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Herbal Iris* jest klasycznym zbiorem danych w uczeniu maszynowym, który powstał w 1936 roku. Zawiera on informacje na 150 egzemplarzy roślin, które należą do jednej z 3 odmian."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**zad. 2** Wczytaj do zmiennej ``data`` zbiór *Herbal Iris*, który znajduje się w pliku ``iris.data``. Jest to plik csv.\n",
    "\n",
    "Kolumny są następujące:\n",
    "\n",
    "1. sepal length in cm\n",
    "2. sepal width in cm\n",
    "3. petal length in cm\n",
    "4. petal width in cm\n",
    "5. class: \n",
    "  * Iris Setosa\n",
    "  * Iris Versicolour\n",
    "  * Iris Virginica"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5.1,3.5,1.4,0.2,Iris-setosa\r\n",
      "4.9,3.0,1.4,0.2,Iris-setosa\r\n",
      "4.7,3.2,1.3,0.2,Iris-setosa\r\n",
      "4.6,3.1,1.5,0.2,Iris-setosa\r\n",
      "5.0,3.6,1.4,0.2,Iris-setosa\r\n",
      "5.4,3.9,1.7,0.4,Iris-setosa\r\n",
      "4.6,3.4,1.4,0.3,Iris-setosa\r\n",
      "5.0,3.4,1.5,0.2,Iris-setosa\r\n",
      "4.4,2.9,1.4,0.2,Iris-setosa\r\n",
      "4.9,3.1,1.5,0.1,Iris-setosa\r\n"
     ]
    }
   ],
   "source": [
    "!head iris.data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "data = pd.read_csv('iris.data', names=('sepal_length', 'sepal_width', 'petal_length', 'petal_width','class'),index_col=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal_length</th>\n",
       "      <th>sepal_width</th>\n",
       "      <th>petal_length</th>\n",
       "      <th>petal_width</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.9</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>6.2</td>\n",
       "      <td>3.4</td>\n",
       "      <td>5.4</td>\n",
       "      <td>2.3</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>5.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.8</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>150 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     sepal_length  sepal_width  petal_length  petal_width           class\n",
       "0             5.1          3.5           1.4          0.2     Iris-setosa\n",
       "1             4.9          3.0           1.4          0.2     Iris-setosa\n",
       "2             4.7          3.2           1.3          0.2     Iris-setosa\n",
       "3             4.6          3.1           1.5          0.2     Iris-setosa\n",
       "4             5.0          3.6           1.4          0.2     Iris-setosa\n",
       "..            ...          ...           ...          ...             ...\n",
       "145           6.7          3.0           5.2          2.3  Iris-virginica\n",
       "146           6.3          2.5           5.0          1.9  Iris-virginica\n",
       "147           6.5          3.0           5.2          2.0  Iris-virginica\n",
       "148           6.2          3.4           5.4          2.3  Iris-virginica\n",
       "149           5.9          3.0           5.1          1.8  Iris-virginica\n",
       "\n",
       "[150 rows x 5 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**zad. 3** Odpowiedz na poniższe pytania:\n",
    " 1. Które atrybuty są wejściowe, a w której kolumnie znajduje się klasa wyjściowa?\n",
    " 1. Ile jest różnych klas? Wypisz je ekran.\n",
    " 1. Jaka jest średnia wartość w kolumnie ``sepal_length``? Jak zachowuje się średnia, jeżeli policzymy ją dla każdej z klas osobno?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. wejściowe są sepal_length, sepal_width, petal_length, petal_width. Klasa wyjściowa to class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Iris-setosa', 'Iris-versicolor', 'Iris-virginica'], dtype=object)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['class'].unique()\n",
    "# 3 klasy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.843333333333334"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['sepal_length'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "class\n",
       "Iris-setosa        5.006\n",
       "Iris-versicolor    5.936\n",
       "Iris-virginica     6.588\n",
       "Name: sepal_length, dtype: float64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.groupby('class')['sepal_length'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wytrenujmy klasyfikator *KNN*, ale najpierw przygotujmy dane. Fukcja ``train_test_split`` dzieli zadany zbiór danych na dwie części. My wykorzystamy ją do podziału na zbiór treningowy (66%) i testowy (33%), służy do tego parametr ``test_size``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X = data.loc[:, 'sepal_length':'petal_width']\n",
    "Y = data['class']\n",
    "\n",
    "(train_X, test_X, train_Y, test_Y) = train_test_split(X, Y, test_size=0.33, random_state=42)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Trenowanie klasyfikatora wygląda bardzo podobnie do treningi modelu regresji liniowej:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>KNeighborsClassifier(n_neighbors=3)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">KNeighborsClassifier</label><div class=\"sk-toggleable__content\"><pre>KNeighborsClassifier(n_neighbors=3)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "KNeighborsClassifier(n_neighbors=3)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "model = KNeighborsClassifier(n_neighbors=3)\n",
    "model.fit(train_X, train_Y)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mając wytrenowany model możemy wykorzystać go do predykcji na zbiorze testowym."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
      "Zaklasyfikowane: Iris-setosa, Orginalne: Iris-setosa\n",
      "Zaklasyfikowane: Iris-virginica, Orginalne: Iris-virginica\n",
      "Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
      "Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
      "Zaklasyfikowane: Iris-setosa, Orginalne: Iris-setosa\n",
      "Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
      "Zaklasyfikowane: Iris-virginica, Orginalne: Iris-virginica\n",
      "Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
      "Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n"
     ]
    }
   ],
   "source": [
    "predicted = model.predict(test_X)\n",
    "\n",
    "for i in range(10):\n",
    "    print(\"Zaklasyfikowane: {}, Orginalne: {}\".format(predicted[i], test_Y.reset_index()['class'][i]))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Możemy obliczyć *accuracy*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.98\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "print(accuracy_score(test_Y, predicted))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**zad. 4** Wytrenuj nowy model ``model_2`` zmieniając liczbę sąsiadów na 20. Czy zmieniły się wyniki?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.98\n"
     ]
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "model = KNeighborsClassifier(n_neighbors=10)\n",
    "model.fit(train_X, train_Y)\n",
    "predicted = model.predict(test_X)\n",
    "\n",
    "print(accuracy_score(test_Y, predicted))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**zad. 5**   Wytrenuj model z $k=1$. Przeprowadź walidację na zbiorze trenującym zamiast na zbiorze testowym? Jakie wyniki otrzymałeś? Czy jest to wyjątek? Dlaczego tak się dzieje?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "model = KNeighborsClassifier(n_neighbors=1)\n",
    "model.fit(train_X, train_Y)\n",
    "predicted = model.predict(train_X)\n",
    "\n",
    "print(accuracy_score(train_Y, predicted))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Walidacja krzyżowa"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Zbiór *herbal Iris* jest bardzo małym zbiorem. Wydzielenie z niego zbioru testowego jest obciążone dużą wariancją wyników, tj. w zależności od sposoby wyboru zbioru testowego wyniki mogą się bardzo różnic. Żeby temu zaradzić, stosuje się algorytm [walidacji krzyżowej](https://en.wikipedia.org/wiki/Cross-validation_(statistics). Algorytm wygląda następująco:\n",
    " 1. Podziel zbiór danych na $n$ części (losowo).\n",
    " 1. Dla każdego i od 1 do $n$ wykonaj:\n",
    "  1. Weź $i$-tą część jako zbiór testowy, pozostałe dane jako zbiór trenujący.\n",
    "  1. Wytrenuj model na zbiorze trenującym.\n",
    "  1. Uruchom model na danych testowych i zapisz wyniki.\n",
    " 1. Ostateczne wyniki to średnia z $n$ wyników częściowych. \n",
    " \n",
    " W Pythonie służy do tego funkcja ``cross_val_score``, która przyjmuje jako parametry (kolejno) model, zbiór X, zbiór Y. Możemy ustawić parametr ``cv``, który określa na ile części mamy podzielić zbiór danych oraz parametr ``scoring`` określający miarę.\n",
    " \n",
    " W poniższym przykładzie dzielimy zbiór danych na 10 części (10-krotna walidacja krzyżowa) i jako miarę ustawiany celność (ang. accuracy)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wynik walidacji krzyżowej: 0.9666666666666668\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "k=10\n",
    "knn = KNeighborsClassifier(n_neighbors=k)\n",
    "scores = cross_val_score(knn, X, Y, cv=10, scoring='accuracy')\n",
    "print(\"Wynik walidacji krzyżowej:\", scores.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**zad. 6** Klasyfikator $k$ najbliższych sąsiadów posiada jeden parametr: $k$, który określa liczbę sąsiadów podczas klasyfikacji. Jak widzieliśmy, wybór $k$ może mieć duże znaczenie dla jakości klasyfikatora. Wykonaj:\n",
    " 1. Stwórz listę ``neighbors`` wszystkich liczb nieparzystych od 1 do 50.\n",
    " 1. Dla każdego elementu ``i`` z listy ``neighbors`` zbuduj klasyfikator *KNN* o liczbie sąsiadów równej ``i``. Nastepnie przeprowadz walidację krzyżową (parametry takie same jak powyżej) i zapisz wyniki do tablicy ``cv_scores``.\n",
    " 1. Znajdź ``k``, dla którego klasyfikator osiąga najwyższy wynik."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "neighbors = list(range(1,50,2))\n",
    "cv_scores = list()\n",
    "max_score = -1\n",
    "for neighbor_num in neighbors:\n",
    "    knn = KNeighborsClassifier(n_neighbors=neighbor_num)\n",
    "    score = cross_val_score(knn, X, Y, cv=10, scoring='accuracy').mean()\n",
    "    max_score = score if score > max_score else max_score\n",
    "    neighbor_num_best = neighbor_num if score == max_score else neighbor_num_best\n",
    "    cv_scores.append(score)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13\n",
      "0.9800000000000001\n"
     ]
    }
   ],
   "source": [
    "print(neighbor_num_best)\n",
    "print(max_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wykres przedstawiający precent błedów w zależnosci od liczby sąsiadów."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAGxCAYAAACKvAkXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABbOklEQVR4nO3dd3hUZdo/8O/MZErqhJBe6CWVBFIwoIKaFQQpghhYVxTRXV0QNKuv4C7ivv4UfXdRcGV12XfV1101iAgqKsVQVFpIgySEFkp6hfQ+c35/JDMkJoFkMpMz5fu5rrkuOHPmzD0nwNw8z/3cj0QQBAFERERENkQqdgBEREREg40JEBEREdkcJkBERERkc5gAERERkc1hAkREREQ2hwkQERER2RwmQERERGRzmAARERGRzbETOwBzpdVqUVRUBGdnZ0gkErHDISIioj4QBAG1tbXw9fWFVNr7OA8ToF4UFRUhICBA7DCIiIjIAPn5+fD39+/1eSZAvXB2dgbQfgNdXFxEjoaIiIj6oqamBgEBAfrv8d4wAeqFbtrLxcWFCRAREZGFuVX5CougiYiIyOYwASIiIiKbwwSIiIiIbA4TICIiIrI5TICIiIjI5jABIiIiIpvDBIiIiIhsDhMgIiIisjlMgIiIiMjmMAEiIiIim8MEiIiIiGwOEyAiIiKyOdwMlYiIyEpdLKvDB0cuo6VNa5Tr+bnaY9U9YyGT3nyjUUvABIiIiMhKvb3/PL7NLDbqNUd7OmFuuK9RrykGJkBERERW6lRBFQDgkduGw9fVfkDXSsu7jv1nSrHtZB4TICIiIjJPVQ0tKLjeCAB4/t7xUDvIB3S9/GsN+CGnFEcuViKvsgHDhjoYI0zRsAiaiIjICmUX1QAAAtzsB5z8tF/HAbePcQcAfJ6SP+DriY0JEBERkRXKKqwGAIT5qY12zcXRwwAA21Pz0aYxTmG1WJgAERERWaGsjhGgEF/jJUC/CvaCm6MCpTXNOHy+3GjXFQMTICIiIiuU3TECFGrEESCFnRQLJ/kBAD5LtuxpMCZAREREVqa2qRWXKuoBACG+Lka9dnx0AADg4LkylNY0GfXag4kJEBERkZXJKa4FAPioVXB3Uhr12mM8nRE1fAg0WgFfpBYY9dqDiQkQERGRldEVQBuz/qcz3SjQ5yn50GoFk7yHqTEBIiIisjJZRbr6H+NOf+nMnuADZ6UdrlY24PjlSpO8h6kxASIiIrIyuhGgUBONADko7DA3or0bdKKFFkMzASIiIrIijS0aXCyrA2DcFWC/pOsJtCerBNfrW0z2PqbCBIiIiMiK5JTUQCsA7k4KeLkYtwC6s1A/FwT7uKBFo8WujEKTvY+pMAEiIiKyIp37/0gkEpO9j0QiweKY9mLoxOR8CIJlFUMzASIiIrIiWYXtHaBNVf/T2bwIPyjtpDhXWouM/CqTv58xMQEiIiKyIqZeAdaZ2l6O2WE+AIBtJy2rGJoJEBERkZVobtPgfGl7E0RT9QD6JV1PoK9PFaGuuW1Q3tMYmAARERFZiQuldWjVCFDby+E/xH5Q3jNmpBtGujuioUWDb08XDcp7GgMTICIiIiuh7//j52LSAujOJBKJfhQo0YKmwZgAERERWQl9/c8gTX/pLJzkDzupBOl5VThXUjuo720oJkBERERWIrNjBViICRsg9sTDWYm4IC8AQOLJvEF9b0MxASIiIrICrRotcop1S+BNvwLsl+I7egLtTC9EU6tm0N+/v5gAERERWYHc8jq0tGnhpLTDiKGOg/7+d471gI9ahaqGVuw7Uzro799fTICIiIisgK4BYrCvC6TSwSmA7kwmlWBRVPso0DYLmAZjAkRERGQFTL0DfF88FOUPiQQ4crESVyvrRYujL5gAERERWYHsQewA3Rv/IQ64Y6wHAODzFPNeEs8EiIiIyMJptQKyizoKoAd5BdgvLe7oCbQ9pQBtGq2osdwMEyAiIiILd7myHg0tGqjkUoxyH/wC6M7igrzg5qhAWW0zDp0rFzWWm2ECREREZOF09T9BPi6wk4n71a6wk2LhJD8A5t0ZmgkQERGRhdNPf4lYAN2ZbmuMg+fKUFrTJHI0PWMCREREZOEyC8QvgO5sjKczokcMgUYr4IvUArHD6RETICIiIgsmCMKNPcBELoDuLD56GABg28l8aLWCyNF0xwSIiIjIguVfa0RtUxsUMinGejqLHY7erDBvOCvtkHetAccvVYodTjdMgIiIiCyYbvRnvLczFHbm87XuoLDD3AhfAOZZDG0+d4qIiIj6Td8B2kzqfzpb3DENtierBNfrW0SOpismQERERBYsq2MFWIiZrADrLMxfjRBfF7RotNiZXih2OF0wASIiIrJQgiAgu9D8CqA703WG3nYyH4JgPsXQTICIiIgsVElNEyrrWyCTShDobT4F0J3NjfCD0k6Kc6W1yMivEjscPSZAREREFiqrsH36a6ynE1RymcjR9ExtL8fsMB8A7aNA5oIJEBERkYXSFUCbY/1PZ7rO0F+fKkJdc5vI0bRjAkRERGShzHkFWGcxI90wyt0RDS0a7D5VJHY4AJgAERERWSxdD6AwMy2A1pFIJPpRIHPpCcQEiIiIyAKV1TahtKYZEkn7LvDmbsEkf9hJJcjIr8LZkhqxwzGPBGjLli0YMWIEVCoVJk+ejOTk5Juev337dgQGBkKlUiEsLAzfffddt3NycnIwd+5cqNVqODo6Ijo6Gnl5eab6CERERINKtwP8KHdHOCrtRI7m1jyclYgL8gJgHsXQoidA27ZtQ0JCAtavX4+0tDSEh4djxowZKCsr6/H8o0ePYsmSJVi+fDnS09Mxf/58zJ8/H1lZWfpzcnNzcfvttyMwMBCHDh3C6dOnsW7dOqhUqsH6WERERCZl7v1/ehIf0z4NtjO9EE2tGlFjkQgidyWaPHkyoqOj8e677wIAtFotAgIC8Mwzz2DNmjXdzo+Pj0d9fT12796tP3bbbbchIiIC77//PgBg8eLFkMvl+Pe//21wXDU1NVCr1aiuroaLi/kPLRIRkW156t+p2JNdgj/OCsKTd44SO5w+0WgF3PHmARRVN2Hz4gjMi/Az+nv09ftb1BGglpYWpKamIi4uTn9MKpUiLi4Ox44d6/E1x44d63I+AMyYMUN/vlarxbfffotx48ZhxowZ8PT0xOTJk7Fr166bxtLc3IyampouDyIiInOlK4AOMfMVYJ3JpBIsirrRGVpMoiZAFRUV0Gg08PLy6nLcy8sLJSUlPb6mpKTkpueXlZWhrq4Ob7zxBmbOnIl9+/bhgQcewIIFC3D48OFeY9mwYQPUarX+ERAQMMBPR0REZBpVDS0ouN4IwPx7AP3Soih/SCTA0dxKXK2sFy0O86+a6ietVgsAmDdvHp577jkAQEREBI4ePYr3338f06ZN6/F1a9euRUJCgv73NTU1TIKIiMgs6Qqgh7k5QG0vFzma/vEf4oAV08cg0McZ3mrxanNFTYDc3d0hk8lQWlra5XhpaSm8vb17fI23t/dNz3d3d4ednR2Cg4O7nBMUFISff/6511iUSiWUSqUhH4OIiGhQWUoDxN48P2O82CGIOwWmUCgQGRmJpKQk/TGtVoukpCTExsb2+JrY2Ngu5wPA/v379ecrFApER0fj3LlzXc45f/48hg8fbuRPQERENPgyLXAFmLkRfQosISEBjz76KKKiohATE4NNmzahvr4ey5YtAwAsXboUfn5+2LBhAwBg9erVmDZtGjZu3IjZs2cjMTERKSkp2Lp1q/6aL7zwAuLj43HnnXfirrvuwp49e/DNN9/g0KFDYnxEIiIio9JNgYVaWP2PORE9AYqPj0d5eTlefvlllJSUICIiAnv27NEXOufl5UEqvTFQNWXKFHz66af405/+hJdeegljx47Frl27EBoaqj/ngQcewPvvv48NGzZg1apVGD9+PHbs2IHbb7990D8fERGRMdU2teJyRXvxcIivZU6BmQPR+wCZK/YBIiIic3TiUiXitx6Hr1qFo2vvETscs2MRfYCIiIiof7I6pr9CWP8zIEyAiIiILIh+CwzW/wwIEyAiIiILousAbalL4M0FEyAiIiIL0diiwcWyOgBcAj9QTICIiIgsRE5JDbQC4O6khKczm/cOBBMgIiIiC6HrAB3m5wKJRCJyNJaNCRAREZGFyGIHaKNhAkRERGQhsgo7lsBzBdiAMQEiIiKyAM1tGpwvrQXAFWDGwASIiIjIApwvqUObVoCrgxx+rvZih2PxmAARERFZAH3/H181C6CNgAkQERGRBdAVQIdw+ssomAARERFZAN0eYNwCwziYABEREZm5Vo0WOcUdCRCXwBsFEyAiIiIzl1teh5Y2LZyUdhju5iB2OFaBCRAREZGZyyzoqP/xdYFUygJoY2ACREREZOayizj9ZWxMgIiIiMzcjS0wuALMWJgAERERmTGNVsCZYq4AMzYmQERERGbsckU9Glo0UMmlGOXhJHY4VoMJEBERkRnL7ugAHezjAhkLoI2GCRAREZEZu1H/w+kvY2ICREREZMayCln/YwpMgIiIiMyUIAj6TVC5B5hxMQEiIiIyU/nXGlHb1AaFTIpxXs5ih2NVmAARERGZqcyO+p9AH2fIZfzKNibeTSIiIjOln/5i/Y/RMQEiIiIyU+wAbTpMgIiIiMyQIAg39gDjCJDRMQEiIiIyQ8XVTbhW3wKZVILx3iyANjYmQERERGZIN/011tMJKrlM5GisDxMgIiIiM5Slm/5iB2iTYAJERERkhrJ1BdC+LIA2BSZAREREZki3BD7MnyNApsAEiIiIyMyU1TahtKYZEgkQ5MMRIFNgAkRERGRmsjs2QB3t4QQHhZ3I0VgnJkBERERmJov1PybHBIiIiMjM6Op/uALMdJgAERERmZmsjikw7gFmOkyAiIiIzMj1+hYUVjUCAII5BWYyrKwiIrIigiBg64+XcKGsTuxQTMrTWYln48ZBYWd9/4/X7f81fKgD1PZykaOxXkyAiIisyLHcSmz4/qzYYQwKX1d7/Oa24WKHYXQ/XSwHAISx/sekmAAREVmRz07mAwBuH+OOqWPcRY7GNLKLqrH7dDG2ncy3ugSoVaPFjtQCAMD9E3xFjsa6MQEiIrIS1+tbsDerBACw5r5Aq11BVFnXjL3ZJcgsrEZWYbVVfc6knDJU1LXA3UmJe4I8xQ7Hqlnf5CkRkY3amV6IFo0WIb4uVpUU/NJQJyXuDfYGAHyeki9yNMa17WQeAODBSH/IZfyKNiXeXSIiKyAIArZ1TH8tjg4QORrTWxzT/hl3pheiqVUjcjTGUVTViMPn2+t/4m3gZyg2JkBERFYgI78K50proZJLMTfCT+xwTG7qaHf4udqjtqkN32cVix2OUWxPKYBWACaPdMNId0exw7F6TICIiKxAYnL76M+sMB+bWDotlUr0oySfJVv+NJhGK+in85bEDBM5GtvABIiIyMLVNbfhm9NFAIDF0bbz5flgpD+kEiD58jVcKrfsvkdHLlagsKoRLio7zAz1Fjscm8AEiIjIwu0+VYSGFg1GuTsiesQQscMZNL6u9pg2zgMAsM3Ci6F19VsPTPSDSi4TORrbwASIiMjCJXZ8ecZHB0AikYgczeBa3DFdtCO1AK0arcjRGKayrhn7zrS3L4i3oRE8sTEBIiKyYGdLapCRXwU7qQQLI/3FDmfQ3R3oCXcnJSrqWpCUUyZ2OAb5Mq0QrRoBE/zV3PtrEDEBIiKyYLri518Fe8HdSSlyNINPLpPiwY7EL7Gjh44lEQRBH7ct1W+ZAyZAREQWqqlVg53phQBsu2+M7rMfPl+Ooo5d1C1F6tXryC2vh71chjnhPmKHY1OYABERWai92SWobmyFr1qFO8Z6iB2OaEa6O+K2UW4QhPZeOpZEV791/wQfOKusv32BOWECRERkoXQrhxZFBUAmta3i51/STR99npIPjVYQOZq+qWlqxben25s46jpb0+BhAkREZIGuVtbjaG4lJBLgIRue/tKZGeoNF5UdCqsaceRihdjh9MnXGUVobNVgjKcTJg2znfYF5oIJEBGRBdKN/tw51gN+rvYiRyM+lVyGBya2bwFiKcXQnfdus7X2BeaACRARkYVp02ixPbW91sUWNj7tK10Pnf1nSlFR1yxyNDeXVViNzMJqyGUSLJhke+0LzAETICIiC3PwXDnKa5sx1FGBe4K8xA7HbAT7uiDcX41WjYCdaYVih3NTun2/7g3xhpujQuRobBMTICIiC7OtY4pnYaQ/FHb8Z7wz3ShQ4sk8CIJ5FkN3bl/AETzxGPQ358CBA2hqajJaEFu2bMGIESOgUqkwefJkJCcn3/T87du3IzAwECqVCmFhYfjuu++6PP/YY49BIpF0ecycOdNo8RIRiaWkugkHzrZ3PLbl3j+9mRPuA3u5DLnl9Ui9el3scHr0XWYxapva4D/EHlNHu4sdjs0yKAGaO3cuXF1dcccdd2DdunX44Ycf0NhoWPOpbdu2ISEhAevXr0daWhrCw8MxY8YMlJX13NL86NGjWLJkCZYvX4709HTMnz8f8+fPR1ZWVpfzZs6cieLiYv3js88+Myg+IiJz8kVqPrQCEDPCDaM9nMQOx+w4q+S4f0J7Q8HPks1zg1T93m1RAZDaePsCMRmUAF2/fh1JSUm47777kJycjAceeACurq6YOnUq/vSnP/XrWm+99RaefPJJLFu2DMHBwXj//ffh4OCADz74oMfzN2/ejJkzZ+KFF15AUFAQXn31VUyaNAnvvvtul/OUSiW8vb31jyFDuMSQiCybVivodz3n6E/vdBukfptZhJqmVpGj6epSeR2SL1+DVAI8GMXiZzEZlADJ5XJMnToVL730Evbu3Yvjx49jyZIlSE5OxoYNG/p8nZaWFqSmpiIuLu5GQFIp4uLicOzYsR5fc+zYsS7nA8CMGTO6nX/o0CF4enpi/PjxePrpp1FZWXnTWJqbm1FTU9PlQURkTo5dqkT+tUY4K+0wK4zbJvRm0jBXjPV0QlOrFl9nFIkdThe6BHb6eE/4qNm+QEwGJUDnz5/H1q1b8etf/xp+fn6YNm0aqqur8de//hVpaWl9vk5FRQU0Gg28vLquYvDy8kJJSUmPrykpKbnl+TNnzsTHH3+MpKQkvPnmmzh8+DDuu+8+aDSaXmPZsGED1Gq1/hEQwP9dEZF50U2dzJvoC3uFTORozJdEItGPkOl67ZiDVo0WOzraF3AET3x2hrwoMDAQHh4eWL16NdasWYOwsDCzauK0ePFi/a/DwsIwYcIEjB49GocOHcI999zT42vWrl2LhIQE/e9ramqYBBGR2bhW34K9We3/0eOu4be2YJI/3txzFpmF1cgqrEaon1rskJCUU4qKuha4Oylxd6Cn2OHYPINGgFatWgU/Pz/893//N5566in88Y9/xL59+9DQ0NCv67i7u0Mmk6G0tLTL8dLSUnh7e/f4Gm9v736dDwCjRo2Cu7s7Ll682Os5SqUSLi4uXR5EROZiZ3ohWjRahPq5mMWXublzc1Tg3pD27wVzGQVK1O/d5g+5jO0LxGbQT2DTpk1IS0tDSUkJ1q5di5aWFvzxj3+Eu7s7pk6d2ufrKBQKREZGIikpSX9Mq9UiKSkJsbGxPb4mNja2y/kAsH///l7PB4CCggJUVlbCx4dz5kRkeQRB0Pf+iefoT58t6bhXuzIK0djSewnEYCiqasTh8+UAgIeiOLtgDgaUgmo0GrS2tqK5uRlNTU1obm7GuXPn+nWNhIQE/POf/8T//d//IScnB08//TTq6+uxbNkyAMDSpUuxdu1a/fmrV6/Gnj17sHHjRpw9exavvPIKUlJSsHLlSgBAXV0dXnjhBRw/fhxXrlxBUlIS5s2bhzFjxmDGjBkD+bhERKJIz6/C+dI6qORSzA33FTscizFl9FD4D7FHbVMbvs8qFjWW7SkFEATgtlFuGOnuKGos1M7gKbAJEybAy8sLv/vd71BUVIQnn3wS6enpKC8v79e14uPj8de//hUvv/wyIiIikJGRgT179ugLnfPy8lBcfOMP7pQpU/Dpp59i69atCA8PxxdffIFdu3YhNDQUACCTyXD69GnMnTsX48aNw/LlyxEZGYmffvoJSqXSkI9LRCSqbR39bGaF+UBtLxc5GsshlUoQ3zHakijiNJhGK+i3vmD9lvmQCAb0Cl+0aBGmTZuG6dOn6xMPa1NTUwO1Wo3q6mrWAxGRaOqa2xDz2g9oaNHg89/FImakm9ghWZTi6kZMfeMAtAJw4A/TMEqE5pE/ni/H0g+SobaX48RL90Al5wo+U+rr97dBq8C2b99ucGBERNR335wqQkOLBqM8HBE9gg1d+8tHbY+7xnsi6WwZtp3Mx9pZQYMeQ2JH/dYDE/2Y/JgRg2uAcnNz8cwzzyAuLg5xcXFYtWoVcnNzjRkbEZHN003dLI4OMKt2I5ZE13NnR1oBWtq0g/reFXXN2H+mtEscZB76lAClpaV1aSK4d+9eBAcHIzk5GRMmTMCECRNw4sQJhISEYP/+/SYLlojIluQU1+BUfhXspBIsmMRtEwx1V6AnPJyVqKhrwYGzpbd+gRHtTCtEq0ZAuL8aQT4spzAnfUqADh8+jNmzZ6O+vh4AsGbNGjz33HM4ceIE3nrrLbz11ls4ceIEnn32Wbz44osmDZiIyFbo+tf8KtgL7k5cxGEouUyKByPbE8jBLIYWBEE//cX2BeanTwnQc889hzvvvBPTpk0DAOTk5GD58uXdznv88cdx5swZ40ZIRGSDmlo12JleCIBTJ8agWw12+Hw5iqoaB+U9U69eR255PRwUMsyNYPsCc9PnGqCXXnoJb7/9NgDAw8MDGRkZ3c7JyMiApyfbexMRDdTe7BJUN7bCz9Ued4z1EDscizfC3RGxo4ZCEKBfkm5qn3W0L7h/gg+clAatOSIT6tdP5I477gAAPPnkk/jtb3+LS5cuYcqUKQCAI0eO4M033+yynxYRERkmMfnGtgkyKYufjWFxTACOXarE9pQCPHP3WJPe15qmVnyb2b4TPae/zJNBKem6devg7OyMjRs36rs0+/r64pVXXsGqVauMGiARka25UlGPY5cqIZEAi7htgtHMCPGG2l6OwqpG/HyxAtPGmW5k7euMIjS1ajHW0wmThrma7H3IcAYtg5dIJHjuuedQUFCA6upqVFdXo6CgAKtXr+YyTSKiAdrWMUVz51gP+LnaixyN9VDJZXhgoh8A6PdWMxVdAXs82xeYrQFvR+vs7AxnZ2djxEJEZPNaNVp8kVoAoL33DxmXrqB8/5lSVNQ1m+Q9sgqrkVlYDYVMyvYFZqzPU2ATJ07scxablpZmcEBERLbs4NkylNc2w91JgXuCvMQOx+oE+bggPMAVp/Kr8GVaAX5752ijv4du9OfeEC+4OSqMfn0yjj6PAM2fPx/z5s3DvHnzMGPGDOTm5kKpVGL69OmYPn06VCoVcnNzueM6EdEA6L48F07yh8JuwIP01APdyFriyXwYsB3mTTW2aLAro7DjfVj8bM76PAK0fv16/a+feOIJrFq1Cq+++mq3c/Lzxdtxl4jIkpVUN+HguTIAwEOc/jKZOeG+eHX3GVwqr0fK1euIHmG8DWa/zypGbVMb/IfYY8rooUa7LhmfQf+92L59O5YuXdrt+G9+8xvs2LFjwEEREdmi7Sn50ApAzAg3jBZh13Jb4aS0w/0TfADcaDdgLLpO0/FRAZCyfYFZMygBsre3x5EjR7odP3LkCFQq1YCDIiKyNVqtoF/9xc7Pprc4pn166tvMItQ0tRrlmpfK65B8+RqkbF9gEQzqA/Tss8/i6aefRlpaGmJiYgAAJ06cwAcffIB169YZNUAiIltwNLcSBdcb4ayyw6wwH7HDsXoTA1wxzssJ50vr8FVGER65bfiAr6mr37prvCe81RwMMHcGJUBr1qzBqFGjsHnzZvznP/8BAAQFBeHDDz/EQw89ZNQAiYhsgW7TzPkRfrBXyESOxvpJJBLERw/Dq7vP4P1DuTiVXzXga+4/077TPEfwLINEMHYJvJWoqamBWq1GdXU1XFxcxA6HiKxYm0aL0Ff2oqlVi69XTsUEf1exQ7IJ1+tbcNuGJDS3aY12TS8XJX5+8W7IZVzBJ5a+fn8PaHe2AwcO4NSpUyguLsaaNWtQXV0NhUIBP7/2Tpv19fVwdHQcyFsQEVm93PJ6NLVq4aiQIdRXLXY4NmOIowKfPDEZJ69cN8r1JBJg2jgPJj8WwqAEKC8vDw888ABycnIwbtw4ZGdn46mnnsKxY8ewZ88e/Pvf/4YgCAgJCcGVK1eMHDIRkXXJKqwGAIT4qrlyaJBFjXBDlBGXwZPlMCgBeuqpp+Dt7Y29e/fC3d1dvxVGREQEHnnkEVy7dg2jR4+GRqMxarBERNYoq6gjAfLjdDvRYDEoATp06BBOnjwJd3f3LsfVajXkcjn+8pe/YM+ePdi1a5cxYiQismrZhTUAwOkvokFkUALk5OSEioqKbsdLSkrg4eGB4OBgBAcHDzg4IiJrp9UKyO4YAQr1YwJENFgMqtSaO3cufv/73+PUqVMAoN8k9e9//zsWLlxovOiIiKzclcp61LdooLSTYrQHF40QDRaDEqCNGzfC19cXkyZNgo+PDxoaGnDbbbchNzcXr7/+urFjJCKyWllF7dNfQT4usOPqIaJBY9AUmFqtxv79+/Hzzz/j9OnTqKurw6RJkxAXF2fs+IiIrFp2oW76iwXQRIPJ4GXwn3zyCdauXYvbb7/d2DEREdkM3QqwMNb/EA2qPidA77zzjv7XbW1t2LBhAxoaGuDh4dHt3FWrVhknOiIiKyYIArI6VoCFcAUY0aDq81YYI0eO7HaspKQELi4ucHBwuHFBiQSXLl0yXoQi4VYYRGRq+dcacMf/HIRcJkH2n2dCYccaIKKBMvpWGJcvX+527JNPPsE333yDxMREw6IkIrJhug7Q472dmfwQDbIB7QX28MMPY8GCBcaKhYjIpujqf9gAkWjwGfxfjqSkJNx///0IDQ3F6NGjcf/99+OHH34wZmxERFZNX//DAmiiQWdQAvT3v/8dM2fOhLOzM1avXo3Vq1fDxcUFs2bNwpYtW4wdIxGR1WkvgNaNALHOkGiwGTQF9vrrr+Ptt9/GypUr9cdWrVqFqVOn4vXXX8eKFSuMFiARkTUqrWlGZX0LZFIJgnyYABENNoNGgKqqqjBz5sxux++9915UV1cPOCgiImunG/0Z4+EElVwmcjREtsfgvcB27tzZ7fhXX32F+++/f8BBERFZuyxugEokKoMaIQYHB+O1117DoUOHEBsbCwA4fvw4jhw5gj/84Q/Gj5KIyMroCqC5BQaROAbUCLHHC7IRIhHRLd32ehJKapqw/alYRI9wEzscIqsxKI0QiYio/8prm1FS0wSJBCyAJhIJW48SEQ2y7I76n5HujnBSDqgfLREZiAkQEdEgyy7qqP9hB2gi0TABIiIaZPoGiCyAJhINEyAiokHGPcCIxGdQApSXl4eeFo8JgoC8vLwBB0VEZK2qG1qRf60RABDCBIhINAYlQCNHjkR5eXm349euXevzcnkiIlukK4AOcLOH2kEucjREtsugBEgQBEgkkm7H6+rqoFKpBhwUEZG10k1/hbEDNJGo+rX+MiEhAUB7s8N169bBwcFB/5xGo8GJEycQERFh1ACJiKxJZkcHaE5/EYmrXwlQeno6gPYRoMzMTCgUCv1zCoUC4eHheP75540bIRGRFcku5B5gROagXwnQwYMHAQDLli3D5s2buUUEEVE/1Da14lJFPQAgxJf/fhKJyaAWpB9++KGx4yAisno5xbUAAB+1Cu5OSpGjIbJtBiVA9fX1eOONN5CUlISysjJotdouz1vDZqhERMama4DI+h8i8RmUAD3xxBM4fPgwHnnkEfj4+PS4IoyIiLrSN0BkB2gi0RmUAH3//ff49ttvMXXqVGPHQ0RktbILuQcYkbkwqA/QkCFD4ObmZuxYiIisVmOLBhfK2muAuAKMSHwGJUCvvvoqXn75ZTQ0NBg7HiIiq3S2pAZaAXB3UsDLhQXQRGIzaAps48aNyM3NhZeXF0aMGAG5vGs797S0NKMER0RkLbI69f9h3SSR+AxKgObPn2/kMIiIrFsW63+IzIpBCdD69euNHQcRkVXjCjAi82JQDRAAVFVV4X//93+xdu1aXLt2DUD71FdhYaHRgiMisgbNbRqcL20vgGYPICLzYNAI0OnTpxEXFwe1Wo0rV67gySefhJubG7788kvk5eXh448/NnacREQW60JpHVo1AtT2cvgPsRc7HCKCgSNACQkJeOyxx3DhwgWoVCr98VmzZuHHH380WnBERNbgRgG0CwugicyEQQnQyZMn8bvf/a7bcT8/P5SUlPT7elu2bMGIESOgUqkwefJkJCcn3/T87du3IzAwECqVCmFhYfjuu+96Pfepp56CRCLBpk2b+h0XEZEx6Ot/OP1FZDYMSoCUSiVqamq6HT9//jw8PDz6da1t27YhISEB69evR1paGsLDwzFjxgyUlZX1eP7Ro0exZMkSLF++HOnp6Zg/fz7mz5+PrKysbufu3LkTx48fh6+vb79iIiIyJt0KsBA2QCQyGwYlQHPnzsV///d/o7W1FQAgkUiQl5eHF198EQsXLuzXtd566y08+eSTWLZsGYKDg/H+++/DwcEBH3zwQY/nb968GTNnzsQLL7yAoKAgvPrqq5g0aRLefffdLucVFhbimWeewSeffNKtTxER0WBp02iRU6xbAs8VYETmwqAEaOPGjairq4OnpycaGxsxbdo0jBkzBs7Oznjttdf6fJ2WlhakpqYiLi7uRkBSKeLi4nDs2LEeX3Ps2LEu5wPAjBkzupyv1WrxyCOP4IUXXkBISEifYmlubkZNTU2Xh7HVNLViy8GLeOL/TkIQBKNfn4jMT255PZrbtHBS2mHEUEexwyGiDgatAlOr1di/fz+OHDmCU6dOoa6uDpMmTeqWmNxKRUUFNBoNvLy8uhz38vLC2bNne3xNSUlJj+d3rj168803YWdnh1WrVvU5lg0bNuDPf/5zP6I3zN8OXEBTqxbp+VWYNGyIyd+PiMSV2VEAHezrAqmUBdBE5sKgBEhn6tSpZrcjfGpqKjZv3oy0tLR+rbZYu3YtEhIS9L+vqalBQECAUWNzUckxO8wXO9IKkJicxwSIyAboV4CxAJrIrBg0BbZq1Sq888473Y6/++67ePbZZ/t8HXd3d8hkMpSWlnY5XlpaCm9v7x5f4+3tfdPzf/rpJ5SVlWHYsGGws7ODnZ0drl69ij/84Q8YMWJEr7EolUq4uLh0eZjC4pj2pOqbU8WobWo1yXsQkfnIZgdoIrNkUAK0Y8eOHkd+pkyZgi+++KLP11EoFIiMjERSUpL+mFarRVJSEmJjY3t8TWxsbJfzAWD//v368x955BGcPn0aGRkZ+oevry9eeOEF7N27t8+xmUrU8CEY5eGIxlYNdp8uFjscIjIhrVZAdlFHATRXgBGZFYOmwCorK6FWd//L7OLigoqKin5dKyEhAY8++iiioqIQExODTZs2ob6+HsuWLQMALF26FH5+ftiwYQMAYPXq1Zg2bRo2btyI2bNnIzExESkpKdi6dSsAYOjQoRg6dGiX95DL5fD29sb48eMN+bhGJZFIsDg6AK9/dxaJJ/OxJGaY2CERkYlcrqxHQ4sGKrkUo9xZAE1kTgwaARozZgz27NnT7fj333+PUaNG9eta8fHx+Otf/4qXX34ZERERyMjIwJ49e/SFznl5eSguvjFSMmXKFHz66afYunUrwsPD8cUXX2DXrl0IDQ015KOIYsEkf8hlEpzKr9IvjyUi66Or/wnycYGdzOCtF4nIBAwaAUpISMDKlStRXl6Ou+++GwCQlJSEjRs3GtRxeeXKlVi5cmWPzx06dKjbsUWLFmHRokV9vv6VK1f6HZMpuTsp8atgL3yXWYJtJ/Pxyty+LdUnIsuin/5iATSR2TEoAXr88cfR3NyM1157Da+++ioAYMSIEXjvvfewdOlSowZoreKjh+G7zBJ8mVaANfcFQiWXiR0SERlZ5z3AiMi8GDwm+/TTT6OgoAClpaWoqanBpUuXmPz0w+1j3OHnao+apjbsze7//mlEZN4EQdAnQCEcASIyOwOalC4vL8e5c+eQkZHR7+JnWyeTSrAoyh8AkJicL3I0RGRs+dcaUdPUBoVMinFezmKHQ0S/YFACVF9fj8cffxw+Pj648847ceedd8LHxwfLly9HQ0ODsWO0WouiAiCRAMcuVeJKRb3Y4RCREel2gB/v7QyFHQugicyNQX8rExIScPjwYXzzzTeoqqpCVVUVvvrqKxw+fBh/+MMfjB2j1fJztce0cR4AgG0pHAUisias/yEybwY3QvzXv/6F++67T981edasWfjnP//Zr0aIBCyObu8M/UVqAVo1WpGjISJjyepYAcb6HyLzZFAC1NDQ0G1DUgDw9PTkFFg/3R3oBXcnBcprm3HwbJnY4RCREQiCgGz9CBATICJzZFACFBsbi/Xr16OpqUl/rLGxEX/+85973cKCeqawk2LhpPZi6G0nOQ1GZA1KappQWd8CmVSCQG8WQBOZI4P6AG3atAkzZ86Ev78/wsPDAQCnTp2CSqUyi/22LM1D0QH4x4+XcPBcGUqqm+CtVokdEhENQFZh+/TXWE8n9vgiMlMGjQCFhYXhwoUL2LBhAyIiIhAREYE33ngDFy5cQEgIuxr312gPJ8SMdINWALazGJrI4rH/D5H56/cIUGtrKwIDA7F79248+eSTpojJJi2ODkDy5WvYlpKPFXeNgVQqETskIjJQdhFXgBGZu36PAMnl8i61P2Qc94X6wFllh4LrjTiaWyl2OEQ0AJksgCYyewZNga1YsQJvvvkm2trajB2PzbJXyDA/wg8AkHgyT+RoiMhQZbVNKK1phkQCBPtwBIjIXBlUBH3y5EkkJSVh3759CAsLg6OjY5fnv/zyS6MEZ2viowPw7+NXsS+7FNfqW+DmqBA7JCLqJ90O8KPcHeGoNOifWCIaBAb97XR1dcXChQuNHYvNC/VTI8xPjczCanyZVoAn7hgldkhE1E/s/0NkGQxKgD788ENjx0Ed4qMDkFlYjW0n87H89pGQSFgMTWRJdEvgQ7kCjMis9asGSKvV4s0338TUqVMRHR2NNWvWoLGx0VSx2aS5Eb5QyaW4UFaHtLwqscMhon7SbYIawhVgRGatXwnQa6+9hpdeeglOTk7w8/PD5s2bsWLFClPFZpNcVHLMDvMFAGxjMTSRRalqaEHB9fb/FLIHEJF561cC9PHHH+Pvf/879u7di127duGbb77BJ598Aq2Wm3ga05KY9g1SvzlVjNqmVpGjIaK+0hVAD3NzgNpeLnI0RHQz/UqA8vLyMGvWLP3v4+LiIJFIUFRUZPTAbFnk8CEY7eGIxlYNvjlVLHY4RNRHWYVsgEhkKfqVALW1tUGl6rpPlVwuR2srRymMSSKRYHH0MACcBiOyJFkdI0Cc/iIyf/1aBSYIAh577DEolUr9saamJjz11FNdegGxD9DAPTDJD/+z9yxOFVTjTFENgn35P0oic5fFJfBEFqNfCdCjjz7a7dhvfvMbowVDN7g7KfGrYC98l1mCz1Py8cpcbjJLZM5qm1pxuaIeABDK/7AQmb1+JUDs/zO4FkcPw3eZJfgyrQBr7guESi4TOyQi6sWZjukvX7UKQ52UtzibiMRm0F5gNDhuH+MOP1d71DS1YU9WidjhENFN6Ot/OP1FZBGYAJkxqVSCh6Lal8Rzg1Qi86bfAoMF0EQWgQmQmVsU5Q+JBDh+6Zq+voCIzI+uAzSXwBNZBiZAZs7X1R7TxnkAAD5PyRc5GiLqSWOLBhfL6gBwBRiRpWACZAF0PYG+SC1Aq4Zdt4nMTU5JDbRC++pNT2cWQBNZAiZAFuCeIE+4OylQXtuMA2fLxA6HiH4hu1MHaIlEInI0RNQXTIAsgFwmxcJIfwDAtpOcBiMyN5ksgCayOEyALER8x2qwQ+fKUFzdKHI0RNRZVmH7EngWQBNZDiZAFmKUhxNiRrpBKwBfpBSIHQ4RdWhu0+B8aS0A7gFGZEmYAFmQJTHto0DbUvKh1QoiR0NEAHC+pA5tWgGuDnL4D7EXOxwi6iMmQBbkvlAfOKvsUHC9EUdyK8QOh4jQqf+Pr5oF0EQWhAmQBVHJZXhgoh8AIJHF0ERmQbcDfAjrf4gsChMgCxMf3T4Nti+7BNfqW0SOhoh0e4BxBRiRZWECZGFCfNUI81OjVSPgyzQWQxOJqVWjRU6xbgUYEyAiS8IEyAIt1hVDn8yHILAYmkgsueV1aGnTwklph+FuDmKHQ0T9wATIAs0N94W9XIYLZXVIy7sudjhENiuzoL3+J9jXBVIpC6CJLImd2AFQ/zmr5Jg9wQdfpBbgT7uyEeJrXsWXjgoZVt49Fh5mtCfSVxmFAIB5EX4iR2Jbtp3Mg6PSDvdP8BU7FL2S6ib8/dBFNLRoBnytLHaAJrJYTIAs1JKYYfgitQA5xTX6GgRzohEE/L/5YWKHAQC4XFGP1YkZkEiAqWPc4e5kPomZNcssqMaLOzIhlQCThg2Br6t59Mj5675z+CLVuPVzk4a7GvV6RGR6TIAsVOTwIXj/N5G4XFEvdihdlNc244Mjl/FVRhH+OCsY9gqZ2CHp908ThPY9m+4a7ylyRLbhs5N5AACtAGxPKcDquLEiRwTUNLVi9+kiAMDv7hwFVwfFgK851FGB+0J9BnwdIhpcTIAs2MxQb7FD6EarFbA/pwT51xrxXWaxfhNXsbRqtF3+t5/NBGhQNLS04euMIv3vP0/Jx8q7x0Amcp3M1xlFaGrVYqynE9bcF8jGhUQ2jEXQZFRSqUS/cas57Fx/4GwZKuqa9b/XbVpJpvXt6WLUNbchwM0eans5CqsaceSi+N3LdX8m46MDmPwQ2TgmQGR0i6ICIJUAyVeuIbe8TtRYdF94EQGuAG5sW0Cmpbvvi6OH6buXi50QZxVWI7OwGgqZFAsmiTsySUTiYwJERuflosLdge3TTGJ+6RVXN+LQuTIAwJ/nhgAACq43oqqBHbRN6WJZLVKuXodMKsGDkf43upefKUFlp9G4wab7s3hviBfcHAde+0NElo0JEJlEfPQwAMCO1AK0tGlFiWF7SgG0AhAz0g3hAa4YPrS9UV12EafBTCkxuT3RuGu8J7xcVAjycUG4v657eaEoMTW2aLCroxXC4o4/m0Rk25gAkUncNd4Dns5KVNa3ICmndNDfX6sVOk3DtI9A6Hq16Hq3kPE1t2nwZbou0QjQH9clxIkn80TpXv59VjFqm9rgP8QeU0YPHfT3JyLzwwSITMJOJsWDHSvAxNi5/khuBQqrGuGsssOssPYlyrrdurM4AmQyP5wpw7X6Fni5KDF9vIf++JxwH9jLZcgtr0fq1cHvXq77MxgfFcCOzUQEgAkQmZCu9uPHC+UorGoc1PfWfeE9MNEPKnl7LyLdCFA2R4BMJrGj98+iyADYyW788+KskmNOuE/HOYObEF8qr0Py5WuQStoL9ImIACZAZELDhzpiyuihEATg80H80rtW34J92SUAbiRhAPRbhlyqqEdtU+ugxWMr8q814OeOpe4P9ZBo6KbBvj1djJpBvP+6qdC7xnvCW60atPclIvPGBIhMSpeAbE/Jh0Y7OLUfX6YVoFUjIMxPjZBOezQNdVLCt+ML8AynwYxue0o+BAGYOmYohg3tvjP6pGGuGOvphMZWTZcmiabU0qbFjrT2Rpidk2EiIiZAZFIzQryhtpejqLoJP10oN/n7CYKgn2JZHNP9Cy/Er6MQmgmQUWm0Aj5P0SUaPa+ykkgk+iRksNojHDhbioq6Fng4K3FXIDuAE9ENTIDIpFRy2aA2wkvLu46LZXWwl8swN7z7DuRhfqwDMoUfz5ejpKYJrg5yzAjx6vW8BZP8oZBJkVlYPSir8XTJ8IOR/pDL+M8dEd3AfxHI5HQjMfvPlKK81rSN8HQ9aGZP8IGzSt7t+VD9SjAmQMakK35eMNEfSrveN8B1c1Tg3o4E6fMU0ybEhVWNOHy+fdQxnsXPRPQLTIDI5AK9XRAR4Io2rYAv0wpu/QID1Ta1YvfpYgBde9B0plsJdrGsDo0tGpPFYkvKapuQlNPecbsvdTa6RoQ70wvR1Gq6n4GuJil21FCMcHc02fsQkWViAkSDYnGn2g9TNcL7+lQRGls1GOPphMjhQ3o8x9NFBQ9nJbQCkFPCOiBj2JFaiDatgInDXDHe2/mW508ZPRT+Q+xR29SG7zKLTRKTRitge0dNUk+1YERETIBoUNwf7gsHhQyXKupx8oppGuF17vx8s52+QzuWw7MOaOAEQcC2jumv3kbdfkkqleinpEzVE+jni+2NMNX2cswI8TbJexCRZWMCRIPCSWmHORPai5J19SLGlF1UjdMF1ZDLJPqi696E6laCFXIEaKBOXL6GK5UNcFTIcP+E7kXnvXkwyh9SCZB8+RouldcZPS5dUta5ESYRUWdMgGjQ6KYivsssRnWjcRvh6Rot3hvsjaFOypueq+sNxELogdONus2N8IWj0q7Pr/NR22P6+PZl6duMXAxdUdeM/Wfa959j7x8i6o1ZJEBbtmzBiBEjoFKpMHnyZCQnJ9/0/O3btyMwMBAqlQphYWH47rvvujz/yiuvIDAwEI6OjhgyZAji4uJw4sQJU34E6oOIAFeM93JGU6sWX2cYb1fwplYNdnZswNmXLzzdSrDzpbVobmMhtKGqG1r1NTy99f65Gd2U2Y7UArRqtEaLS9cIMzzAFUE+Lka7LhFZF9EToG3btiEhIQHr169HWloawsPDMWPGDJSVlfV4/tGjR7FkyRIsX74c6enpmD9/PubPn4+srCz9OePGjcO7776LzMxM/PzzzxgxYgTuvfdelJebvhEf9a5zIzxj1n58n1WMmqY2+Lna4/Yx7rc838/VHq4OcrRqBJwvMf70i63YlVGI5jYtAr2dEe6vvvULfuGuQE94OCtRUdeCpJxSo8TUpREmR3+I6CZET4DeeustPPnkk1i2bBmCg4Px/vvvw8HBAR988EGP52/evBkzZ87ECy+8gKCgILz66quYNGkS3n33Xf05v/71rxEXF4dRo0YhJCQEb731FmpqanD69OnB+ljUiwcm+kEhkyK7qMZojfB0vX/io/u207dEItEvh+c0mGEEQcBnyTeKn29WdN4buUyKByP9ARgvIU65eh2XyuvhoJBhTg+NMImIdERNgFpaWpCamoq4uDj9MalUiri4OBw7dqzH1xw7dqzL+QAwY8aMXs9vaWnB1q1boVarER4e3msszc3NqKmp6fIg4xviqMCM0PZVOcYohr5UXocTHTt9675M+yJE1xCRK8EMkllYjbMltVDYSTH/FkXnN6PbNPXw+XIUVTUOOC5dMnz/BB849aMmiYhsj6gJUEVFBTQaDby8urbO9/LyQklJSY+vKSkp6dP5u3fvhpOTE1QqFd5++23s378f7u69T49s2LABarVa/wgI4PC5qeimJr5KL0JDS9uArqXbf2raOA/4utr3+XVh3BNsQHQjNveFesPVQWHwdUa6O+K2UW4QBOj79hiqurEV32a2b7JqSE0SEdkW0afATOWuu+5CRkYGjh49ipkzZ+Khhx7qta4IANauXYvq6mr9Iz9/cDZrtEWxo4ZimJsDapvb8F1mz4luX7RqtPgi9eYbcPZGNwWWU1xj1AJcW9DQ0qbfzd0Yq6yWxLT/7D5PyYdGa3iTzK9PFaGpVYtxXk6YNMx1wHERkXUTNQFyd3eHTCZDaWnXAsjS0lJ4e/fcvMzb27tP5zs6OmLMmDG47bbb8K9//Qt2dnb417/+1WssSqUSLi4uXR5kGlJp513BDZ8GS8opQ0VdM9ydlLgnqH87fQ9zc4Cz0g4tbVrkmqAPjTXbfboYdc1tGD7UAbeNHDrg680I8YbaXo7Cqkb8fLHC4Ovo/izFRw8zqCaJiGyLqAmQQqFAZGQkkpKS9Me0Wi2SkpIQGxvb42tiY2O7nA8A+/fv7/X8ztdtbjbtRpzUdw9GtjfCO3mlffd2Q+i+8AzZ6VsqlSDYV1cHxGmw/tD1/ulr0fmtqOQyffNKQxPirMJqZBXWQCGT3rIRJhERYAZTYAkJCfjnP/+J//u//0NOTg6efvpp1NfXY9myZQCApUuXYu3atfrzV69ejT179mDjxo04e/YsXnnlFaSkpGDlypUAgPr6erz00ks4fvw4rl69itTUVDz++OMoLCzEokWLRPmM1J2Xiwp3B3Y0wjPgS6+o807fBk7D3OgIzULovrpQWovUq9chk0rw4KS+F53fiu5nuP9MKSrq+v8fFV1B/b0hXnBzNLwmiYhsh+gJUHx8PP7617/i5ZdfRkREBDIyMrBnzx59oXNeXh6Ki29smDhlyhR8+umn2Lp1K8LDw/HFF19g165dCA0NBQDIZDKcPXsWCxcuxLhx4zBnzhxUVlbip59+QkhIiCifkXqmq9vZkVaIlrb+1eF8kVoArQBMHumGkQbu9K1riJjNpfB9phv9uTvQE54uKqNdN8jHBeH+arRqBOxM61+TzMYWDb5Kb69JWsziZyLqI7NYJ7py5Ur9CM4vHTp0qNuxRYsW9Tqao1Kp8OWXXxozPDKRu8Z7wNNZibLaZvyQU4pZYT59ep1WK9zY+HQAO33rCqGzi2qg0QqQGWE6x5o1t2nwZUfHbVM0GVwcMwynCjKReDIPT9wxss91PN9lFqO2uQ0BbvaYMnrgNUlEZBtEHwEi22Unk2JRVP8b4el2+nZR2eG+0L4lTT0Z5eEElVyKhhYNLlfUG3wdW7H/TCmu1bfAy0WJaeM8jH79OeG+cFDIkFtej5Sr1/v8On1NUpRxapKIyDYwASJR6Rrh/XShHAXXG/r0Gt0X3kB3+pZJJQj24TRYX+nu+0NRAbDrZ9F5Xzgp7XD/hPaEVtfQ8FZyy+uQfEXXCJO9u4io75gAkaiGD3XElNFDIQg3mhreTGVdM/adae8dZIxmdyyE7pv8aw346UL7EnVd0moKup/pt5lFqGlqveX5uqTsrvGe8FYbryaJiKwfEyASnW4F0PY+NMLbmV6IVo2ACf5q/TL2gdDvCcal8De1PaU90bh9jDsC3BxM9j6ThrlirKcTmlq1+maLvWlp02KHvhEmR3+IqH+YAJHoZoR4w9VBjuLqJvx4obzX8zrv9G2sLzz9CFBRNQTB8C7E1kyjFfSjc6ZONCQSCRZ3dIbedou6sKScUlTWt8DTWalvqUBE1FdMgEh0XRrh3aT2I/Vqe9NEe7kMc4200/dYLycoZFLUNrUh/9rAN+O0RofPl6GkpglDHOS4N8Tr1i8YoAcm+kEhkyKzsPqmU5O6ZPjBSH+T1CQRkXXjvxpkFnQjCz/klKK8tudGeLovvPsn+MBZJTfK+8plUgT6OANoHwWi7nQFyQsm+UNpZ3jReV+5OSr0iVZvo0AF1xv0o4WmrEkiIuvFBIjMQqC3CyICXNGmFbAjrXsxdE1TK7493d4QcyC9f3oS4stC6N6U1TYh6Wz7JsKDWWeja2i4K6MQjS2abs9vTymAILRvrDvCwEaYRGTbmACR2Vis3yA1v1s9zjenitDYqsEYTydMGjbEqO+r6widVcRC6F/akVoIjVbApGGuGOflPGjvO2X0UAS42aO2qQ3fZxV3eU6jFfRF2cZOhonIdjABIrMxJ9wXjgoZLlfUI/nytS7P6Ts/RwcYfafv0E4jQCyEvkEQBP0+bYO9xYRUKkF8x9TWL5tk/nShHEXVTVDbyzEjxHtQ4yIi68EEiMyGo9IOczqKmzvXfmQXVeN0QTXkMgkWGHEDTp3x3s6QSSW4Vt+C4uomo1/fUh2/dA1XKhvgpLTD7AmGd9w21IORAZBKgOTL15BbXqc/bqxGmERk25gAkVnR1Zl8m1mM6ob2Rni6L7x7Q7xNstO3Si7DWE8nAKwD6kw3+jMn3BeOysHfNtBbrcJd49uXt3/e8WegvLYZ+8+UAmDvHyIaGCZAZFYiAlwx3ssZzW1afHWqEE2tGuw04QacOjf6AbEOCACqG1rxXVZ7x21T3vdb0SU5O9IK0NKmxZdpBWjTCggPcEWQz8AbYRKR7WICRGZFIpHov/Q+S85v3+m7qQ3+Q+wxdbS7yd43tKOrdDZHgAC0r75qadMi0NsZE/zVosVxV6AnPJyVqKhrQVJOaZdaMCKigWACRGZnwSQ/KOykyCmuwcZ95wGYfqfvMP8bHaFtnSAI+Cy5ffprScwwoxed94dcJsWiyPa6r//3bQ4uVdTDQSHT14oRERmKCRCZHVcHBWZ2rO4prGps3+k7yvjFz50F+bhAIgFKa5pRVmvbhdCnC6pxtqQWCjsp5kf4iR2OvtFhYVV7p+45E3zhJEJNEhFZF/4rQmZpcXQAvj7Vvhnm9PGe8FHbm/T9HBR2GO3hhItldcguqoHneNPvLH6tvgXvJF1AXXObyd+rP3KK2+ugZoV6Q+1gnI7bAzHC3RGxo4bi2KVKAEA8e/8QkREwASKzdNuooRjp7ojLFfVYEjM4PWhCfV3aE6DCav3qI1P624EL+OjoFZO/j6EWD9J974tfTx6GY5cqEejtjIkBrmKHQ0RWgAkQmSWpVIIPH4vGudJa/CrY9BtwAu0rwXZlFCFzEAqhO69ue2zKCHi5mH7EqT+GD3XAbaOGih2G3v0TfCCTShDmpxa1JomIrAcTIDJbI9wdB3Wfpxt7gpl+Kfze7BJUNbTCR63CuvuDITNhgbc1kEgkmBU2+M0Yich6sQiaqENwx1L4wqpGXK9vMel76ZZzL4oKYPJDRCQCJkBEHdT2cgwf6gAAyDZhQ8SrlfU4mlsJiQR4yMSr24iIqGdMgIg60W+MasJ+QJ937GR+x1gP+A9xMNn7EBFR75gAEXUS4tc+DWaqPcHaNFpsTykAwG7GRERiYgJE1IluBMhUU2AHz5WjrLYZbo4KxAUNzuo2IiLqjgkQUSe6TVEvV9SjtqnV6NfX7bC+sGO7DyIiEgf/BSbqxM1RAT/X9q7TZ4w8ClRS3YQDZ8sAAPHR5tNkkIjIFjEBIvqFkI7l8FlGToB2pBVAKwDRI4ZgjKeTUa9NRET9wwSI6Bd002DGLITWagV97x+O/hARiY8JENEvhJpgJdixS5XIu9YAZ6UdZoV5G+26RERkGCZARL+gWwmWW16Hhhbj7NSe2DH6M2+iLxwU3IGGiEhsTICIfsHTRQUPZyW0ApBTXDvg612vb8HerBIAwGJOfxERmQUmQEQ9CO0ohM42QkfonemFaNFoEeLroq8vIiIicTEBIuqBsQqhBeFG8TM7PxMRmQ8mQEQ9CNHtCVY4sKXw6flVOFdaC5VcirkRfsYIjYiIjIAJEFEPwvzbE6DzpbVobtMYfJ1tye2jP7PCfKC2lxslNiIiGjgmQEQ98FWrMMRBjjatgPMldQZdo665Dd+cLgLA4mciInPDBIioBxKJ5EYdkIGF0LtPFaGhRYNR7o6IHjHEmOEREdEAMQEi6oWuDijTwELoRH3n5wBIJBKjxUVERAPHBIioF7qO0NkGJEBnS2qQkV8FO6kECyP9jR0aERENEBMgol7oOkLnlNSiVaPt12sTO4qffxXsBXcnpdFjIyKigWECRNSLYW4OcFbaoaVNi4tlfS+EbmrVYGd6IYD26S8iIjI/TICIeiGVShDs2/+NUfdml6C6sRW+ahXuGOthqvCIiGgAmAAR3YRuJVh2Ud8bIuo6Py+KCoBMyuJnIiJzxASI6CZ0hdB9HQG6WlmPo7mVkEiAhzj9RURktpgAEd2ErhD6THENNFrhlufrRn/uHOsBP1d7k8ZGRESGYwJEdBOjPJxgL5ehoUWDyxX1Nz23TaPF9tQCANz4lIjI3DEBIroJWT8KoQ+eK0d5bTOGOipwT5DXYIRHREQGYgJEdAuhfUyAtp3MAwA8GOkPhR3/ahERmTP+K010CyF92BOspLoJB86WAWDxMxGRJWACRHQLukLo7MIaaHsphN6RVgCtAMSMcMNoD6fBDI+IiAzABIjoFsZ6OUEhk6K2uQ351xu6Pa/VCkjsmP5i52ciIsvABIjoFuQyKQJ9nAEAWYXdGyIeu1SJ/GuNcFbZYVaYz2CHR0REBmACRNQHIb691wEldvT+mR/hB3uFbFDjIiIiwzABIuqD3jpCX69vwd6sEgCc/iIisiRMgIj6QF8IXVQDQbhRCL0zvRAtGi1C/Vz0+4YREZH5YwJE1AfjvZ1hJ5XgWn0LiqubAACC0Ln4eZiY4RERUT8xASLqA5VchrFe7YXQmR3TYOn5VThfWgeVXIp5Eb5ihkdERP3EBIioj3QdobM7EqBtye3Fz7PDfOGikosWFxER9R8TIKI+CtV3hK5BXXMbvjldBABYHMPiZyIiS8MEiKiPOq8E232qCA0tGozycETU8CEiR0ZERP1lFgnQli1bMGLECKhUKkyePBnJyck3PX/79u0IDAyESqVCWFgYvvvuO/1zra2tePHFFxEWFgZHR0f4+vpi6dKlKCoqMvXHICsX5OMCiQQoq23GP368BABYHB0AiUQicmRERNRfoidA27ZtQ0JCAtavX4+0tDSEh4djxowZKCsr6/H8o0ePYsmSJVi+fDnS09Mxf/58zJ8/H1lZWQCAhoYGpKWlYd26dUhLS8OXX36Jc+fOYe7cuYP5scgKOSjs9Pt8Xa6oh1wmwYJJ/iJHRUREhpAInZuaiGDy5MmIjo7Gu+++CwDQarUICAjAM888gzVr1nQ7Pz4+HvX19di9e7f+2G233YaIiAi8//77Pb7HyZMnERMTg6tXr2LYsL4tV66pqYFarUZ1dTVcXFwM+GRkjZ5NTMeujPbRxFlh3vj7w5EiR0RERJ319ftb1BGglpYWpKamIi4uTn9MKpUiLi4Ox44d6/E1x44d63I+AMyYMaPX8wGguroaEokErq6uvZ7T3NyMmpqaLg+iX+rc7JC9f4iILJeoCVBFRQU0Gg28vLy6HPfy8kJJSUmPrykpKenX+U1NTXjxxRexZMmSm2aCGzZsgFqt1j8CAriyh7qL7Ch4DnCzx+1j3EWOhoiIDCV6DZAptba24qGHHoIgCHjvvfdueu7atWtRXV2tf+Tn5w9SlGRJJg4bgn89GoWPH58MmZTFz0RElspOzDd3d3eHTCZDaWlpl+OlpaXw9vbu8TXe3t59Ol+X/Fy9ehUHDhy4ZR2PUqmEUqk04FOQrbknyOvWJxERkVkTdQRIoVAgMjISSUlJ+mNarRZJSUmIjY3t8TWxsbFdzgeA/fv3dzlfl/xcuHABP/zwA4YOHWqaD0BEREQWSdQRIABISEjAo48+iqioKMTExGDTpk2or6/HsmXLAABLly6Fn58fNmzYAABYvXo1pk2bho0bN2L27NlITExESkoKtm7dCqA9+XnwwQeRlpaG3bt3Q6PR6OuD3NzcoFAoxPmgREREZDZET4Di4+NRXl6Ol19+GSUlJYiIiMCePXv0hc55eXmQSm8MVE2ZMgWffvop/vSnP+Gll17C2LFjsWvXLoSGhgIACgsL8fXXXwMAIiIiurzXwYMHMX369EH5XERERGS+RO8DZK7YB4iIiMjyWEQfICIiIiIxMAEiIiIim8MEiIiIiGwOEyAiIiKyOUyAiIiIyOYwASIiIiKbwwSIiIiIbA4TICIiIrI5TICIiIjI5oi+FYa50jXIrqmpETkSIiIi6ivd9/atNrpgAtSL2tpaAEBAQIDIkRAREVF/1dbWQq1W9/o89wLrhVarRVFREZydnSGRSPr0mpqaGgQEBCA/P5/7hw0i3ndx8L4PPt5zcfC+i8PQ+y4IAmpra+Hr69tlM/Vf4ghQL6RSKfz9/Q16rYuLC/+SiID3XRy874OP91wcvO/iMOS+32zkR4dF0ERERGRzmAARERGRzWECZERKpRLr16+HUqkUOxSbwvsuDt73wcd7Lg7ed3GY+r6zCJqIiIhsDkeAiIiIyOYwASIiIiKbwwSIiIiIbA4TICIiIrI5TICMZMuWLRgxYgRUKhUmT56M5ORksUOyKj/++CPmzJkDX19fSCQS7Nq1q8vzgiDg5Zdfho+PD+zt7REXF4cLFy6IE6wV2bBhA6Kjo+Hs7AxPT0/Mnz8f586d63JOU1MTVqxYgaFDh8LJyQkLFy5EaWmpSBFbh/feew8TJkzQN4CLjY3F999/r3+e99z03njjDUgkEjz77LP6Y7zvpvHKK69AIpF0eQQGBuqfN9V9ZwJkBNu2bUNCQgLWr1+PtLQ0hIeHY8aMGSgrKxM7NKtRX1+P8PBwbNmypcfn/+d//gfvvPMO3n//fZw4cQKOjo6YMWMGmpqaBjlS63L48GGsWLECx48fx/79+9Ha2op7770X9fX1+nOee+45fPPNN9i+fTsOHz6MoqIiLFiwQMSoLZ+/vz/eeOMNpKamIiUlBXfffTfmzZuH7OxsALznpnby5En84x//wIQJE7oc5303nZCQEBQXF+sfP//8s/45k913gQYsJiZGWLFihf73Go1G8PX1FTZs2CBiVNYLgLBz507977VareDt7S385S9/0R+rqqoSlEql8Nlnn4kQofUqKysTAAiHDx8WBKH9PsvlcmH79u36c3JycgQAwrFjx8QK0yoNGTJE+N///V/ecxOrra0Vxo4dK+zfv1+YNm2asHr1akEQ+GfdlNavXy+Eh4f3+Jwp7ztHgAaopaUFqampiIuL0x+TSqWIi4vDsWPHRIzMdly+fBklJSVdfgZqtRqTJ0/mz8DIqqurAQBubm4AgNTUVLS2tna594GBgRg2bBjvvZFoNBokJiaivr4esbGxvOcmtmLFCsyePbvL/QX4Z93ULly4AF9fX4waNQoPP/ww8vLyAJj2vnMz1AGqqKiARqOBl5dXl+NeXl44e/asSFHZlpKSEgDo8Wege44GTqvV4tlnn8XUqVMRGhoKoP3eKxQKuLq6djmX937gMjMzERsbi6amJjg5OWHnzp0IDg5GRkYG77mJJCYmIi0tDSdPnuz2HP+sm87kyZPx0UcfYfz48SguLsaf//xn3HHHHcjKyjLpfWcCRER9smLFCmRlZXWZmyfTGT9+PDIyMlBdXY0vvvgCjz76KA4fPix2WFYrPz8fq1evxv79+6FSqcQOx6bcd999+l9PmDABkydPxvDhw/H555/D3t7eZO/LKbABcnd3h0wm61aRXlpaCm9vb5Gisi26+8yfgemsXLkSu3fvxsGDB+Hv768/7u3tjZaWFlRVVXU5n/d+4BQKBcaMGYPIyEhs2LAB4eHh2Lx5M++5iaSmpqKsrAyTJk2CnZ0d7OzscPjwYbzzzjuws7ODl5cX7/sgcXV1xbhx43Dx4kWT/nlnAjRACoUCkZGRSEpK0h/TarVISkpCbGysiJHZjpEjR8Lb27vLz6CmpgYnTpzgz2CABEHAypUrsXPnThw4cAAjR47s8nxkZCTkcnmXe3/u3Dnk5eXx3huZVqtFc3Mz77mJ3HPPPcjMzERGRob+ERUVhYcfflj/a973wVFXV4fc3Fz4+PiY9s/7gEqoSRAEQUhMTBSUSqXw0UcfCWfOnBF++9vfCq6urkJJSYnYoVmN2tpaIT09XUhPTxcACG+99ZaQnp4uXL16VRAEQXjjjTcEV1dX4auvvhJOnz4tzJs3Txg5cqTQ2NgocuSW7emnnxbUarVw6NAhobi4WP9oaGjQn/PUU08Jw4YNEw4cOCCkpKQIsbGxQmxsrIhRW741a9YIhw8fFi5fviycPn1aWLNmjSCRSIR9+/YJgsB7Plg6rwITBN53U/nDH/4gHDp0SLh8+bJw5MgRIS4uTnB3dxfKysoEQTDdfWcCZCR/+9vfhGHDhgkKhUKIiYkRjh8/LnZIVuXgwYMCgG6PRx99VBCE9qXw69atE7y8vASlUincc889wrlz58QN2gr0dM8BCB9++KH+nMbGRuH3v/+9MGTIEMHBwUF44IEHhOLiYvGCtgKPP/64MHz4cEGhUAgeHh7CPffco09+BIH3fLD8MgHifTeN+Ph4wcfHR1AoFIKfn58QHx8vXLx4Uf+8qe67RBAEYWBjSERERESWhTVAREREZHOYABEREZHNYQJERERENocJEBEREdkcJkBERERkc5gAERERkc1hAkREREQ2hwkQEZEZunjxIl5//XU0NjaKHQqRVWICREQGkUgk2LVr14Cvc+XKFUgkEmRkZAz4WoPlo48+gqur64CvM2LECGzatKnb8aamJjz44IPw9fU16W7YRLbMTuwAiMg8PfbYY6iqquo1ySkuLsaQIUMGNygzER8fj1mzZpns+s888wzmz5+Pxx57zGTvQWTrmAARkUG8vb3FDkE09vb2Jh2Z+ec//2myaxNRO06BEZFBfjkFVlBQgCVLlsDNzQ2Ojo6IiorCiRMnALRP9Ugkkm6Pzs6ePYspU6ZApVIhNDQUhw8f1j+n0WiwfPlyjBw5Evb29hg/fjw2b9580/iuX7+Ohx9+GB4eHrC3t8fYsWPx4Ycf6p9/8cUXMW7cODg4OGDUqFFYt24dWltb9c+fOnUKd911F5ydneHi4oLIyEikpKQA6D4Flpubi3nz5sHLywtOTk6Ijo7GDz/80CWesrIyzJkzB/b29hg5ciQ++eSTbjHn5eVh3rx5cHJygouLCx566CGUlpYCAKqrqyGTyfQxaLVauLm54bbbbtO//j//+Q8CAgJuel+IqB1HgIhowOrq6jBt2jT4+fnh66+/hre3N9LS0qDVagEAJ0+ehEajAdCezDz44IOQy+VdrvHCCy9g06ZNCA4OxltvvYU5c+bg8uXLGDp0KLRaLfz9/bF9+3YMHToUR48exW9/+1v4+PjgoYce6jGmdevW4cyZM/j+++/h7u6OixcvdikodnZ2xkcffQRfX19kZmbiySefhLOzM/7rv/4LAPDwww9j4sSJeO+99yCTyZCRkdEt5s6ff9asWXjttdegVCrx8ccfY86cOTh37hyGDRsGoH1KsaioCAcPHoRcLseqVatQVlamv4ZWq9UnP4cPH0ZbWxtWrFiB+Ph4HDp0CGq1GhERETh06BCioqKQmZkJiUSC9PR01NXV6V83bdo0A3+KRDZmwPvJE5FVevTRR4V58+b1+jwAYefOnYIgCMI//vEPwdnZWaisrLzldVetWiUMHz5cKCsrEwRBEC5fviwAEN544w39Oa2trYK/v7/w5ptv9nqdFStWCAsXLuz1+Tlz5gjLli27ZTw6f/nLX4TIyEj9752dnYWPPvqox3M//PBDQa1W3/R6ISEhwt/+9jdBEATh3LlzAgAhOTlZ/3xOTo4AQHj77bcFQRCEffv2CTKZTMjLy9Ofk52d3eV1CQkJwuzZswVBEIRNmzYJ8fHxQnh4uPD9998LgiAIY8aMEbZu3drnz0xkyzgFRkQDlpGRgYkTJ8LNze2m523duhX/+te/8PXXX8PDw6PLc7Gxsfpf29nZISoqCjk5OfpjW7ZsQWRkJDw8PODk5IStW7ciLy+v1/d6+umnkZiYiIiICPzXf/0Xjh492uX5bdu2YerUqfD29oaTkxP+9Kc/dbleQkICnnjiCcTFxeGNN95Abm5ur+9VV1eH559/HkFBQXB1dYWTkxNycnL018vJyYGdnR0iIyP1rwkMDOwyjZaTk4OAgIAuU1jBwcFwdXXV34dp06bh559/hkajweHDhzF9+nRMnz4dhw4dQlFRES5evIjp06f3GicR3cAEiIgGrC8FwQcPHsQzzzyDjz/+GBMmTOjX9RMTE/H8889j+fLl2LdvHzIyMrBs2TK0tLT0+pr77rsPV69exXPPPYeioiLcc889eP755wEAx44dw8MPP4xZs2Zh9+7dSE9Pxx//+Mcu13vllVeQnZ2N2bNn48CBAwgODsbOnTt7fK/nn38eO3fuxOuvv46ffvoJGRkZCAsLu2l8hrjzzjtRW1uLtLQ0/Pjjj10SoMOHD8PX1xdjx4416nsSWSsmQEQ0YBMmTEBGRgauXbvW4/MXL17Egw8+iJdeegkLFizo8Zzjx4/rf93W1obU1FQEBQUBAI4cOYIpU6bg97//PSZOnIgxY8bcdERGx8PDA48++ij+85//YNOmTdi6dSsA4OjRoxg+fDj++Mc/IioqCmPHjsXVq1e7vX7cuHF47rnnsG/fPixYsKBLEXVnR44cwWOPPYYHHngAYWFh8Pb2xpUrV/TPBwYG6j+Tzrlz51BVVaX/fVBQEPLz85Gfn68/dubMGVRVVSE4OBgA4OrqigkTJuDdd9+FXC5HYGAg7rzzTqSnp2P37t2s/yHqByZARNSr6upqZGRkdHl0/oLWWbJkCby9vTF//nwcOXIEly5dwo4dO3Ds2DE0NjZizpw5mDhxIn7729+ipKRE/+hsy5Yt2LlzJ86ePYsVK1bg+vXrePzxxwEAY8eORUpKCvbu3Yvz589j3bp1OHny5E1jf/nll/HVV1/h4sWLyM7Oxu7du/UJ1dixY5GXl4fExETk5ubinXfe6TK609jYiJUrV+LQoUO4evUqjhw5gpMnT+pf/0tjx47Fl19+iYyMDJw6dQq//vWv9QXgADB+/HjMnDkTv/vd73DixAmkpqbiiSee6DJyFhcXh7CwMDz88MNIS0tDcnIyli5dimnTpiEqKkp/3vTp0/HJJ5/okx03NzcEBQVh27ZtTICI+kPsIiQiMk+PPvqoAKDbY/ny5YIgdC2CFgRBuHLlirBw4ULBxcVFcHBwEKKiooQTJ07oi5x7egjCjSLoTz/9VIiJiREUCoUQHBwsHDhwQH/tpqYm4bHHHhPUarXg6uoqPP3008KaNWuE8PDwXuN/9dVXhaCgIMHe3l5wc3MT5s2bJ1y6dEn//AsvvCAMHTpUcHJyEuLj44W3335bX9jc3NwsLF68WAgICBAUCoXg6+srrFy5UmhsbBQEoXsR9OXLl4W77rpLsLe3FwICAoR3331XmDZtmrB69Wr9OcXFxcLs2bMFpVIpDBs2TPj444+F4cOH64ugBUEQrl69KsydO1dwdHQUnJ2dhUWLFgklJSVdPtfOnTsFAMJ7772nP7Z69WoBgHD27Nmb/kyJ6AaJIAiCCHkXEZFZWbBgAdasWYOYmBixQyGiQcAEiIhsXlVVFYYPH47r168DAKRSVgcQWTv+LScim+fi4oKYmBj4+/vjnXfeETscIhoEHAEiIiIim8MRICIiIrI5TICIiIjI5jABIiIiIpvDBIiIiIhsDhMgIiIisjlMgIiIiMjmMAEiIiIim8MEiIiIiGwOEyAiIiKyOf8fRA8JbyMXIqUAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "# changing to misclassification error\n",
    "MSE = [1 - x for x in cv_scores]\n",
    "\n",
    "# plot misclassification error vs k\n",
    "plt.plot(neighbors, MSE)\n",
    "plt.xlabel('Liczba sąsiadów')\n",
    "plt.ylabel('Procent błędów')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TF IDF Vectorizer\n",
    "\n",
    "Czasami, żeby wytrenować model nie da się zastosować bezpośrednio danego typu danych, ponieważ najczęściej wejściem do algorytmu ML jest wektor, macierz lub tensor.\n",
    "Dane tekstowe musimy również przekształcić do wektorów. Przydatny w tym przypadku jest TF IDF Vectorizer.\n",
    "Oto przyład z dokumentacji jak można z niego skorzystać (https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(4, 9)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "corpus = [\n",
    "    'This is the first document.',\n",
    "    'This document is the second document.',\n",
    "    'And this is the third one.',\n",
    "    'Is this the first document?',\n",
    "]\n",
    "vectorizer = TfidfVectorizer()\n",
    "X = vectorizer.fit_transform(corpus)\n",
    "vectorizer.get_feature_names_out()\n",
    "print(X.shape)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['and', 'document', 'first', 'is', 'one', 'second', 'the', 'third',\n",
       "       'this'], dtype=object)"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vectorizer.get_feature_names_out()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<4x9 sparse matrix of type '<class 'numpy.float64'>'\n",
       "\twith 21 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[0.        , 0.46979139, 0.58028582, 0.38408524, 0.        ,\n",
       "         0.        , 0.38408524, 0.        , 0.38408524],\n",
       "        [0.        , 0.6876236 , 0.        , 0.28108867, 0.        ,\n",
       "         0.53864762, 0.28108867, 0.        , 0.28108867],\n",
       "        [0.51184851, 0.        , 0.        , 0.26710379, 0.51184851,\n",
       "         0.        , 0.26710379, 0.51184851, 0.26710379],\n",
       "        [0.        , 0.46979139, 0.58028582, 0.38408524, 0.        ,\n",
       "         0.        , 0.38408524, 0.        , 0.38408524]])"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.todense()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Na podstawie tych danych możemy wytrenowąc model regresji logistycznej. Jest to model regresji liniowej z dodatkową nałożoną funkcją logistyczną:\n",
    "    ( https://en.wikipedia.org/wiki/Logistic_function )\n",
    "    \n",
    "    \n",
    "![Przykład 1](./logistic.png)\n",
    "\n",
    "\n",
    "Dzięki wyjściu modelu zawsze pomiędzy 0, a 1 można traktować wynik jako prawdopodobieństwo\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 1, 0])"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = [0,0,1,1]\n",
    "model = LogisticRegression()\n",
    "model.fit(X, y)\n",
    "model.predict(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.51514316, 0.48485684],\n",
       "       [0.56428483, 0.43571517],\n",
       "       [0.40543928, 0.59456072],\n",
       "       [0.51514316, 0.48485684]])"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict_proba(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sieci neuronowe\n",
    "\n",
    "Warto zauważyć, że sieci neuronowe w najprostszym wariancie to tak naprawdę złożenie wielu funkcji regresji logistycznej ze sobą, gdzie wejściem jednego modelu regresji logistycznej jest wyjście poprzedniej. W przypadku danych tekstowych zazwyczaj jest wybierana wtedy inna reprezentacja danych niż TF IDF, ponieważ TF IDF nie uwzględnia kolejności słów"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standard Scaler\n",
    "\n",
    "**Zadanie 7**\n",
    "\n",
    "\n",
    "Sprawdź dokumentację https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html\n",
    "\n",
    "KNN jest wrażliwy na liniowe skalowanie danych (w przeciwieństwie do modeli bazujących na regresji, gdyż współczynniki liniowe rekompensują skalowanie liniowe).\n",
    "\n",
    "Wytrenuj dowolny model KNN na cechach pozyskanych ze StandardScaler. Pamiętaj, żeby wyskalować zarówno dane ze zbioru test jak i train.\n",
    "\n",
    "\n",
    "\n",
    "Zauważ, że scaler ma podobne API (fit, transform) jak TF IDF Vectorier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}