949 lines
62 KiB
Plaintext
949 lines
62 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Klasyfikacja w Pythonie"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"**zad. 1** Które z poniższych problemów jest problemem regresji, a które klasyfikacji?\n",
|
||
" 1. Sprawdzenie, czy wiadomość jest spamem.\n",
|
||
" 1. Przewidzenie oceny (od 1 do 5 gwiazdek) na podstawie komentarza.\n",
|
||
" 1. OCR cyfr: rozpoznanie cyfry z obrazka.\n",
|
||
" \n",
|
||
" Jeżeli problem jest klasyfikacyjny, to jakie mamy klasy?"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"1. klasyfikacja\n",
|
||
"2. można traktować jako klasyfikację lub regresję. Jeżeli jako regresję to należy sprowadzić liczbę rzeczywistą do jednej z {1,2,3,4,5}\n",
|
||
"3. klasyfikacja"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Miary dla klasyfikacji"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Istnieje wieje miar (metryk), na podstawie których możemy ocenić jakość modelu. Podobnie jak w przypadku regresji liniowej potrzebne są dwie listy: lista poprawnych klas i lista predykcji z modelu. Najpopularniejszą z metryk jest trafność, którą definiuje się w następujący sposób:\n",
|
||
" $$ACC = \\frac{k}{N}$$ \n",
|
||
" \n",
|
||
" gdzie: \n",
|
||
" * $k$ to liczba poprawnie zaklasyfikowanych przypadków,\n",
|
||
" * $N$ liczebność zbioru testującego."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"**zadanie** Napisz funkcję, która jako parametry przyjmnie dwie listy (lista poprawnych klas i wyjście z klasyfikatora) i zwróci trafność."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"ACC: 0.4\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"def accuracy_measure(true, predicted):\n",
|
||
" return sum([1 if t==p else 0 for t,p in zip(true, predicted)]) / len(true)\n",
|
||
"\n",
|
||
"true_label = [1, 1, 1, 0, 0]\n",
|
||
"predicted = [0, 1, 0, 1, 0]\n",
|
||
"print(\"ACC:\", accuracy_measure(true_label, predicted))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Klasyfikator $k$ najbliższych sąsiadów *(ang. k-nearest neighbors, KNN)*"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Klasyfikator [KNN](https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm), który został wprowadzony na ostatnim wykładzie, jest bardzo intuicyjny. Pomysł, który stoi za tym klasyfikatorem jest bardzo prosty: Jeżeli mamy nowy obiekt do zaklasyfikowania, to szukamy wśród danych trenujących $k$ najbardziej podobnych do niego przykładów i na ich podstawie decydujemy (np. biorąc większość) do jakie klasy powinien należeć dany obiekt."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"** Przykład 1** Mamy za zadanie przydzielenie obiektów do dwóch klas: trójkątów lub kwadratów. Rozpatrywany obiekt jest zaznaczony zielonym kółkiem. Przyjmując $k=3$, mamy wśród sąsiadów 2 trójkąty i 1 kwadrat. Stąd obiekt powinienm zostać zaklasyfikowany jako trójkąt. Jak zmienia się sytuacja, gdy przyjmiemy $k=5$?\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"( Grafika pochodzi z https://pl.wikipedia.org/wiki/K_najbli%C5%BCszych_s%C4%85siad%C3%B3w )"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Herbal Iris"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"*Herbal Iris* jest klasycznym zbiorem danych w uczeniu maszynowym, który powstał w 1936 roku. Zawiera on informacje na 150 egzemplarzy roślin, które należą do jednej z 3 odmian."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"**zad. 2** Wczytaj do zmiennej ``data`` zbiór *Herbal Iris*, który znajduje się w pliku ``iris.data``. Jest to plik csv.\n",
|
||
"\n",
|
||
"Kolumny są następujące:\n",
|
||
"\n",
|
||
"1. sepal length in cm\n",
|
||
"2. sepal width in cm\n",
|
||
"3. petal length in cm\n",
|
||
"4. petal width in cm\n",
|
||
"5. class: \n",
|
||
" * Iris Setosa\n",
|
||
" * Iris Versicolour\n",
|
||
" * Iris Virginica"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"5.1,3.5,1.4,0.2,Iris-setosa\r\n",
|
||
"4.9,3.0,1.4,0.2,Iris-setosa\r\n",
|
||
"4.7,3.2,1.3,0.2,Iris-setosa\r\n",
|
||
"4.6,3.1,1.5,0.2,Iris-setosa\r\n",
|
||
"5.0,3.6,1.4,0.2,Iris-setosa\r\n",
|
||
"5.4,3.9,1.7,0.4,Iris-setosa\r\n",
|
||
"4.6,3.4,1.4,0.3,Iris-setosa\r\n",
|
||
"5.0,3.4,1.5,0.2,Iris-setosa\r\n",
|
||
"4.4,2.9,1.4,0.2,Iris-setosa\r\n",
|
||
"4.9,3.1,1.5,0.1,Iris-setosa\r\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"!head iris.data"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import pandas as pd\n",
|
||
"data = pd.read_csv('iris.data', names=('sepal_length', 'sepal_width', 'petal_length', 'petal_width','class'),index_col=False)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<div>\n",
|
||
"<style scoped>\n",
|
||
" .dataframe tbody tr th:only-of-type {\n",
|
||
" vertical-align: middle;\n",
|
||
" }\n",
|
||
"\n",
|
||
" .dataframe tbody tr th {\n",
|
||
" vertical-align: top;\n",
|
||
" }\n",
|
||
"\n",
|
||
" .dataframe thead th {\n",
|
||
" text-align: right;\n",
|
||
" }\n",
|
||
"</style>\n",
|
||
"<table border=\"1\" class=\"dataframe\">\n",
|
||
" <thead>\n",
|
||
" <tr style=\"text-align: right;\">\n",
|
||
" <th></th>\n",
|
||
" <th>sepal_length</th>\n",
|
||
" <th>sepal_width</th>\n",
|
||
" <th>petal_length</th>\n",
|
||
" <th>petal_width</th>\n",
|
||
" <th>class</th>\n",
|
||
" </tr>\n",
|
||
" </thead>\n",
|
||
" <tbody>\n",
|
||
" <tr>\n",
|
||
" <th>0</th>\n",
|
||
" <td>5.1</td>\n",
|
||
" <td>3.5</td>\n",
|
||
" <td>1.4</td>\n",
|
||
" <td>0.2</td>\n",
|
||
" <td>Iris-setosa</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>1</th>\n",
|
||
" <td>4.9</td>\n",
|
||
" <td>3.0</td>\n",
|
||
" <td>1.4</td>\n",
|
||
" <td>0.2</td>\n",
|
||
" <td>Iris-setosa</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>2</th>\n",
|
||
" <td>4.7</td>\n",
|
||
" <td>3.2</td>\n",
|
||
" <td>1.3</td>\n",
|
||
" <td>0.2</td>\n",
|
||
" <td>Iris-setosa</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>3</th>\n",
|
||
" <td>4.6</td>\n",
|
||
" <td>3.1</td>\n",
|
||
" <td>1.5</td>\n",
|
||
" <td>0.2</td>\n",
|
||
" <td>Iris-setosa</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>4</th>\n",
|
||
" <td>5.0</td>\n",
|
||
" <td>3.6</td>\n",
|
||
" <td>1.4</td>\n",
|
||
" <td>0.2</td>\n",
|
||
" <td>Iris-setosa</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>...</th>\n",
|
||
" <td>...</td>\n",
|
||
" <td>...</td>\n",
|
||
" <td>...</td>\n",
|
||
" <td>...</td>\n",
|
||
" <td>...</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>145</th>\n",
|
||
" <td>6.7</td>\n",
|
||
" <td>3.0</td>\n",
|
||
" <td>5.2</td>\n",
|
||
" <td>2.3</td>\n",
|
||
" <td>Iris-virginica</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>146</th>\n",
|
||
" <td>6.3</td>\n",
|
||
" <td>2.5</td>\n",
|
||
" <td>5.0</td>\n",
|
||
" <td>1.9</td>\n",
|
||
" <td>Iris-virginica</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>147</th>\n",
|
||
" <td>6.5</td>\n",
|
||
" <td>3.0</td>\n",
|
||
" <td>5.2</td>\n",
|
||
" <td>2.0</td>\n",
|
||
" <td>Iris-virginica</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>148</th>\n",
|
||
" <td>6.2</td>\n",
|
||
" <td>3.4</td>\n",
|
||
" <td>5.4</td>\n",
|
||
" <td>2.3</td>\n",
|
||
" <td>Iris-virginica</td>\n",
|
||
" </tr>\n",
|
||
" <tr>\n",
|
||
" <th>149</th>\n",
|
||
" <td>5.9</td>\n",
|
||
" <td>3.0</td>\n",
|
||
" <td>5.1</td>\n",
|
||
" <td>1.8</td>\n",
|
||
" <td>Iris-virginica</td>\n",
|
||
" </tr>\n",
|
||
" </tbody>\n",
|
||
"</table>\n",
|
||
"<p>150 rows × 5 columns</p>\n",
|
||
"</div>"
|
||
],
|
||
"text/plain": [
|
||
" sepal_length sepal_width petal_length petal_width class\n",
|
||
"0 5.1 3.5 1.4 0.2 Iris-setosa\n",
|
||
"1 4.9 3.0 1.4 0.2 Iris-setosa\n",
|
||
"2 4.7 3.2 1.3 0.2 Iris-setosa\n",
|
||
"3 4.6 3.1 1.5 0.2 Iris-setosa\n",
|
||
"4 5.0 3.6 1.4 0.2 Iris-setosa\n",
|
||
".. ... ... ... ... ...\n",
|
||
"145 6.7 3.0 5.2 2.3 Iris-virginica\n",
|
||
"146 6.3 2.5 5.0 1.9 Iris-virginica\n",
|
||
"147 6.5 3.0 5.2 2.0 Iris-virginica\n",
|
||
"148 6.2 3.4 5.4 2.3 Iris-virginica\n",
|
||
"149 5.9 3.0 5.1 1.8 Iris-virginica\n",
|
||
"\n",
|
||
"[150 rows x 5 columns]"
|
||
]
|
||
},
|
||
"execution_count": 4,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"data"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"**zad. 3** Odpowiedz na poniższe pytania:\n",
|
||
" 1. Które atrybuty są wejściowe, a w której kolumnie znajduje się klasa wyjściowa?\n",
|
||
" 1. Ile jest różnych klas? Wypisz je ekran.\n",
|
||
" 1. Jaka jest średnia wartość w kolumnie ``sepal_length``? Jak zachowuje się średnia, jeżeli policzymy ją dla każdej z klas osobno?"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"1. wejściowe są sepal_length, sepal_width, petal_length, petal_width. Klasa wyjściowa to class"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"array(['Iris-setosa', 'Iris-versicolor', 'Iris-virginica'], dtype=object)"
|
||
]
|
||
},
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"data['class'].unique()\n",
|
||
"# 3 klasy"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"5.843333333333334"
|
||
]
|
||
},
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"data['sepal_length'].mean()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"class\n",
|
||
"Iris-setosa 5.006\n",
|
||
"Iris-versicolor 5.936\n",
|
||
"Iris-virginica 6.588\n",
|
||
"Name: sepal_length, dtype: float64"
|
||
]
|
||
},
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"data.groupby('class')['sepal_length'].mean()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Wytrenujmy klasyfikator *KNN*, ale najpierw przygotujmy dane. Fukcja ``train_test_split`` dzieli zadany zbiór danych na dwie części. My wykorzystamy ją do podziału na zbiór treningowy (66%) i testowy (33%), służy do tego parametr ``test_size``."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from sklearn.model_selection import train_test_split\n",
|
||
"\n",
|
||
"X = data.loc[:, 'sepal_length':'petal_width']\n",
|
||
"Y = data['class']\n",
|
||
"\n",
|
||
"(train_X, test_X, train_Y, test_Y) = train_test_split(X, Y, test_size=0.33, random_state=42)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Trenowanie klasyfikatora wygląda bardzo podobnie do treningi modelu regresji liniowej:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<style>#sk-container-id-1 {color: black;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>KNeighborsClassifier(n_neighbors=3)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">KNeighborsClassifier</label><div class=\"sk-toggleable__content\"><pre>KNeighborsClassifier(n_neighbors=3)</pre></div></div></div></div></div>"
|
||
],
|
||
"text/plain": [
|
||
"KNeighborsClassifier(n_neighbors=3)"
|
||
]
|
||
},
|
||
"execution_count": 9,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"from sklearn.neighbors import KNeighborsClassifier\n",
|
||
"\n",
|
||
"model = KNeighborsClassifier(n_neighbors=3)\n",
|
||
"model.fit(train_X, train_Y)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Mając wytrenowany model możemy wykorzystać go do predykcji na zbiorze testowym."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
|
||
"Zaklasyfikowane: Iris-setosa, Orginalne: Iris-setosa\n",
|
||
"Zaklasyfikowane: Iris-virginica, Orginalne: Iris-virginica\n",
|
||
"Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
|
||
"Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
|
||
"Zaklasyfikowane: Iris-setosa, Orginalne: Iris-setosa\n",
|
||
"Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
|
||
"Zaklasyfikowane: Iris-virginica, Orginalne: Iris-virginica\n",
|
||
"Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n",
|
||
"Zaklasyfikowane: Iris-versicolor, Orginalne: Iris-versicolor\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"predicted = model.predict(test_X)\n",
|
||
"\n",
|
||
"for i in range(10):\n",
|
||
" print(\"Zaklasyfikowane: {}, Orginalne: {}\".format(predicted[i], test_Y.reset_index()['class'][i]))\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Możemy obliczyć *accuracy*:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"0.98\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from sklearn.metrics import accuracy_score\n",
|
||
"\n",
|
||
"print(accuracy_score(test_Y, predicted))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"**zad. 4** Wytrenuj nowy model ``model_2`` zmieniając liczbę sąsiadów na 20. Czy zmieniły się wyniki?"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 44,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"0.98\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from sklearn.neighbors import KNeighborsClassifier\n",
|
||
"\n",
|
||
"model = KNeighborsClassifier(n_neighbors=10)\n",
|
||
"model.fit(train_X, train_Y)\n",
|
||
"predicted = model.predict(test_X)\n",
|
||
"\n",
|
||
"print(accuracy_score(test_Y, predicted))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"**zad. 5** Wytrenuj model z $k=1$. Przeprowadź walidację na zbiorze trenującym zamiast na zbiorze testowym? Jakie wyniki otrzymałeś? Czy jest to wyjątek? Dlaczego tak się dzieje?"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 45,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"1.0\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from sklearn.neighbors import KNeighborsClassifier\n",
|
||
"\n",
|
||
"model = KNeighborsClassifier(n_neighbors=1)\n",
|
||
"model.fit(train_X, train_Y)\n",
|
||
"predicted = model.predict(train_X)\n",
|
||
"\n",
|
||
"print(accuracy_score(train_Y, predicted))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Walidacja krzyżowa"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Zbiór *herbal Iris* jest bardzo małym zbiorem. Wydzielenie z niego zbioru testowego jest obciążone dużą wariancją wyników, tj. w zależności od sposoby wyboru zbioru testowego wyniki mogą się bardzo różnic. Żeby temu zaradzić, stosuje się algorytm [walidacji krzyżowej](https://en.wikipedia.org/wiki/Cross-validation_(statistics). Algorytm wygląda następująco:\n",
|
||
" 1. Podziel zbiór danych na $n$ części (losowo).\n",
|
||
" 1. Dla każdego i od 1 do $n$ wykonaj:\n",
|
||
" 1. Weź $i$-tą część jako zbiór testowy, pozostałe dane jako zbiór trenujący.\n",
|
||
" 1. Wytrenuj model na zbiorze trenującym.\n",
|
||
" 1. Uruchom model na danych testowych i zapisz wyniki.\n",
|
||
" 1. Ostateczne wyniki to średnia z $n$ wyników częściowych. \n",
|
||
" \n",
|
||
" W Pythonie służy do tego funkcja ``cross_val_score``, która przyjmuje jako parametry (kolejno) model, zbiór X, zbiór Y. Możemy ustawić parametr ``cv``, który określa na ile części mamy podzielić zbiór danych oraz parametr ``scoring`` określający miarę.\n",
|
||
" \n",
|
||
" W poniższym przykładzie dzielimy zbiór danych na 10 części (10-krotna walidacja krzyżowa) i jako miarę ustawiany celność (ang. accuracy)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 46,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Wynik walidacji krzyżowej: 0.9666666666666668\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from sklearn.model_selection import cross_val_score\n",
|
||
"\n",
|
||
"k=10\n",
|
||
"knn = KNeighborsClassifier(n_neighbors=k)\n",
|
||
"scores = cross_val_score(knn, X, Y, cv=10, scoring='accuracy')\n",
|
||
"print(\"Wynik walidacji krzyżowej:\", scores.mean())"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"**zad. 6** Klasyfikator $k$ najbliższych sąsiadów posiada jeden parametr: $k$, który określa liczbę sąsiadów podczas klasyfikacji. Jak widzieliśmy, wybór $k$ może mieć duże znaczenie dla jakości klasyfikatora. Wykonaj:\n",
|
||
" 1. Stwórz listę ``neighbors`` wszystkich liczb nieparzystych od 1 do 50.\n",
|
||
" 1. Dla każdego elementu ``i`` z listy ``neighbors`` zbuduj klasyfikator *KNN* o liczbie sąsiadów równej ``i``. Nastepnie przeprowadz walidację krzyżową (parametry takie same jak powyżej) i zapisz wyniki do tablicy ``cv_scores``.\n",
|
||
" 1. Znajdź ``k``, dla którego klasyfikator osiąga najwyższy wynik."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 66,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"neighbors = list(range(1,50,2))\n",
|
||
"cv_scores = list()\n",
|
||
"max_score = -1\n",
|
||
"for neighbor_num in neighbors:\n",
|
||
" knn = KNeighborsClassifier(n_neighbors=neighbor_num)\n",
|
||
" score = cross_val_score(knn, X, Y, cv=10, scoring='accuracy').mean()\n",
|
||
" max_score = score if score > max_score else max_score\n",
|
||
" neighbor_num_best = neighbor_num if score == max_score else neighbor_num_best\n",
|
||
" cv_scores.append(score)\n",
|
||
" "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 67,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"13\n",
|
||
"0.9800000000000001\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(neighbor_num_best)\n",
|
||
"print(max_score)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Wykres przedstawiający precent błedów w zależnosci od liczby sąsiadów."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 68,
|
||
"metadata": {
|
||
"scrolled": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAGxCAYAAACKvAkXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABbOklEQVR4nO3dd3hUZdo/8O/MZErqhJBe6CWVBFIwoIKaFQQpghhYVxTRXV0QNKuv4C7ivv4UfXdRcGV12XfV1101iAgqKsVQVFpIgySEFkp6hfQ+c35/JDMkJoFkMpMz5fu5rrkuOHPmzD0nwNw8z/3cj0QQBAFERERENkQqdgBEREREg40JEBEREdkcJkBERERkc5gAERERkc1hAkREREQ2hwkQERER2RwmQERERGRzmAARERGRzbETOwBzpdVqUVRUBGdnZ0gkErHDISIioj4QBAG1tbXw9fWFVNr7OA8ToF4UFRUhICBA7DCIiIjIAPn5+fD39+/1eSZAvXB2dgbQfgNdXFxEjoaIiIj6oqamBgEBAfrv8d4wAeqFbtrLxcWFCRAREZGFuVX5CougiYiIyOYwASIiIiKbwwSIiIiIbA4TICIiIrI5TICIiIjI5jABIiIiIpvDBIiIiIhsDhMgIiIisjlMgIiIiMjmMAEiIiIim8MEiIiIiGwOEyAiIiKyOdwMlYiIyEpdLKvDB0cuo6VNa5Tr+bnaY9U9YyGT3nyjUUvABIiIiMhKvb3/PL7NLDbqNUd7OmFuuK9RrykGJkBERERW6lRBFQDgkduGw9fVfkDXSsu7jv1nSrHtZB4TICIiIjJPVQ0tKLjeCAB4/t7xUDvIB3S9/GsN+CGnFEcuViKvsgHDhjoYI0zRsAiaiIjICmUX1QAAAtzsB5z8tF/HAbePcQcAfJ6SP+DriY0JEBERkRXKKqwGAIT5qY12zcXRwwAA21Pz0aYxTmG1WJgAERERWaGsjhGgEF/jJUC/CvaCm6MCpTXNOHy+3GjXFQMTICIiIiuU3TECFGrEESCFnRQLJ/kBAD5LtuxpMCZAREREVqa2qRWXKuoBACG+Lka9dnx0AADg4LkylNY0GfXag4kJEBERkZXJKa4FAPioVXB3Uhr12mM8nRE1fAg0WgFfpBYY9dqDiQkQERGRldEVQBuz/qcz3SjQ5yn50GoFk7yHqTEBIiIisjJZRbr6H+NOf+nMnuADZ6UdrlY24PjlSpO8h6kxASIiIrIyuhGgUBONADko7DA3or0bdKKFFkMzASIiIrIijS0aXCyrA2DcFWC/pOsJtCerBNfrW0z2PqbCBIiIiMiK5JTUQCsA7k4KeLkYtwC6s1A/FwT7uKBFo8WujEKTvY+pMAEiIiKyIp37/0gkEpO9j0QiweKY9mLoxOR8CIJlFUMzASIiIrIiWYXtHaBNVf/T2bwIPyjtpDhXWouM/CqTv58xMQEiIiKyIqZeAdaZ2l6O2WE+AIBtJy2rGJoJEBERkZVobtPgfGl7E0RT9QD6JV1PoK9PFaGuuW1Q3tMYmAARERFZiQuldWjVCFDby+E/xH5Q3jNmpBtGujuioUWDb08XDcp7GgMTICIiIiuh7//j52LSAujOJBKJfhQo0YKmwZgAERERWQl9/c8gTX/pLJzkDzupBOl5VThXUjuo720oJkBERERWIrNjBViICRsg9sTDWYm4IC8AQOLJvEF9b0MxASIiIrICrRotcop1S+BNvwLsl+I7egLtTC9EU6tm0N+/v5gAERERWYHc8jq0tGnhpLTDiKGOg/7+d471gI9ahaqGVuw7Uzro799fTICIiIisgK4BYrCvC6TSwSmA7kwmlWBRVPso0DYLmAZjAkRERGQFTL0DfF88FOUPiQQ4crESVyvrRYujL5gAERERWYHsQewA3Rv/IQ64Y6wHAODzFPNeEs8EiIiIyMJptQKyizoKoAd5BdgvLe7oCbQ9pQBtGq2osdwMEyAiIiILd7myHg0tGqjkUoxyH/wC6M7igrzg5qhAWW0zDp0rFzWWm2ECREREZOF09T9BPi6wk4n71a6wk2LhJD8A5t0ZmgkQERGRhdNPf4lYAN2ZbmuMg+fKUFrTJHI0PWMCREREZOEyC8QvgO5sjKczokcMgUYr4IvUArHD6RETICIiIgsmCMKNPcBELoDuLD56GABg28l8aLWCyNF0xwSIiIjIguVfa0RtUxsUMinGejqLHY7erDBvOCvtkHetAccvVYodTjdMgIiIiCyYbvRnvLczFHbm87XuoLDD3AhfAOZZDG0+d4qIiIj6Td8B2kzqfzpb3DENtierBNfrW0SOpismQERERBYsq2MFWIiZrADrLMxfjRBfF7RotNiZXih2OF0wASIiIrJQgiAgu9D8CqA703WG3nYyH4JgPsXQTICIiIgsVElNEyrrWyCTShDobT4F0J3NjfCD0k6Kc6W1yMivEjscPSZAREREFiqrsH36a6ynE1RymcjR9ExtL8fsMB8A7aNA5oIJEBERkYXSFUCbY/1PZ7rO0F+fKkJdc5vI0bRjAkRERGShzHkFWGcxI90wyt0RDS0a7D5VJHY4AJgAERERWSxdD6AwMy2A1pFIJPpRIHPpCcQEiIiIyAKV1TahtKYZEkn7LvDmbsEkf9hJJcjIr8LZkhqxwzGPBGjLli0YMWIEVCoVJk+ejOTk5Juev337dgQGBkKlUiEsLAzfffddt3NycnIwd+5cqNVqODo6Ijo6Gnl5eab6CERERINKtwP8KHdHOCrtRI7m1jyclYgL8gJgHsXQoidA27ZtQ0JCAtavX4+0tDSEh4djxowZKCsr6/H8o0ePYsmSJVi+fDnS09Mxf/58zJ8/H1lZWfpzcnNzcfvttyMwMBCHDh3C6dOnsW7dOqhUqsH6WERERCZl7v1/ehIf0z4NtjO9EE2tGlFjkQgidyWaPHkyoqOj8e677wIAtFotAgIC8Mwzz2DNmjXdzo+Pj0d9fT12796tP3bbbbchIiIC77//PgBg8eLFkMvl+Pe//21wXDU1NVCr1aiuroaLi/kPLRIRkW156t+p2JNdgj/OCsKTd44SO5w+0WgF3PHmARRVN2Hz4gjMi/Az+nv09ftb1BGglpYWpKamIi4uTn9MKpUiLi4Ox44d6/E1x44d63I+AMyYMUN/vlarxbfffotx48ZhxowZ8PT0xOTJk7Fr166bxtLc3IyampouDyIiInOlK4AOMfMVYJ3JpBIsirrRGVpMoiZAFRUV0Gg08PLy6nLcy8sLJSUlPb6mpKTkpueXlZWhrq4Ob7zxBmbOnIl9+/bhgQcewIIFC3D48OFeY9mwYQPUarX+ERAQMMBPR0REZBpVDS0ouN4IwPx7AP3Soih/SCTA0dxKXK2sFy0O86+a6ietVgsAmDdvHp577jkAQEREBI4ePYr3338f06ZN6/F1a9euRUJCgv73NTU1TIKIiMgs6Qqgh7k5QG0vFzma/vEf4oAV08cg0McZ3mrxanNFTYDc3d0hk8lQWlra5XhpaSm8vb17fI23t/dNz3d3d4ednR2Cg4O7nBMUFISff/6511iUSiWUSqUhH4OIiGhQWUoDxN48P2O82CGIOwWmUCgQGRmJpKQk/TGtVoukpCTExsb2+JrY2Ngu5wPA/v379ecrFApER0fj3LlzXc45f/48hg8fbuRPQERENPgyLXAFmLkRfQosISEBjz76KKKiohATE4NNmzahvr4ey5YtAwAsXboUfn5+2LBhAwBg9erVmDZtGjZu3IjZs2cjMTERKSkp2Lp1q/6aL7zwAuLj43HnnXfirrvuwp49e/DNN9/g0KFDYnxEIiIio9JNgYVaWP2PORE9AYqPj0d5eTlefvlllJSUICIiAnv27NEXOufl5UEqvTFQNWXKFHz66af405/+hJdeegljx47Frl27EBoaqj/ngQcewPvvv48NGzZg1apVGD9+PHbs2IHbb7990D8fERGRMdU2teJyRXvxcIivZU6BmQPR+wCZK/YBIiIic3TiUiXitx6Hr1qFo2vvETscs2MRfYCIiIiof7I6pr9CWP8zIEyAiIiILIh+CwzW/wwIEyAiIiILousAbalL4M0FEyAiIiIL0diiwcWyOgBcAj9QTICIiIgsRE5JDbQC4O6khKczm/cOBBMgIiIiC6HrAB3m5wKJRCJyNJaNCRAREZGFyGIHaKNhAkRERGQhsgo7lsBzBdiAMQEiIiKyAM1tGpwvrQXAFWDGwASIiIjIApwvqUObVoCrgxx+rvZih2PxmAARERFZAH3/H181C6CNgAkQERGRBdAVQIdw+ssomAARERFZAN0eYNwCwziYABEREZm5Vo0WOcUdCRCXwBsFEyAiIiIzl1teh5Y2LZyUdhju5iB2OFaBCRAREZGZyyzoqP/xdYFUygJoY2ACREREZOayizj9ZWxMgIiIiMzcjS0wuALMWJgAERERmTGNVsCZYq4AMzYmQERERGbsckU9Glo0UMmlGOXhJHY4VoMJEBERkRnL7ugAHezjAhkLoI2GCRAREZEZu1H/w+kvY2ICREREZMayCln/YwpMgIiIiMyUIAj6TVC5B5hxMQEiIiIyU/nXGlHb1AaFTIpxXs5ih2NVmAARERGZqcyO+p9AH2fIZfzKNibeTSIiIjOln/5i/Y/RMQEiIiIyU+wAbTpMgIiIiMyQIAg39gDjCJDRMQEiIiIyQ8XVTbhW3wKZVILx3iyANjYmQERERGZIN/011tMJKrlM5GisDxMgIiIiM5Slm/5iB2iTYAJERERkhrJ1BdC+LIA2BSZAREREZki3BD7MnyNApsAEiIiIyMyU1TahtKYZEgkQ5MMRIFNgAkRERGRmsjs2QB3t4QQHhZ3I0VgnJkBERERmJov1PybHBIiIiMjM6Op/uALMdJgAERERmZmsjikw7gFmOkyAiIiIzMj1+hYUVjUCAII5BWYyrKwiIrIigiBg64+XcKGsTuxQTMrTWYln48ZBYWd9/4/X7f81fKgD1PZykaOxXkyAiIisyLHcSmz4/qzYYQwKX1d7/Oa24WKHYXQ/XSwHAISx/sekmAAREVmRz07mAwBuH+OOqWPcRY7GNLKLqrH7dDG2ncy3ugSoVaPFjtQCAMD9E3xFjsa6MQEiIrIS1+tbsDerBACw5r5Aq11BVFnXjL3ZJcgsrEZWYbVVfc6knDJU1LXA3UmJe4I8xQ7Hqlnf5CkRkY3amV6IFo0WIb4uVpUU/NJQJyXuDfYGAHyeki9yNMa17WQeAODBSH/IZfyKNiXeXSIiKyAIArZ1TH8tjg4QORrTWxzT/hl3pheiqVUjcjTGUVTViMPn2+t/4m3gZyg2JkBERFYgI78K50proZJLMTfCT+xwTG7qaHf4udqjtqkN32cVix2OUWxPKYBWACaPdMNId0exw7F6TICIiKxAYnL76M+sMB+bWDotlUr0oySfJVv+NJhGK+in85bEDBM5GtvABIiIyMLVNbfhm9NFAIDF0bbz5flgpD+kEiD58jVcKrfsvkdHLlagsKoRLio7zAz1Fjscm8AEiIjIwu0+VYSGFg1GuTsiesQQscMZNL6u9pg2zgMAsM3Ci6F19VsPTPSDSi4TORrbwASIiMjCJXZ8ecZHB0AikYgczeBa3DFdtCO1AK0arcjRGKayrhn7zrS3L4i3oRE8sTEBIiKyYGdLapCRXwU7qQQLI/3FDmfQ3R3oCXcnJSrqWpCUUyZ2OAb5Mq0QrRoBE/zV3PtrEDEBIiKyYLri518Fe8HdSSlyNINPLpPiwY7EL7Gjh44lEQRBH7ct1W+ZAyZAREQWqqlVg53phQBsu2+M7rMfPl+Ooo5d1C1F6tXryC2vh71chjnhPmKHY1OYABERWai92SWobmyFr1qFO8Z6iB2OaEa6O+K2UW4QhPZeOpZEV791/wQfOKusv32BOWECRERkoXQrhxZFBUAmta3i51/STR99npIPjVYQOZq+qWlqxben25s46jpb0+BhAkREZIGuVtbjaG4lJBLgIRue/tKZGeoNF5UdCqsaceRihdjh9MnXGUVobNVgjKcTJg2znfYF5oIJEBGRBdKN/tw51gN+rvYiRyM+lVyGBya2bwFiKcXQnfdus7X2BeaACRARkYVp02ixPbW91sUWNj7tK10Pnf1nSlFR1yxyNDeXVViNzMJqyGUSLJhke+0LzAETICIiC3PwXDnKa5sx1FGBe4K8xA7HbAT7uiDcX41WjYCdaYVih3NTun2/7g3xhpujQuRobBMTICIiC7OtY4pnYaQ/FHb8Z7wz3ShQ4sk8CIJ5FkN3bl/AETzxGPQ358CBA2hqajJaEFu2bMGIESOgUqkwefJkJCcn3/T87du3IzAwECqVCmFhYfjuu++6PP/YY49BIpF0ecycOdNo8RIRiaWkugkHzrZ3PLbl3j+9mRPuA3u5DLnl9Ui9el3scHr0XWYxapva4D/EHlNHu4sdjs0yKAGaO3cuXF1dcccdd2DdunX44Ycf0NhoWPOpbdu2ISEhAevXr0daWhrCw8MxY8YMlJX13NL86NGjWLJkCZYvX4709HTMnz8f8+fPR1ZWVpfzZs6cieLiYv3js88+Myg+IiJz8kVqPrQCEDPCDaM9nMQOx+w4q+S4f0J7Q8HPks1zg1T93m1RAZDaePsCMRmUAF2/fh1JSUm47777kJycjAceeACurq6YOnUq/vSnP/XrWm+99RaefPJJLFu2DMHBwXj//ffh4OCADz74oMfzN2/ejJkzZ+KFF15AUFAQXn31VUyaNAnvvvtul/OUSiW8vb31jyFDuMSQiCybVivodz3n6E/vdBukfptZhJqmVpGj6epSeR2SL1+DVAI8GMXiZzEZlADJ5XJMnToVL730Evbu3Yvjx49jyZIlSE5OxoYNG/p8nZaWFqSmpiIuLu5GQFIp4uLicOzYsR5fc+zYsS7nA8CMGTO6nX/o0CF4enpi/PjxePrpp1FZWXnTWJqbm1FTU9PlQURkTo5dqkT+tUY4K+0wK4zbJvRm0jBXjPV0QlOrFl9nFIkdThe6BHb6eE/4qNm+QEwGJUDnz5/H1q1b8etf/xp+fn6YNm0aqqur8de//hVpaWl9vk5FRQU0Gg28vLquYvDy8kJJSUmPrykpKbnl+TNnzsTHH3+MpKQkvPnmmzh8+DDuu+8+aDSaXmPZsGED1Gq1/hEQwP9dEZF50U2dzJvoC3uFTORozJdEItGPkOl67ZiDVo0WOzraF3AET3x2hrwoMDAQHh4eWL16NdasWYOwsDCzauK0ePFi/a/DwsIwYcIEjB49GocOHcI999zT42vWrl2LhIQE/e9ramqYBBGR2bhW34K9We3/0eOu4be2YJI/3txzFpmF1cgqrEaon1rskJCUU4qKuha4Oylxd6Cn2OHYPINGgFatWgU/Pz/893//N5566in88Y9/xL59+9DQ0NCv67i7u0Mmk6G0tLTL8dLSUnh7e/f4Gm9v736dDwCjRo2Cu7s7Ll682Os5SqUSLi4uXR5EROZiZ3ohWjRahPq5mMWXublzc1Tg3pD27wVzGQVK1O/d5g+5jO0LxGbQT2DTpk1IS0tDSUkJ1q5di5aWFvzxj3+Eu7s7pk6d2ufrKBQKREZGIikpSX9Mq9UiKSkJsbGxPb4mNja2y/kAsH///l7PB4CCggJUVlbCx4dz5kRkeQRB0Pf+iefoT58t6bhXuzIK0djSewnEYCiqasTh8+UAgIeiOLtgDgaUgmo0GrS2tqK5uRlNTU1obm7GuXPn+nWNhIQE/POf/8T//d//IScnB08//TTq6+uxbNkyAMDSpUuxdu1a/fmrV6/Gnj17sHHjRpw9exavvPIKUlJSsHLlSgBAXV0dXnjhBRw/fhxXrlxBUlIS5s2bhzFjxmDGjBkD+bhERKJIz6/C+dI6qORSzA33FTscizFl9FD4D7FHbVMbvs8qFjWW7SkFEATgtlFuGOnuKGos1M7gKbAJEybAy8sLv/vd71BUVIQnn3wS6enpKC8v79e14uPj8de//hUvv/wyIiIikJGRgT179ugLnfPy8lBcfOMP7pQpU/Dpp59i69atCA8PxxdffIFdu3YhNDQUACCTyXD69GnMnTsX48aNw/LlyxEZGYmffvoJSqXSkI9LRCSqbR39bGaF+UBtLxc5GsshlUoQ3zHakijiNJhGK+i3vmD9lvmQCAb0Cl+0aBGmTZuG6dOn6xMPa1NTUwO1Wo3q6mrWAxGRaOqa2xDz2g9oaNHg89/FImakm9ghWZTi6kZMfeMAtAJw4A/TMEqE5pE/ni/H0g+SobaX48RL90Al5wo+U+rr97dBq8C2b99ucGBERNR335wqQkOLBqM8HBE9gg1d+8tHbY+7xnsi6WwZtp3Mx9pZQYMeQ2JH/dYDE/2Y/JgRg2uAcnNz8cwzzyAuLg5xcXFYtWoVcnNzjRkbEZHN003dLI4OMKt2I5ZE13NnR1oBWtq0g/reFXXN2H+mtEscZB76lAClpaV1aSK4d+9eBAcHIzk5GRMmTMCECRNw4sQJhISEYP/+/SYLlojIluQU1+BUfhXspBIsmMRtEwx1V6AnPJyVqKhrwYGzpbd+gRHtTCtEq0ZAuL8aQT4spzAnfUqADh8+jNmzZ6O+vh4AsGbNGjz33HM4ceIE3nrrLbz11ls4ceIEnn32Wbz44osmDZiIyFbo+tf8KtgL7k5cxGEouUyKByPbE8jBLIYWBEE//cX2BeanTwnQc889hzvvvBPTpk0DAOTk5GD58uXdznv88cdx5swZ40ZIRGSDmlo12JleCIBTJ8agWw12+Hw5iqoaB+U9U69eR255PRwUMsyNYPsCc9PnGqCXXnoJb7/9NgDAw8MDGRkZ3c7JyMiApyfbexMRDdTe7BJUN7bCz9Ued4z1EDscizfC3RGxo4ZCEKBfkm5qn3W0L7h/gg+clAatOSIT6tdP5I477gAAPPnkk/jtb3+LS5cuYcqUKQCAI0eO4M033+yynxYRERkmMfnGtgkyKYufjWFxTACOXarE9pQCPHP3WJPe15qmVnyb2b4TPae/zJNBKem6devg7OyMjRs36rs0+/r64pVXXsGqVauMGiARka25UlGPY5cqIZEAi7htgtHMCPGG2l6OwqpG/HyxAtPGmW5k7euMIjS1ajHW0wmThrma7H3IcAYtg5dIJHjuuedQUFCA6upqVFdXo6CgAKtXr+YyTSKiAdrWMUVz51gP+LnaixyN9VDJZXhgoh8A6PdWMxVdAXs82xeYrQFvR+vs7AxnZ2djxEJEZPNaNVp8kVoAoL33DxmXrqB8/5lSVNQ1m+Q9sgqrkVlYDYVMyvYFZqzPU2ATJ07scxablpZmcEBERLbs4NkylNc2w91JgXuCvMQOx+oE+bggPMAVp/Kr8GVaAX5752ijv4du9OfeEC+4OSqMfn0yjj6PAM2fPx/z5s3DvHnzMGPGDOTm5kKpVGL69OmYPn06VCoVcnNzueM6EdEA6L48F07yh8JuwIP01APdyFriyXwYsB3mTTW2aLAro7DjfVj8bM76PAK0fv16/a+feOIJrFq1Cq+++mq3c/Lzxdtxl4jIkpVUN+HguTIAwEOc/jKZOeG+eHX3GVwqr0fK1euIHmG8DWa/zypGbVMb/IfYY8rooUa7LhmfQf+92L59O5YuXdrt+G9+8xvs2LFjwEEREdmi7Sn50ApAzAg3jBZh13Jb4aS0w/0TfADcaDdgLLpO0/FRAZCyfYFZMygBsre3x5EjR7odP3LkCFQq1YCDIiKyNVqtoF/9xc7Pprc4pn166tvMItQ0tRrlmpfK65B8+RqkbF9gEQzqA/Tss8/i6aefRlpaGmJiYgAAJ06cwAcffIB169YZNUAiIltwNLcSBdcb4ayyw6wwH7HDsXoTA1wxzssJ50vr8FVGER65bfiAr6mr37prvCe81RwMMHcGJUBr1qzBqFGjsHnzZvznP/8BAAQFBeHDDz/EQw89ZNQAiYhsgW7TzPkRfrBXyESOxvpJJBLERw/Dq7vP4P1DuTiVXzXga+4/077TPEfwLINEMHYJvJWoqamBWq1GdXU1XFxcxA6HiKxYm0aL0Ff2oqlVi69XTsUEf1exQ7IJ1+tbcNuGJDS3aY12TS8XJX5+8W7IZVzBJ5a+fn8PaHe2AwcO4NSpUyguLsaaNWtQXV0NhUIBP7/2Tpv19fVwdHQcyFsQEVm93PJ6NLVq4aiQIdRXLXY4NmOIowKfPDEZJ69cN8r1JBJg2jgPJj8WwqAEKC8vDw888ABycnIwbtw4ZGdn46mnnsKxY8ewZ88e/Pvf/4YgCAgJCcGVK1eMHDIRkXXJKqwGAIT4qrlyaJBFjXBDlBGXwZPlMCgBeuqpp+Dt7Y29e/fC3d1dvxVGREQEHnnkEVy7dg2jR4+GRqMxarBERNYoq6gjAfLjdDvRYDEoATp06BBOnjwJd3f3LsfVajXkcjn+8pe/YM+ePdi1a5cxYiQismrZhTUAwOkvokFkUALk5OSEioqKbsdLSkrg4eGB4OBgBAcHDzg4IiJrp9UKyO4YAQr1YwJENFgMqtSaO3cufv/73+PUqVMAoN8k9e9//zsWLlxovOiIiKzclcp61LdooLSTYrQHF40QDRaDEqCNGzfC19cXkyZNgo+PDxoaGnDbbbchNzcXr7/+urFjJCKyWllF7dNfQT4usOPqIaJBY9AUmFqtxv79+/Hzzz/j9OnTqKurw6RJkxAXF2fs+IiIrFp2oW76iwXQRIPJ4GXwn3zyCdauXYvbb7/d2DEREdkM3QqwMNb/EA2qPidA77zzjv7XbW1t2LBhAxoaGuDh4dHt3FWrVhknOiIiKyYIArI6VoCFcAUY0aDq81YYI0eO7HaspKQELi4ucHBwuHFBiQSXLl0yXoQi4VYYRGRq+dcacMf/HIRcJkH2n2dCYccaIKKBMvpWGJcvX+527JNPPsE333yDxMREw6IkIrJhug7Q472dmfwQDbIB7QX28MMPY8GCBcaKhYjIpujqf9gAkWjwGfxfjqSkJNx///0IDQ3F6NGjcf/99+OHH34wZmxERFZNX//DAmiiQWdQAvT3v/8dM2fOhLOzM1avXo3Vq1fDxcUFs2bNwpYtW4wdIxGR1WkvgNaNALHOkGiwGTQF9vrrr+Ptt9/GypUr9cdWrVqFqVOn4vXXX8eKFSuMFiARkTUqrWlGZX0LZFIJgnyYABENNoNGgKqqqjBz5sxux++9915UV1cPOCgiImunG/0Z4+EElVwmcjREtsfgvcB27tzZ7fhXX32F+++/f8BBERFZuyxugEokKoMaIQYHB+O1117DoUOHEBsbCwA4fvw4jhw5gj/84Q/Gj5KIyMroCqC5BQaROAbUCLHHC7IRIhHRLd32ehJKapqw/alYRI9wEzscIqsxKI0QiYio/8prm1FS0wSJBCyAJhIJW48SEQ2y7I76n5HujnBSDqgfLREZiAkQEdEgyy7qqP9hB2gi0TABIiIaZPoGiCyAJhINEyAiokHGPcCIxGdQApSXl4eeFo8JgoC8vLwBB0VEZK2qG1qRf60RABDCBIhINAYlQCNHjkR5eXm349euXevzcnkiIlukK4AOcLOH2kEucjREtsugBEgQBEgkkm7H6+rqoFKpBhwUEZG10k1/hbEDNJGo+rX+MiEhAUB7s8N169bBwcFB/5xGo8GJEycQERFh1ACJiKxJZkcHaE5/EYmrXwlQeno6gPYRoMzMTCgUCv1zCoUC4eHheP75540bIRGRFcku5B5gROagXwnQwYMHAQDLli3D5s2buUUEEVE/1Da14lJFPQAgxJf/fhKJyaAWpB9++KGx4yAisno5xbUAAB+1Cu5OSpGjIbJtBiVA9fX1eOONN5CUlISysjJotdouz1vDZqhERMama4DI+h8i8RmUAD3xxBM4fPgwHnnkEfj4+PS4IoyIiLrSN0BkB2gi0RmUAH3//ff49ttvMXXqVGPHQ0RktbILuQcYkbkwqA/QkCFD4ObmZuxYiIisVmOLBhfK2muAuAKMSHwGJUCvvvoqXn75ZTQ0NBg7HiIiq3S2pAZaAXB3UsDLhQXQRGIzaAps48aNyM3NhZeXF0aMGAG5vGs797S0NKMER0RkLbI69f9h3SSR+AxKgObPn2/kMIiIrFsW63+IzIpBCdD69euNHQcRkVXjCjAi82JQDRAAVFVV4X//93+xdu1aXLt2DUD71FdhYaHRgiMisgbNbRqcL20vgGYPICLzYNAI0OnTpxEXFwe1Wo0rV67gySefhJubG7788kvk5eXh448/NnacREQW60JpHVo1AtT2cvgPsRc7HCKCgSNACQkJeOyxx3DhwgWoVCr98VmzZuHHH380WnBERNbgRgG0CwugicyEQQnQyZMn8bvf/a7bcT8/P5SUlPT7elu2bMGIESOgUqkwefJkJCcn3/T87du3IzAwECqVCmFhYfjuu+96Pfepp56CRCLBpk2b+h0XEZEx6Ot/OP1FZDYMSoCUSiVqamq6HT9//jw8PDz6da1t27YhISEB69evR1paGsLDwzFjxgyUlZX1eP7Ro0exZMkSLF++HOnp6Zg/fz7mz5+PrKysbufu3LkTx48fh6+vb79iIiIyJt0KsBA2QCQyGwYlQHPnzsV///d/o7W1FQAgkUiQl5eHF198EQsXLuzXtd566y08+eSTWLZsGYKDg/H+++/DwcEBH3zwQY/nb968GTNnzsQLL7yAoKAgvPrqq5g0aRLefffdLucVFhbimWeewSeffNKtTxER0WBp02iRU6xbAs8VYETmwqAEaOPGjairq4OnpycaGxsxbdo0jBkzBs7Oznjttdf6fJ2WlhakpqYiLi7uRkBSKeLi4nDs2LEeX3Ps2LEu5wPAjBkzupyv1WrxyCOP4IUXXkBISEifYmlubkZNTU2Xh7HVNLViy8GLeOL/TkIQBKNfn4jMT255PZrbtHBS2mHEUEexwyGiDgatAlOr1di/fz+OHDmCU6dOoa6uDpMmTeqWmNxKRUUFNBoNvLy8uhz38vLC2bNne3xNSUlJj+d3rj168803YWdnh1WrVvU5lg0bNuDPf/5zP6I3zN8OXEBTqxbp+VWYNGyIyd+PiMSV2VEAHezrAqmUBdBE5sKgBEhn6tSpZrcjfGpqKjZv3oy0tLR+rbZYu3YtEhIS9L+vqalBQECAUWNzUckxO8wXO9IKkJicxwSIyAboV4CxAJrIrBg0BbZq1Sq888473Y6/++67ePbZZ/t8HXd3d8hkMpSWlnY5XlpaCm9v7x5f4+3tfdPzf/rpJ5SVlWHYsGGws7ODnZ0drl69ij/84Q8YMWJEr7EolUq4uLh0eZjC4pj2pOqbU8WobWo1yXsQkfnIZgdoIrNkUAK0Y8eOHkd+pkyZgi+++KLP11EoFIiMjERSUpL+mFarRVJSEmJjY3t8TWxsbJfzAWD//v368x955BGcPn0aGRkZ+oevry9eeOEF7N27t8+xmUrU8CEY5eGIxlYNdp8uFjscIjIhrVZAdlFHATRXgBGZFYOmwCorK6FWd//L7OLigoqKin5dKyEhAY8++iiioqIQExODTZs2ob6+HsuWLQMALF26FH5+ftiwYQMAYPXq1Zg2bRo2btyI2bNnIzExESkpKdi6dSsAYOjQoRg6dGiX95DL5fD29sb48eMN+bhGJZFIsDg6AK9/dxaJJ/OxJGaY2CERkYlcrqxHQ4sGKrkUo9xZAE1kTgwaARozZgz27NnT7fj333+PUaNG9eta8fHx+Otf/4qXX34ZERERyMjIwJ49e/SFznl5eSguvjFSMmXKFHz66afYunUrwsPD8cUXX2DXrl0IDQ015KOIYsEkf8hlEpzKr9IvjyUi66Or/wnycYGdzOCtF4nIBAwaAUpISMDKlStRXl6Ou+++GwCQlJSEjRs3GtRxeeXKlVi5cmWPzx06dKjbsUWLFmHRokV9vv6VK1f6HZMpuTsp8atgL3yXWYJtJ/Pxyty+LdUnIsuin/5iATSR2TEoAXr88cfR3NyM1157Da+++ioAYMSIEXjvvfewdOlSowZoreKjh+G7zBJ8mVaANfcFQiWXiR0SERlZ5z3AiMi8GDwm+/TTT6OgoAClpaWoqanBpUuXmPz0w+1j3OHnao+apjbsze7//mlEZN4EQdAnQCEcASIyOwOalC4vL8e5c+eQkZHR7+JnWyeTSrAoyh8AkJicL3I0RGRs+dcaUdPUBoVMinFezmKHQ0S/YFACVF9fj8cffxw+Pj648847ceedd8LHxwfLly9HQ0ODsWO0WouiAiCRAMcuVeJKRb3Y4RCREel2gB/v7QyFHQugicyNQX8rExIScPjwYXzzzTeoqqpCVVUVvvrqKxw+fBh/+MMfjB2j1fJztce0cR4AgG0pHAUisias/yEybwY3QvzXv/6F++67T981edasWfjnP//Zr0aIBCyObu8M/UVqAVo1WpGjISJjyepYAcb6HyLzZFAC1NDQ0G1DUgDw9PTkFFg/3R3oBXcnBcprm3HwbJnY4RCREQiCgGz9CBATICJzZFACFBsbi/Xr16OpqUl/rLGxEX/+85973cKCeqawk2LhpPZi6G0nOQ1GZA1KappQWd8CmVSCQG8WQBOZI4P6AG3atAkzZ86Ev78/wsPDAQCnTp2CSqUyi/22LM1D0QH4x4+XcPBcGUqqm+CtVokdEhENQFZh+/TXWE8n9vgiMlMGjQCFhYXhwoUL2LBhAyIiIhAREYE33ngDFy5cQEgIuxr312gPJ8SMdINWALazGJrI4rH/D5H56/cIUGtrKwIDA7F79248+eSTpojJJi2ODkDy5WvYlpKPFXeNgVQqETskIjJQdhFXgBGZu36PAMnl8i61P2Qc94X6wFllh4LrjTiaWyl2OEQ0AJksgCYyewZNga1YsQJvvvkm2trajB2PzbJXyDA/wg8AkHgyT+RoiMhQZbVNKK1phkQCBPtwBIjIXBlUBH3y5EkkJSVh3759CAsLg6OjY5fnv/zyS6MEZ2viowPw7+NXsS+7FNfqW+DmqBA7JCLqJ90O8KPcHeGoNOifWCIaBAb97XR1dcXChQuNHYvNC/VTI8xPjczCanyZVoAn7hgldkhE1E/s/0NkGQxKgD788ENjx0Ed4qMDkFlYjW0n87H89pGQSFgMTWRJdEvgQ7kCjMis9asGSKvV4s0338TUqVMRHR2NNWvWoLGx0VSx2aS5Eb5QyaW4UFaHtLwqscMhon7SbYIawhVgRGatXwnQa6+9hpdeeglOTk7w8/PD5s2bsWLFClPFZpNcVHLMDvMFAGxjMTSRRalqaEHB9fb/FLIHEJF561cC9PHHH+Pvf/879u7di127duGbb77BJ598Aq2Wm3ga05KY9g1SvzlVjNqmVpGjIaK+0hVAD3NzgNpeLnI0RHQz/UqA8vLyMGvWLP3v4+LiIJFIUFRUZPTAbFnk8CEY7eGIxlYNvjlVLHY4RNRHWYVsgEhkKfqVALW1tUGl6rpPlVwuR2srRymMSSKRYHH0MACcBiOyJFkdI0Cc/iIyf/1aBSYIAh577DEolUr9saamJjz11FNdegGxD9DAPTDJD/+z9yxOFVTjTFENgn35P0oic5fFJfBEFqNfCdCjjz7a7dhvfvMbowVDN7g7KfGrYC98l1mCz1Py8cpcbjJLZM5qm1pxuaIeABDK/7AQmb1+JUDs/zO4FkcPw3eZJfgyrQBr7guESi4TOyQi6sWZjukvX7UKQ52UtzibiMRm0F5gNDhuH+MOP1d71DS1YU9WidjhENFN6Ot/OP1FZBGYAJkxqVSCh6Lal8Rzg1Qi86bfAoMF0EQWgQmQmVsU5Q+JBDh+6Zq+voCIzI+uAzSXwBNZBiZAZs7X1R7TxnkAAD5PyRc5GiLqSWOLBhfL6gBwBRiRpWACZAF0PYG+SC1Aq4Zdt4nMTU5JDbRC++pNT2cWQBNZAiZAFuCeIE+4OylQXtuMA2fLxA6HiH4hu1MHaIlEInI0RNQXTIAsgFwmxcJIfwDAtpOcBiMyN5ksgCayOEyALER8x2qwQ+fKUFzdKHI0RNRZVmH7EngWQBNZDiZAFmKUhxNiRrpBKwBfpBSIHQ4RdWhu0+B8aS0A7gFGZEmYAFmQJTHto0DbUvKh1QoiR0NEAHC+pA5tWgGuDnL4D7EXOxwi6iMmQBbkvlAfOKvsUHC9EUdyK8QOh4jQqf+Pr5oF0EQWhAmQBVHJZXhgoh8AIJHF0ERmQbcDfAjrf4gsChMgCxMf3T4Nti+7BNfqW0SOhoh0e4BxBRiRZWECZGFCfNUI81OjVSPgyzQWQxOJqVWjRU6xbgUYEyAiS8IEyAIt1hVDn8yHILAYmkgsueV1aGnTwklph+FuDmKHQ0T9wATIAs0N94W9XIYLZXVIy7sudjhENiuzoL3+J9jXBVIpC6CJLImd2AFQ/zmr5Jg9wQdfpBbgT7uyEeJrXsWXjgoZVt49Fh5mtCfSVxmFAIB5EX4iR2Jbtp3Mg6PSDvdP8BU7FL2S6ib8/dBFNLRoBnytLHaAJrJYTIAs1JKYYfgitQA5xTX6GgRzohEE/L/5YWKHAQC4XFGP1YkZkEiAqWPc4e5kPomZNcssqMaLOzIhlQCThg2Br6t59Mj5675z+CLVuPVzk4a7GvV6RGR6TIAsVOTwIXj/N5G4XFEvdihdlNc244Mjl/FVRhH+OCsY9gqZ2CHp908ThPY9m+4a7ylyRLbhs5N5AACtAGxPKcDquLEiRwTUNLVi9+kiAMDv7hwFVwfFgK851FGB+0J9BnwdIhpcTIAs2MxQb7FD6EarFbA/pwT51xrxXWaxfhNXsbRqtF3+t5/NBGhQNLS04euMIv3vP0/Jx8q7x0Amcp3M1xlFaGrVYqynE9bcF8jGhUQ2jEXQZFRSqUS/cas57Fx/4GwZKuqa9b/XbVpJpvXt6WLUNbchwM0eans5CqsaceSi+N3LdX8m46MDmPwQ2TgmQGR0i6ICIJUAyVeuIbe8TtRYdF94EQGuAG5sW0Cmpbvvi6OH6buXi50QZxVWI7OwGgqZFAsmiTsySUTiYwJERuflosLdge3TTGJ+6RVXN+LQuTIAwJ/nhgAACq43oqqBHbRN6WJZLVKuXodMKsGDkf43upefKUFlp9G4wab7s3hviBfcHAde+0NElo0JEJlEfPQwAMCO1AK0tGlFiWF7SgG0AhAz0g3hAa4YPrS9UV12EafBTCkxuT3RuGu8J7xcVAjycUG4v657eaEoMTW2aLCroxXC4o4/m0Rk25gAkUncNd4Dns5KVNa3ICmndNDfX6sVOk3DtI9A6Hq16Hq3kPE1t2nwZbou0QjQH9clxIkn80TpXv59VjFqm9rgP8QeU0YPHfT3JyLzwwSITMJOJsWDHSvAxNi5/khuBQqrGuGsssOssPYlyrrdurM4AmQyP5wpw7X6Fni5KDF9vIf++JxwH9jLZcgtr0fq1cHvXq77MxgfFcCOzUQEgAkQmZCu9uPHC+UorGoc1PfWfeE9MNEPKnl7LyLdCFA2R4BMJrGj98+iyADYyW788+KskmNOuE/HOYObEF8qr0Py5WuQStoL9ImIACZAZELDhzpiyuihEATg80H80rtW34J92SUAbiRhAPRbhlyqqEdtU+ugxWMr8q814OeOpe4P9ZBo6KbBvj1djJpBvP+6qdC7xnvCW60atPclIvPGBIhMSpeAbE/Jh0Y7OLUfX6YVoFUjIMxPjZBOezQNdVLCt+ML8AynwYxue0o+BAGYOmYohg3tvjP6pGGuGOvphMZWTZcmiabU0qbFjrT2Rpidk2EiIiZAZFIzQryhtpejqLoJP10oN/n7CYKgn2JZHNP9Cy/Er6MQmgmQUWm0Aj5P0SUaPa+ykkgk+iRksNojHDhbioq6Fng4K3FXIDuAE9ENTIDIpFRy2aA2wkvLu46LZXWwl8swN7z7DuRhfqwDMoUfz5ejpKYJrg5yzAjx6vW8BZP8oZBJkVlYPSir8XTJ8IOR/pDL+M8dEd3AfxHI5HQjMfvPlKK81rSN8HQ9aGZP8IGzSt7t+VD9SjAmQMakK35eMNEfSrveN8B1c1Tg3o4E6fMU0ybEhVWNOHy+fdQxnsXPRPQLTIDI5AK9XRAR4Io2rYAv0wpu/QID1Ta1YvfpYgBde9B0plsJdrGsDo0tGpPFYkvKapuQlNPecbsvdTa6RoQ70wvR1Gq6n4GuJil21FCMcHc02fsQkWViAkSDYnGn2g9TNcL7+lQRGls1GOPphMjhQ3o8x9NFBQ9nJbQCkFPCOiBj2JFaiDatgInDXDHe2/mW508ZPRT+Q+xR29SG7zKLTRKTRitge0dNUk+1YERETIBoUNwf7gsHhQyXKupx8oppGuF17vx8s52+QzuWw7MOaOAEQcC2jumv3kbdfkkqleinpEzVE+jni+2NMNX2cswI8TbJexCRZWMCRIPCSWmHORPai5J19SLGlF1UjdMF1ZDLJPqi696E6laCFXIEaKBOXL6GK5UNcFTIcP+E7kXnvXkwyh9SCZB8+RouldcZPS5dUta5ESYRUWdMgGjQ6KYivsssRnWjcRvh6Rot3hvsjaFOypueq+sNxELogdONus2N8IWj0q7Pr/NR22P6+PZl6duMXAxdUdeM/Wfa959j7x8i6o1ZJEBbtmzBiBEjoFKpMHnyZCQnJ9/0/O3btyMwMBAqlQphYWH47rvvujz/yiuvIDAwEI6OjhgyZAji4uJw4sQJU34E6oOIAFeM93JGU6sWX2cYb1fwplYNdnZswNmXLzzdSrDzpbVobmMhtKGqG1r1NTy99f65Gd2U2Y7UArRqtEaLS9cIMzzAFUE+Lka7LhFZF9EToG3btiEhIQHr169HWloawsPDMWPGDJSVlfV4/tGjR7FkyRIsX74c6enpmD9/PubPn4+srCz9OePGjcO7776LzMxM/PzzzxgxYgTuvfdelJebvhEf9a5zIzxj1n58n1WMmqY2+Lna4/Yx7rc838/VHq4OcrRqBJwvMf70i63YlVGI5jYtAr2dEe6vvvULfuGuQE94OCtRUdeCpJxSo8TUpREmR3+I6CZET4DeeustPPnkk1i2bBmCg4Px/vvvw8HBAR988EGP52/evBkzZ87ECy+8gKCgILz66quYNGkS3n33Xf05v/71rxEXF4dRo0YhJCQEb731FmpqanD69OnB+ljUiwcm+kEhkyK7qMZojfB0vX/io/u207dEItEvh+c0mGEEQcBnyTeKn29WdN4buUyKByP9ARgvIU65eh2XyuvhoJBhTg+NMImIdERNgFpaWpCamoq4uDj9MalUiri4OBw7dqzH1xw7dqzL+QAwY8aMXs9vaWnB1q1boVarER4e3msszc3NqKmp6fIg4xviqMCM0PZVOcYohr5UXocTHTt9675M+yJE1xCRK8EMkllYjbMltVDYSTH/FkXnN6PbNPXw+XIUVTUOOC5dMnz/BB849aMmiYhsj6gJUEVFBTQaDby8urbO9/LyQklJSY+vKSkp6dP5u3fvhpOTE1QqFd5++23s378f7u69T49s2LABarVa/wgI4PC5qeimJr5KL0JDS9uArqXbf2raOA/4utr3+XVh3BNsQHQjNveFesPVQWHwdUa6O+K2UW4QBOj79hiqurEV32a2b7JqSE0SEdkW0afATOWuu+5CRkYGjh49ipkzZ+Khhx7qta4IANauXYvq6mr9Iz9/cDZrtEWxo4ZimJsDapvb8F1mz4luX7RqtPgi9eYbcPZGNwWWU1xj1AJcW9DQ0qbfzd0Yq6yWxLT/7D5PyYdGa3iTzK9PFaGpVYtxXk6YNMx1wHERkXUTNQFyd3eHTCZDaWnXAsjS0lJ4e/fcvMzb27tP5zs6OmLMmDG47bbb8K9//Qt2dnb417/+1WssSqUSLi4uXR5kGlJp513BDZ8GS8opQ0VdM9ydlLgnqH87fQ9zc4Cz0g4tbVrkmqAPjTXbfboYdc1tGD7UAbeNHDrg680I8YbaXo7Cqkb8fLHC4Ovo/izFRw8zqCaJiGyLqAmQQqFAZGQkkpKS9Me0Wi2SkpIQGxvb42tiY2O7nA8A+/fv7/X8ztdtbjbtRpzUdw9GtjfCO3mlffd2Q+i+8AzZ6VsqlSDYV1cHxGmw/tD1/ulr0fmtqOQyffNKQxPirMJqZBXWQCGT3rIRJhERYAZTYAkJCfjnP/+J//u//0NOTg6efvpp1NfXY9myZQCApUuXYu3atfrzV69ejT179mDjxo04e/YsXnnlFaSkpGDlypUAgPr6erz00ks4fvw4rl69itTUVDz++OMoLCzEokWLRPmM1J2Xiwp3B3Y0wjPgS6+o807fBk7D3OgIzULovrpQWovUq9chk0rw4KS+F53fiu5nuP9MKSrq+v8fFV1B/b0hXnBzNLwmiYhsh+gJUHx8PP7617/i5ZdfRkREBDIyMrBnzx59oXNeXh6Ki29smDhlyhR8+umn2Lp1K8LDw/HFF19g165dCA0NBQDIZDKcPXsWCxcuxLhx4zBnzhxUVlbip59+QkhIiCifkXqmq9vZkVaIlrb+1eF8kVoArQBMHumGkQbu9K1riJjNpfB9phv9uTvQE54uKqNdN8jHBeH+arRqBOxM61+TzMYWDb5Kb69JWsziZyLqI7NYJ7py5Ur9CM4vHTp0qNuxRYsW9Tqao1Kp8OWXXxozPDKRu8Z7wNNZibLaZvyQU4pZYT59ep1WK9zY+HQAO33rCqGzi2qg0QqQGWE6x5o1t2nwZUfHbVM0GVwcMwynCjKReDIPT9wxss91PN9lFqO2uQ0BbvaYMnrgNUlEZBtEHwEi22Unk2JRVP8b4el2+nZR2eG+0L4lTT0Z5eEElVyKhhYNLlfUG3wdW7H/TCmu1bfAy0WJaeM8jH79OeG+cFDIkFtej5Sr1/v8On1NUpRxapKIyDYwASJR6Rrh/XShHAXXG/r0Gt0X3kB3+pZJJQj24TRYX+nu+0NRAbDrZ9F5Xzgp7XD/hPaEVtfQ8FZyy+uQfEXXCJO9u4io75gAkaiGD3XElNFDIQg3mhreTGVdM/adae8dZIxmdyyE7pv8aw346UL7EnVd0moKup/pt5lFqGlqveX5uqTsrvGe8FYbryaJiKwfEyASnW4F0PY+NMLbmV6IVo2ACf5q/TL2gdDvCcal8De1PaU90bh9jDsC3BxM9j6ThrlirKcTmlq1+maLvWlp02KHvhEmR3+IqH+YAJHoZoR4w9VBjuLqJvx4obzX8zrv9G2sLzz9CFBRNQTB8C7E1kyjFfSjc6ZONCQSCRZ3dIbedou6sKScUlTWt8DTWalvqUBE1FdMgEh0XRrh3aT2I/Vqe9NEe7kMc4200/dYLycoZFLUNrUh/9rAN+O0RofPl6GkpglDHOS4N8Tr1i8YoAcm+kEhkyKzsPqmU5O6ZPjBSH+T1CQRkXXjvxpkFnQjCz/klKK8tudGeLovvPsn+MBZJTfK+8plUgT6OANoHwWi7nQFyQsm+UNpZ3jReV+5OSr0iVZvo0AF1xv0o4WmrEkiIuvFBIjMQqC3CyICXNGmFbAjrXsxdE1TK7493d4QcyC9f3oS4stC6N6U1TYh6Wz7JsKDWWeja2i4K6MQjS2abs9vTymAILRvrDvCwEaYRGTbmACR2Vis3yA1v1s9zjenitDYqsEYTydMGjbEqO+r6widVcRC6F/akVoIjVbApGGuGOflPGjvO2X0UAS42aO2qQ3fZxV3eU6jFfRF2cZOhonIdjABIrMxJ9wXjgoZLlfUI/nytS7P6Ts/RwcYfafv0E4jQCyEvkEQBP0+bYO9xYRUKkF8x9TWL5tk/nShHEXVTVDbyzEjxHtQ4yIi68EEiMyGo9IOczqKmzvXfmQXVeN0QTXkMgkWGHEDTp3x3s6QSSW4Vt+C4uomo1/fUh2/dA1XKhvgpLTD7AmGd9w21IORAZBKgOTL15BbXqc/bqxGmERk25gAkVnR1Zl8m1mM6ob2Rni6L7x7Q7xNstO3Si7DWE8nAKwD6kw3+jMn3BeOysHfNtBbrcJd49uXt3/e8WegvLYZ+8+UAmDvHyIaGCZAZFYiAlwx3ssZzW1afHWqEE2tGuw04QacOjf6AbEOCACqG1rxXVZ7x21T3vdb0SU5O9IK0NKmxZdpBWjTCggPcEWQz8AbYRKR7WICRGZFIpHov/Q+S85v3+m7qQ3+Q+wxdbS7yd43tKOrdDZHgAC0r75qadMi0NsZE/zVosVxV6AnPJyVqKhrQVJOaZdaMCKigWACRGZnwSQ/KOykyCmuwcZ95wGYfqfvMP8bHaFtnSAI+Cy5ffprScwwoxed94dcJsWiyPa6r//3bQ4uVdTDQSHT14oRERmKCRCZHVcHBWZ2rO4prGps3+k7yvjFz50F+bhAIgFKa5pRVmvbhdCnC6pxtqQWCjsp5kf4iR2OvtFhYVV7p+45E3zhJEJNEhFZF/4rQmZpcXQAvj7Vvhnm9PGe8FHbm/T9HBR2GO3hhItldcguqoHneNPvLH6tvgXvJF1AXXObyd+rP3KK2+ugZoV6Q+1gnI7bAzHC3RGxo4bi2KVKAEA8e/8QkREwASKzdNuooRjp7ojLFfVYEjM4PWhCfV3aE6DCav3qI1P624EL+OjoFZO/j6EWD9J974tfTx6GY5cqEejtjIkBrmKHQ0RWgAkQmSWpVIIPH4vGudJa/CrY9BtwAu0rwXZlFCFzEAqhO69ue2zKCHi5mH7EqT+GD3XAbaOGih2G3v0TfCCTShDmpxa1JomIrAcTIDJbI9wdB3Wfpxt7gpl+Kfze7BJUNbTCR63CuvuDITNhgbc1kEgkmBU2+M0Yich6sQiaqENwx1L4wqpGXK9vMel76ZZzL4oKYPJDRCQCJkBEHdT2cgwf6gAAyDZhQ8SrlfU4mlsJiQR4yMSr24iIqGdMgIg60W+MasJ+QJ937GR+x1gP+A9xMNn7EBFR75gAEXUS4tc+DWaqPcHaNFpsTykAwG7GRERiYgJE1IluBMhUU2AHz5WjrLYZbo4KxAUNzuo2IiLqjgkQUSe6TVEvV9SjtqnV6NfX7bC+sGO7DyIiEgf/BSbqxM1RAT/X9q7TZ4w8ClRS3YQDZ8sAAPHR5tNkkIjIFjEBIvqFkI7l8FlGToB2pBVAKwDRI4ZgjKeTUa9NRET9wwSI6Bd002DGLITWagV97x+O/hARiY8JENEvhJpgJdixS5XIu9YAZ6UdZoV5G+26RERkGCZARL+gWwmWW16Hhhbj7NSe2DH6M2+iLxwU3IGGiEhsTICIfsHTRQUPZyW0ApBTXDvg612vb8HerBIAwGJOfxERmQUmQEQ9CO0ohM42QkfonemFaNFoEeLroq8vIiIicTEBIuqBsQqhBeFG8TM7PxMRmQ8mQEQ9CNHtCVY4sKXw6flVOFdaC5VcirkRfsYIjYiIjIAJEFEPwvzbE6DzpbVobtMYfJ1tye2jP7PCfKC2lxslNiIiGjgmQEQ98FWrMMRBjjatgPMldQZdo665Dd+cLgLA4mciInPDBIioBxKJ5EYdkIGF0LtPFaGhRYNR7o6IHjHEmOEREdEAMQEi6oWuDijTwELoRH3n5wBIJBKjxUVERAPHBIioF7qO0NkGJEBnS2qQkV8FO6kECyP9jR0aERENEBMgol7oOkLnlNSiVaPt12sTO4qffxXsBXcnpdFjIyKigWECRNSLYW4OcFbaoaVNi4tlfS+EbmrVYGd6IYD26S8iIjI/TICIeiGVShDs2/+NUfdml6C6sRW+ahXuGOthqvCIiGgAmAAR3YRuJVh2Ud8bIuo6Py+KCoBMyuJnIiJzxASI6CZ0hdB9HQG6WlmPo7mVkEiAhzj9RURktpgAEd2ErhD6THENNFrhlufrRn/uHOsBP1d7k8ZGRESGYwJEdBOjPJxgL5ehoUWDyxX1Nz23TaPF9tQCANz4lIjI3DEBIroJWT8KoQ+eK0d5bTOGOipwT5DXYIRHREQGYgJEdAuhfUyAtp3MAwA8GOkPhR3/ahERmTP+K010CyF92BOspLoJB86WAWDxMxGRJWACRHQLukLo7MIaaHsphN6RVgCtAMSMcMNoD6fBDI+IiAzABIjoFsZ6OUEhk6K2uQ351xu6Pa/VCkjsmP5i52ciIsvABIjoFuQyKQJ9nAEAWYXdGyIeu1SJ/GuNcFbZYVaYz2CHR0REBmACRNQHIb691wEldvT+mR/hB3uFbFDjIiIiwzABIuqD3jpCX69vwd6sEgCc/iIisiRMgIj6QF8IXVQDQbhRCL0zvRAtGi1C/Vz0+4YREZH5YwJE1AfjvZ1hJ5XgWn0LiqubAACC0Ln4eZiY4RERUT8xASLqA5VchrFe7YXQmR3TYOn5VThfWgeVXIp5Eb5ihkdERP3EBIioj3QdobM7EqBtye3Fz7PDfOGikosWFxER9R8TIKI+CtV3hK5BXXMbvjldBABYHMPiZyIiS8MEiKiPOq8E232qCA0tGozycETU8CEiR0ZERP1lFgnQli1bMGLECKhUKkyePBnJyck3PX/79u0IDAyESqVCWFgYvvvuO/1zra2tePHFFxEWFgZHR0f4+vpi6dKlKCoqMvXHICsX5OMCiQQoq23GP368BABYHB0AiUQicmRERNRfoidA27ZtQ0JCAtavX4+0tDSEh4djxowZKCsr6/H8o0ePYsmSJVi+fDnS09Mxf/58zJ8/H1lZWQCAhoYGpKWlYd26dUhLS8OXX36Jc+fOYe7cuYP5scgKOSjs9Pt8Xa6oh1wmwYJJ/iJHRUREhpAInZuaiGDy5MmIjo7Gu+++CwDQarUICAjAM888gzVr1nQ7Pz4+HvX19di9e7f+2G233YaIiAi8//77Pb7HyZMnERMTg6tXr2LYsL4tV66pqYFarUZ1dTVcXFwM+GRkjZ5NTMeujPbRxFlh3vj7w5EiR0RERJ319ftb1BGglpYWpKamIi4uTn9MKpUiLi4Ox44d6/E1x44d63I+AMyYMaPX8wGguroaEokErq6uvZ7T3NyMmpqaLg+iX+rc7JC9f4iILJeoCVBFRQU0Gg28vLy6HPfy8kJJSUmPrykpKenX+U1NTXjxxRexZMmSm2aCGzZsgFqt1j8CAriyh7qL7Ch4DnCzx+1j3EWOhoiIDCV6DZAptba24qGHHoIgCHjvvfdueu7atWtRXV2tf+Tn5w9SlGRJJg4bgn89GoWPH58MmZTFz0RElspOzDd3d3eHTCZDaWlpl+OlpaXw9vbu8TXe3t59Ol+X/Fy9ehUHDhy4ZR2PUqmEUqk04FOQrbknyOvWJxERkVkTdQRIoVAgMjISSUlJ+mNarRZJSUmIjY3t8TWxsbFdzgeA/fv3dzlfl/xcuHABP/zwA4YOHWqaD0BEREQWSdQRIABISEjAo48+iqioKMTExGDTpk2or6/HsmXLAABLly6Fn58fNmzYAABYvXo1pk2bho0bN2L27NlITExESkoKtm7dCqA9+XnwwQeRlpaG3bt3Q6PR6OuD3NzcoFAoxPmgREREZDZET4Di4+NRXl6Ol19+GSUlJYiIiMCePXv0hc55eXmQSm8MVE2ZMgWffvop/vSnP+Gll17C2LFjsWvXLoSGhgIACgsL8fXXXwMAIiIiurzXwYMHMX369EH5XERERGS+RO8DZK7YB4iIiMjyWEQfICIiIiIxMAEiIiIim8MEiIiIiGwOEyAiIiKyOUyAiIiIyOYwASIiIiKbwwSIiIiIbA4TICIiIrI5TICIiIjI5oi+FYa50jXIrqmpETkSIiIi6ivd9/atNrpgAtSL2tpaAEBAQIDIkRAREVF/1dbWQq1W9/o89wLrhVarRVFREZydnSGRSPr0mpqaGgQEBCA/P5/7hw0i3ndx8L4PPt5zcfC+i8PQ+y4IAmpra+Hr69tlM/Vf4ghQL6RSKfz9/Q16rYuLC/+SiID3XRy874OP91wcvO/iMOS+32zkR4dF0ERERGRzmAARERGRzWECZERKpRLr16+HUqkUOxSbwvsuDt73wcd7Lg7ed3GY+r6zCJqIiIhsDkeAiIiIyOYwASIiIiKbwwSIiIiIbA4TICIiIrI5TICMZMuWLRgxYgRUKhUmT56M5ORksUOyKj/++CPmzJkDX19fSCQS7Nq1q8vzgiDg5Zdfho+PD+zt7REXF4cLFy6IE6wV2bBhA6Kjo+Hs7AxPT0/Mnz8f586d63JOU1MTVqxYgaFDh8LJyQkLFy5EaWmpSBFbh/feew8TJkzQN4CLjY3F999/r3+e99z03njjDUgkEjz77LP6Y7zvpvHKK69AIpF0eQQGBuqfN9V9ZwJkBNu2bUNCQgLWr1+PtLQ0hIeHY8aMGSgrKxM7NKtRX1+P8PBwbNmypcfn/+d//gfvvPMO3n//fZw4cQKOjo6YMWMGmpqaBjlS63L48GGsWLECx48fx/79+9Ha2op7770X9fX1+nOee+45fPPNN9i+fTsOHz6MoqIiLFiwQMSoLZ+/vz/eeOMNpKamIiUlBXfffTfmzZuH7OxsALznpnby5En84x//wIQJE7oc5303nZCQEBQXF+sfP//8s/45k913gQYsJiZGWLFihf73Go1G8PX1FTZs2CBiVNYLgLBz507977VareDt7S385S9/0R+rqqoSlEql8Nlnn4kQofUqKysTAAiHDx8WBKH9PsvlcmH79u36c3JycgQAwrFjx8QK0yoNGTJE+N///V/ecxOrra0Vxo4dK+zfv1+YNm2asHr1akEQ+GfdlNavXy+Eh4f3+Jwp7ztHgAaopaUFqampiIuL0x+TSqWIi4vDsWPHRIzMdly+fBklJSVdfgZqtRqTJ0/mz8DIqqurAQBubm4AgNTUVLS2tna594GBgRg2bBjvvZFoNBokJiaivr4esbGxvOcmtmLFCsyePbvL/QX4Z93ULly4AF9fX4waNQoPP/ww8vLyAJj2vnMz1AGqqKiARqOBl5dXl+NeXl44e/asSFHZlpKSEgDo8Wege44GTqvV4tlnn8XUqVMRGhoKoP3eKxQKuLq6djmX937gMjMzERsbi6amJjg5OWHnzp0IDg5GRkYG77mJJCYmIi0tDSdPnuz2HP+sm87kyZPx0UcfYfz48SguLsaf//xn3HHHHcjKyjLpfWcCRER9smLFCmRlZXWZmyfTGT9+PDIyMlBdXY0vvvgCjz76KA4fPix2WFYrPz8fq1evxv79+6FSqcQOx6bcd999+l9PmDABkydPxvDhw/H555/D3t7eZO/LKbABcnd3h0wm61aRXlpaCm9vb5Gisi26+8yfgemsXLkSu3fvxsGDB+Hv768/7u3tjZaWFlRVVXU5n/d+4BQKBcaMGYPIyEhs2LAB4eHh2Lx5M++5iaSmpqKsrAyTJk2CnZ0d7OzscPjwYbzzzjuws7ODl5cX7/sgcXV1xbhx43Dx4kWT/nlnAjRACoUCkZGRSEpK0h/TarVISkpCbGysiJHZjpEjR8Lb27vLz6CmpgYnTpzgz2CABEHAypUrsXPnThw4cAAjR47s8nxkZCTkcnmXe3/u3Dnk5eXx3huZVqtFc3Mz77mJ3HPPPcjMzERGRob+ERUVhYcfflj/a973wVFXV4fc3Fz4+PiY9s/7gEqoSRAEQUhMTBSUSqXw0UcfCWfOnBF++9vfCq6urkJJSYnYoVmN2tpaIT09XUhPTxcACG+99ZaQnp4uXL16VRAEQXjjjTcEV1dX4auvvhJOnz4tzJs3Txg5cqTQ2NgocuSW7emnnxbUarVw6NAhobi4WP9oaGjQn/PUU08Jw4YNEw4cOCCkpKQIsbGxQmxsrIhRW741a9YIhw8fFi5fviycPn1aWLNmjSCRSIR9+/YJgsB7Plg6rwITBN53U/nDH/4gHDp0SLh8+bJw5MgRIS4uTnB3dxfKysoEQTDdfWcCZCR/+9vfhGHDhgkKhUKIiYkRjh8/LnZIVuXgwYMCgG6PRx99VBCE9qXw69atE7y8vASlUincc889wrlz58QN2gr0dM8BCB9++KH+nMbGRuH3v/+9MGTIEMHBwUF44IEHhOLiYvGCtgKPP/64MHz4cEGhUAgeHh7CPffco09+BIH3fLD8MgHifTeN+Ph4wcfHR1AoFIKfn58QHx8vXLx4Uf+8qe67RBAEYWBjSERERESWhTVAREREZHOYABEREZHNYQJERERENocJEBEREdkcJkBERERkc5gAERERkc1hAkREREQ2hwkQEZEZunjxIl5//XU0NjaKHQqRVWICREQGkUgk2LVr14Cvc+XKFUgkEmRkZAz4WoPlo48+gqur64CvM2LECGzatKnb8aamJjz44IPw9fU16W7YRLbMTuwAiMg8PfbYY6iqquo1ySkuLsaQIUMGNygzER8fj1mzZpns+s888wzmz5+Pxx57zGTvQWTrmAARkUG8vb3FDkE09vb2Jh2Z+ec//2myaxNRO06BEZFBfjkFVlBQgCVLlsDNzQ2Ojo6IiorCiRMnALRP9Ugkkm6Pzs6ePYspU6ZApVIhNDQUhw8f1j+n0WiwfPlyjBw5Evb29hg/fjw2b9580/iuX7+Ohx9+GB4eHrC3t8fYsWPx4Ycf6p9/8cUXMW7cODg4OGDUqFFYt24dWltb9c+fOnUKd911F5ydneHi4oLIyEikpKQA6D4Flpubi3nz5sHLywtOTk6Ijo7GDz/80CWesrIyzJkzB/b29hg5ciQ++eSTbjHn5eVh3rx5cHJygouLCx566CGUlpYCAKqrqyGTyfQxaLVauLm54bbbbtO//j//+Q8CAgJuel+IqB1HgIhowOrq6jBt2jT4+fnh66+/hre3N9LS0qDVagEAJ0+ehEajAdCezDz44IOQy+VdrvHCCy9g06ZNCA4OxltvvYU5c+bg8uXLGDp0KLRaLfz9/bF9+3YMHToUR48exW9/+1v4+PjgoYce6jGmdevW4cyZM/j+++/h7u6OixcvdikodnZ2xkcffQRfX19kZmbiySefhLOzM/7rv/4LAPDwww9j4sSJeO+99yCTyZCRkdEt5s6ff9asWXjttdegVCrx8ccfY86cOTh37hyGDRsGoH1KsaioCAcPHoRcLseqVatQVlamv4ZWq9UnP4cPH0ZbWxtWrFiB+Ph4HDp0CGq1GhERETh06BCioqKQmZkJiUSC9PR01NXV6V83bdo0A3+KRDZmwPvJE5FVevTRR4V58+b1+jwAYefOnYIgCMI//vEPwdnZWaisrLzldVetWiUMHz5cKCsrEwRBEC5fviwAEN544w39Oa2trYK/v7/w5ptv9nqdFStWCAsXLuz1+Tlz5gjLli27ZTw6f/nLX4TIyEj9752dnYWPPvqox3M//PBDQa1W3/R6ISEhwt/+9jdBEATh3LlzAgAhOTlZ/3xOTo4AQHj77bcFQRCEffv2CTKZTMjLy9Ofk52d3eV1CQkJwuzZswVBEIRNmzYJ8fHxQnh4uPD9998LgiAIY8aMEbZu3drnz0xkyzgFRkQDlpGRgYkTJ8LNze2m523duhX/+te/8PXXX8PDw6PLc7Gxsfpf29nZISoqCjk5OfpjW7ZsQWRkJDw8PODk5IStW7ciLy+v1/d6+umnkZiYiIiICPzXf/0Xjh492uX5bdu2YerUqfD29oaTkxP+9Kc/dbleQkICnnjiCcTFxeGNN95Abm5ur+9VV1eH559/HkFBQXB1dYWTkxNycnL018vJyYGdnR0iIyP1rwkMDOwyjZaTk4OAgIAuU1jBwcFwdXXV34dp06bh559/hkajweHDhzF9+nRMnz4dhw4dQlFRES5evIjp06f3GicR3cAEiIgGrC8FwQcPHsQzzzyDjz/+GBMmTOjX9RMTE/H8889j+fLl2LdvHzIyMrBs2TK0tLT0+pr77rsPV69exXPPPYeioiLcc889eP755wEAx44dw8MPP4xZs2Zh9+7dSE9Pxx//+Mcu13vllVeQnZ2N2bNn48CBAwgODsbOnTt7fK/nn38eO3fuxOuvv46ffvoJGRkZCAsLu2l8hrjzzjtRW1uLtLQ0/Pjjj10SoMOHD8PX1xdjx4416nsSWSsmQEQ0YBMmTEBGRgauXbvW4/MXL17Egw8+iJdeegkLFizo8Zzjx4/rf93W1obU1FQEBQUBAI4cOYIpU6bg97//PSZOnIgxY8bcdERGx8PDA48++ij+85//YNOmTdi6dSsA4OjRoxg+fDj++Mc/IioqCmPHjsXVq1e7vX7cuHF47rnnsG/fPixYsKBLEXVnR44cwWOPPYYHHngAYWFh8Pb2xpUrV/TPBwYG6j+Tzrlz51BVVaX/fVBQEPLz85Gfn68/dubMGVRVVSE4OBgA4OrqigkTJuDdd9+FXC5HYGAg7rzzTqSnp2P37t2s/yHqByZARNSr6upqZGRkdHl0/oLWWbJkCby9vTF//nwcOXIEly5dwo4dO3Ds2DE0NjZizpw5mDhxIn7729+ipKRE/+hsy5Yt2LlzJ86ePYsVK1bg+vXrePzxxwEAY8eORUpKCvbu3Yvz589j3bp1OHny5E1jf/nll/HVV1/h4sWLyM7Oxu7du/UJ1dixY5GXl4fExETk5ubinXfe6TK609jYiJUrV+LQoUO4evUqjhw5gpMnT+pf/0tjx47Fl19+iYyMDJw6dQq//vWv9QXgADB+/HjMnDkTv/vd73DixAmkpqbiiSee6DJyFhcXh7CwMDz88MNIS0tDcnIyli5dimnTpiEqKkp/3vTp0/HJJ5/okx03NzcEBQVh27ZtTICI+kPsIiQiMk+PPvqoAKDbY/ny5YIgdC2CFgRBuHLlirBw4ULBxcVFcHBwEKKiooQTJ07oi5x7egjCjSLoTz/9VIiJiREUCoUQHBwsHDhwQH/tpqYm4bHHHhPUarXg6uoqPP3008KaNWuE8PDwXuN/9dVXhaCgIMHe3l5wc3MT5s2bJ1y6dEn//AsvvCAMHTpUcHJyEuLj44W3335bX9jc3NwsLF68WAgICBAUCoXg6+srrFy5UmhsbBQEoXsR9OXLl4W77rpLsLe3FwICAoR3331XmDZtmrB69Wr9OcXFxcLs2bMFpVIpDBs2TPj444+F4cOH64ugBUEQrl69KsydO1dwdHQUnJ2dhUWLFgklJSVdPtfOnTsFAMJ7772nP7Z69WoBgHD27Nmb/kyJ6AaJIAiCCHkXEZFZWbBgAdasWYOYmBixQyGiQcAEiIhsXlVVFYYPH47r168DAKRSVgcQWTv+LScim+fi4oKYmBj4+/vjnXfeETscIhoEHAEiIiIim8MRICIiIrI5TICIiIjI5jABIiIiIpvDBIiIiIhsDhMgIiIisjlMgIiIiMjmMAEiIiIim8MEiIiIiGwOEyAiIiKyOf8fRA8JbyMXIqUAAAAASUVORK5CYII=",
|
||
"text/plain": [
|
||
"<Figure size 640x480 with 1 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"import matplotlib.pyplot as plt\n",
|
||
"# changing to misclassification error\n",
|
||
"MSE = [1 - x for x in cv_scores]\n",
|
||
"\n",
|
||
"# plot misclassification error vs k\n",
|
||
"plt.plot(neighbors, MSE)\n",
|
||
"plt.xlabel('Liczba sąsiadów')\n",
|
||
"plt.ylabel('Procent błędów')\n",
|
||
"plt.show()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## TF IDF Vectorizer\n",
|
||
"\n",
|
||
"Czasami, żeby wytrenować model nie da się zastosować bezpośrednio danego typu danych, ponieważ najczęściej wejściem do algorytmu ML jest wektor, macierz lub tensor.\n",
|
||
"Dane tekstowe musimy również przekształcić do wektorów. Przydatny w tym przypadku jest TF IDF Vectorizer.\n",
|
||
"Oto przyład z dokumentacji jak można z niego skorzystać (https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html)\n",
|
||
"\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 70,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"(4, 9)\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from sklearn.feature_extraction.text import TfidfVectorizer\n",
|
||
"corpus = [\n",
|
||
" 'This is the first document.',\n",
|
||
" 'This document is the second document.',\n",
|
||
" 'And this is the third one.',\n",
|
||
" 'Is this the first document?',\n",
|
||
"]\n",
|
||
"vectorizer = TfidfVectorizer()\n",
|
||
"X = vectorizer.fit_transform(corpus)\n",
|
||
"vectorizer.get_feature_names_out()\n",
|
||
"print(X.shape)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 73,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"array(['and', 'document', 'first', 'is', 'one', 'second', 'the', 'third',\n",
|
||
" 'this'], dtype=object)"
|
||
]
|
||
},
|
||
"execution_count": 73,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"vectorizer.get_feature_names_out()\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 71,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"<4x9 sparse matrix of type '<class 'numpy.float64'>'\n",
|
||
"\twith 21 stored elements in Compressed Sparse Row format>"
|
||
]
|
||
},
|
||
"execution_count": 71,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"X"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 72,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"matrix([[0. , 0.46979139, 0.58028582, 0.38408524, 0. ,\n",
|
||
" 0. , 0.38408524, 0. , 0.38408524],\n",
|
||
" [0. , 0.6876236 , 0. , 0.28108867, 0. ,\n",
|
||
" 0.53864762, 0.28108867, 0. , 0.28108867],\n",
|
||
" [0.51184851, 0. , 0. , 0.26710379, 0.51184851,\n",
|
||
" 0. , 0.26710379, 0.51184851, 0.26710379],\n",
|
||
" [0. , 0.46979139, 0.58028582, 0.38408524, 0. ,\n",
|
||
" 0. , 0.38408524, 0. , 0.38408524]])"
|
||
]
|
||
},
|
||
"execution_count": 72,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"X.todense()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Na podstawie tych danych możemy wytrenowąc model regresji logistycznej. Jest to model regresji liniowej z dodatkową nałożoną funkcją logistyczną:\n",
|
||
" ( https://en.wikipedia.org/wiki/Logistic_function )\n",
|
||
" \n",
|
||
" \n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"Dzięki wyjściu modelu zawsze pomiędzy 0, a 1 można traktować wynik jako prawdopodobieństwo\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 75,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from sklearn.linear_model import LogisticRegression"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 81,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"array([0, 0, 1, 0])"
|
||
]
|
||
},
|
||
"execution_count": 81,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"y = [0,0,1,1]\n",
|
||
"model = LogisticRegression()\n",
|
||
"model.fit(X, y)\n",
|
||
"model.predict(X)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 82,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"array([[0.51514316, 0.48485684],\n",
|
||
" [0.56428483, 0.43571517],\n",
|
||
" [0.40543928, 0.59456072],\n",
|
||
" [0.51514316, 0.48485684]])"
|
||
]
|
||
},
|
||
"execution_count": 82,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"model.predict_proba(X)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Sieci neuronowe\n",
|
||
"\n",
|
||
"Warto zauważyć, że sieci neuronowe w najprostszym wariancie to tak naprawdę złożenie wielu funkcji regresji logistycznej ze sobą, gdzie wejściem jednego modelu regresji logistycznej jest wyjście poprzedniej. W przypadku danych tekstowych zazwyczaj jest wybierana wtedy inna reprezentacja danych niż TF IDF, ponieważ TF IDF nie uwzględnia kolejności słów"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Standard Scaler\n",
|
||
"\n",
|
||
"**Zadanie 7**\n",
|
||
"\n",
|
||
"\n",
|
||
"Sprawdź dokumentację https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html\n",
|
||
"\n",
|
||
"KNN jest wrażliwy na liniowe skalowanie danych (w przeciwieństwie do modeli bazujących na regresji, gdyż współczynniki liniowe rekompensują skalowanie liniowe).\n",
|
||
"\n",
|
||
"Wytrenuj dowolny model KNN na cechach pozyskanych ze StandardScaler. Pamiętaj, żeby wyskalować zarówno dane ze zbioru test jak i train.\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"Zauważ, że scaler ma podobne API (fit, transform) jak TF IDF Vectorier"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.11.7"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 4
|
||
}
|