{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Uczenie maszynowe\n", "# 10. Sieci neuronowe – wprowadzenie" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [], "source": [ "# Przydatne importy\n", "\n", "import matplotlib\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "%matplotlib inline" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "## 10.1. Perceptron" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "https://www.youtube.com/watch?v=cNxadbrN_aI" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Pierwszy perceptron liniowy\n", "\n", "* Frank Rosenblatt, 1957\n", "* aparat fotograficzny podłączony do 400 fotokomórek (rozdzielczość obrazu: 20 x 20)\n", "* wagi – potencjometry aktualizowane za pomocą silniczków" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Uczenie perceptronu\n", "\n", "Cykl uczenia perceptronu Rosenblatta:\n", "\n", "1. Sfotografuj planszę z kolejnym obiektem.\n", "1. Zaobserwuj, która lampka zapaliła się na wyjściu.\n", "1. Sprawdź, czy to jest właściwa lampka.\n", "1. Wyślij sygnał „nagrody” lub „kary”." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Funkcja aktywacji perceptronu\n", "\n", "Funkcja bipolarna:\n", "\n", "$$ g(z) = \\left\\{ \n", "\\begin{array}{rl}\n", "1 & \\textrm{gdy $z > \\theta_0$} \\\\\n", "-1 & \\textrm{wpp.}\n", "\\end{array}\n", "\\right. $$\n", "\n", "gdzie $z = \\theta_0x_0 + \\ldots + \\theta_nx_n$,
\n", "$\\theta_0$ to próg aktywacji,
\n", "$x_0 = 1$. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [], "source": [ "def bipolar_plot():\n", " matplotlib.rcParams.update({'font.size': 16})\n", "\n", " plt.figure(figsize=(8,5))\n", " x = [-1,-.23,1] \n", " y = [-1, -1, 1]\n", " plt.ylim(-1.2,1.2)\n", " plt.xlim(-1.2,1.2)\n", " plt.plot([-2,2],[1,1], color='black', ls=\"dashed\")\n", " plt.plot([-2,2],[-1,-1], color='black', ls=\"dashed\")\n", " plt.step(x, y, lw=3)\n", " ax = plt.gca()\n", " ax.spines['right'].set_color('none')\n", " ax.spines['top'].set_color('none')\n", " ax.xaxis.set_ticks_position('bottom')\n", " ax.spines['bottom'].set_position(('data',0))\n", " ax.yaxis.set_ticks_position('left')\n", " ax.spines['left'].set_position(('data',0))\n", "\n", " plt.annotate(r'$\\theta_0$',\n", " xy=(-.23,0), xycoords='data',\n", " xytext=(-50, +50), textcoords='offset points', fontsize=26,\n", " arrowprops=dict(arrowstyle=\"->\"))\n", "\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAEeCAYAAAAHLSWiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1yUZf7/8fcoZzEVT7mmwapYQsUqth5yKaxdUpHWFHTDxNrU1HbxlNpiWlp00KLdVkOtNA+PBH0Y5LElxbJsk3bZDAo1wbTNNHU7iKDA/fvDL/NrHA43h2EYeD0fj3nIXNd9DZ+5He4398x1X1gMwxAAAKhaC2cXAACAKyAwAQAwgcAEAMAEAhMAABMITAAATCAwAQAwwa2afq45AZwgIiJCu3btcnYZQHNlqaiRM0ygEfruu++cXQKAqxCYAACYQGACAGACgQkAgAkEJgAAJhCYAACYQGACAGACgQkAgAkEJgAAJhCYAACYQGACAGACgQkAgAkEJgAAJhCYAACYQGACAGACgQkAgAkEJlCFkydP6pFHHtHAgQPl4+Mji8WigoICU2PLysqUmJgof39/eXl56ZZbbtGWLVscWzAAhyEwgSocPXpUKSkpateunYYMGVKjsQsWLNCiRYs0ffp07dy5UwMGDNCYMWO0Y8cOB1ULwJEshmFU1V9lJ9DUlZWVqUWLK79Xrl69Wg899JDy8/Pl7+9f5bjTp0+rW7dumjdvnp544glr+9ChQ3XmzBl9+umnVY4PDQ1VVlZWnesHUCuWiho5wwSqUB6WNbV7925dunRJsbGxNu2xsbE6dOiQ8vPz66M8AA2IwAQcICcnR56enurZs6dNe1BQkCQpNzfXGWUBqIMq35K9/fbb7Trj4uIUFxen7777TqNHj7Yb8/DDDysmJkYnTpzQ+PHj7fpnzZqlyMhI5eXlafLkyXb9CQkJuvPOO5Wdna34+Hi7/qefflqDBg3Shx9+qMcee8yuPykpSSEhIcrIyNCSJUvs+pOTk9W7d2+9/fbbWrZsmV3/unXr1K1bN23atEkrVqyw69+8ebM6dOigNWvWaM2aNXb9O3bskI+Pj5YvX66UlBS7/szMTEnS0qVLtW3bNps+b29v7dy5U5K0ePFivfvuuzb97du3t04amT9/vg4cOGDTf91112n9+vWSpPj4eGVnZ9v0BwYGauXKlZKkSZMm6fDhwzb9ISEhSkpKknTlTOjkyZM2/QMHDlRiYqIk6d5779XZs2dt+ocOHaoFCxZIku6++25dvHjRpn/EiBGaPXu2JOn222/X1aKjozV16lQVFhZq2LBhdv11fe2FjJ2lf/zXXRculdr1NTbfrI1XlwlJzi4DaBAFzwyX1HiOe5mZmRW+JetWq2cHuKCdX1lUXNb4wxJA48SkHzQb/vO2O7sE0zjDRHNSfobZiHCGCZSrzQ9oTWbJvvHGG5owYYKOHDli8znmmjVrNHHiRB07dkwBAQGVjg/NWKisxncQAZo1Jv0ADhARESEPDw9t2LDBpn39+vUKDg6uMiwBNE6cYQLV2Lx5syTpk08+kSTt3LlTHTt2VMeOHRUWFiZJcnNz04QJE/Tqq69Kkjp16qQZM2YoMTFRrVu3Vt++fbVp0ybt2bNHaWlpznkiAOqEwASqMWbMGJv7U6dOlSSFhYVZZz2XlpaqtNR2QtFTTz0lX19fvfTSSzp16pR69+6tlJQURUZGNkjdAOoXgQlUo5qJcZVu07JlSyUkJCghIcERZQFoYHyGCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBiTr73//+p8WLFyskJEStW7eWn5+fwsPDtWPHDmeXBgD1xs3ZBcC1ZWZm6g9/+IO++eYbm/a9e/cqMzNTK1as0OTJk51UHQDUH84wUWuZmZkaNmyYvvnmG8XGxurgwYM6f/68Pv74Yw0cOFCGYWjmzJk6efKks0sFgDojMFErp0+f1tixY3Xx4kU999xzWrdunUJDQ9W2bVv1799faWlp8vX1VWFhoTZu3OjscgGgzghM1Mrs2bP17bffasSIEZozZ45df8eOHTV48GBJ0r59+xq6PACodwQmauyLL77Qhg0bZLFY9Nxzz1W6XceOHSVJx48fb6jSAMBhCEzUWHJyssrKynTnnXfqxhtvrHS7y5cv2/wLAK6MwESNlJWV6c0335Qk3XfffVVue+7cOUmSt7e3w+tylBMnTmj06NFq06aNrrnmGo0aNUpfffWVqbEWi6XCW3Z2toOrBuAIXFaCGsnOztapU6ckSXFxcYqLi6t2TLdu3RxclWMUFhYqPDxcnp6eWrt2rSwWixISEnTHHXfo008/VatWrap9jLi4OLvLagIDAx1VMgAHIjBRI7WZwNOrVy8HVOJ4q1at0rFjx5SXl6eePXtKkm6++Wb16tVLycnJmjlzZrWP0bVrVw0YMMDRpQJoALwlixr517/+JUn63e9+p4sXL1Z627p1q3VM3759nVVunaSnp2vAgAHWsJSkgIAADR48WGlpaU6sDIAzEJiokcOHD0uSunfvLi8vr0pvH330kXXMb37zG5vH2LRpk/r37y9vb2+1b99e0dHROnbsWIM+DzNycnIUHBxs1x4UFKTc3FxTj7FixQp5enrKx8dH4eHhev/99+u7TAANhMBEjZSv2uPn51fldtu3b5ck3Xjjjerevbu1/ZVXXtHYsWPl7u6uF198UTNnztSePXs0cOBA05NpGsq5c+fUrl07u3Y/Pz+dP3++2vGxsbFavny5MjIytHLlSp09e1bh4eHKzMx0QLUAHI3PMFEjFy9elCR5eXlVus0XX3yhzz77TJI0fvx4a/u5c+f06KOPKiQkRPv27ZO7u7sk6e6771b//v312GOPaf369Q6svuYsFotdm2EYpsauW7fO+vWQIUMUFRWl4OBgJSQkaP/+/Xbbr1y5UitXrpQknTlzppYVA3AUzjBRIx4eHpKkCxcuVLrN8uXLJV25nOTns2jfeust/fjjj/rzn/9sDUvpymec4eHh2rJliwoLCx1TeC20a9fOemnMz50/f77CM8/qtG7dWsOHD9fBgwcr7J80aZKysrKUlZVlXfQBQONBYKJGrr/+eklSXl5ehf35+fnWs6Tp06erS5cu1r6PP/5YkqxL5v3cbbfdpqKiIuuZaWMQFBSknJwcu/bc3Fz16dOnVo9pGEaFZ60AGj8CEzUSFhYmSXrnnXf03//+16bvwoULiomJUXFxsQIDA7Vw4UKb/q+//lqSdN1119k9bnlbY/rLJiNHjtRHH31kMyGpoKBAH3zwgUaOHFnjx/vhhx+0fft2/frXv67PMgE0EAITNTJx4kS5ubmpuLhYI0eOtP5Jr3feeUeDBw/WwYMH5efnp9TUVLsL+8vfbvX09LR73PLVgBrTW7IPPfSQ/P39FRUVpbS0NKWnpysqKkrdunWzWYzg+PHjcnNz05NPPmltW7p0qR566CFt3LhRmZmZWrt2rQYPHqxTp05pyZIlzng6AOqIST+okRtvvFELFy7UggUL9Mknn+jWW2+16f/lL3+pLVu26Oabb7Yb6+PjI0kqLi62Wy6vqKjIZpvGoFWrVtqzZ49mzJih8ePHyzAMDR06VElJSfL19bVuZxiGSktLVVZWZm3r3bu3tm7dqq1bt+r777/XNddco8GDB+vVV1+122cAXAOBiRpLSEhQYGCgkpKSlJOTI8MwFBgYqJiYGE2bNq3S0OvataukK2+7Xr36z4kTJyRV/HatM3Xv3l1btmypcht/f3+7mbORkZGKjIx0ZGkAGhiBiVqJjo5WdHR0jcbceuutSk5O1oEDB+wC84MPPpCXl1eFCwUAQGPAZ5hoMFFRUfL19dVLL72kkpISa/u///1vvfvuuxo1alSjeksWAH6OM0w0mPbt2+uZZ57R9OnTFRYWpvvvv19nz57Viy++qPbt2+upp55ydokAUCkCEw1q2rRp8vPz09KlSxUfHy9vb28NHTpUiYmJ8vf3d3Z5AFApAhMNbty4cRo3bpyzywCAGuEzTAAATCAwAQAwgcAEAMAEAhMAABMITDQKeXl5OnXqlLPLAIBKEZhoFN577z3dd999pv84MwA0NAITjcLEiRN1/vx5rV+/3tmlAECFCEw0Cm5ubkpOTtajjz6qc+fOObscALBDYKLR6N+/v0aPHq25c+c6uxQAsENgolFZsmSJdu7cqf379zu7FACwQWCiUWnTpo1efPFFTZkyRZcuXXJ2OQBgRWCi0Rk9erS6d++uF154wdmlAIAVgYlGx2Kx6O9//7uWLl2q/Px8Z5cDAJIITDRSAQEBmj17tqZNm8a1mQAaBQITjdasWbP01VdfafPmzc4uBQAITDRe7u7ueuWVVzRjxgx9//33zi4HQDNHYKJRu+222xQREaGEhARnlwKgmSMw0eg9++yzSk1N1cGDB51dCoBmjMBEo9e+fXs999xzmjx5skpKSpxdDoBmisCESxg/frzatm2rl19+2dmlAGimCEy4BIvFohUrVmjJkiU6efKks8sB0AwRmHAZvXv31rRp0/SnP/3J2aUAaIYITLiU+fPn67PPPtPbb7/t7FIANDMEJlyKl5eXli9frunTp+unn35ydjkAmhECEy7nzjvv1JAhQ/TEE084uxQAzQiBCZe0bNkyrV27Vv/5z3+cXQqAZoLAhEvq3LmzlixZoilTpqisrMzZ5QBoBghMuKw//vGPatGihVauXOnsUgA0AwQmXFaLFi30yiuvaMGCBTp16pSzywHQxBGYcGk33XSTHnjgAc2cOdPZpQBo4ghMuLzHH39cBw4c0DvvvGPXt3r1amVkZDihKgBNDYEJl9eqVSu9/PLLmjp1qi5evGjTl5+fr48++shJlQFoSghMNAnDhw9XSEiInn76aZv2zp078/kmgHpBYKLJeOmll7RixQp9/vnn1rZrr71W3377rROrAtBUEJhoMrp27arHH39cDz/8sAzDkHQlMDnDBFAfCEy4rLNnz+rw4cM2bdOmTdNPP/2kN954QxJvyQKoPwQmXFZ+fr5uu+02zZo1Sz/++KMkqWXLlkpOTtbcuXN19uxZ3pIFUG8ITLis0NBQffbZZzp79qz69OmjLVu2yDAM9evXTzExMXr00Ud1zTXX6PLlyyosLHR2uQBcHIEJl9apUyetWbNG69ev14IFCzRixAjl5+dr8eLFeuedd7R//3517tyZs0wAdUZgokkICwtTdna2brvtNoWGhurll1/W888/r8mTJ9fpc8wTJ05o9OjRatOmja655hqNGjVKX331lamxRUVFmjNnjrp06SJvb28NHDhQ7733Xq3qAOB8BCaaDA8PD82fP19ZWVn64IMP9OSTT6pNmzb64YcfanWGWVhYqPDwcH3xxRdau3at1q1bpyNHjuiOO+7QhQsXqh3/4IMPatWqVXryySe1bds2denSRb/73e+UnZ1dm6cHwMncnF0AUN8CAgK0bds2bd26VY888oi++eYbHTp0SFLfGj3OqlWrdOzYMeXl5alnz56SpJtvvlm9evVScnJylevX/uc//9HGjRv12muvaeLEiZKunAUHBQXp8ccfV3p6eq2fHwDn4AwTTZLFYtGoUaOUl5en4cOHy8vLq8aPkZ6ergEDBljDUroSxoMHD1ZaWlq1Y93d3RUTE2Ntc3Nz09ixY7V7924VFxfXuB4AzkVgoknz9fXV22+/rTlz5tR4bE5OjoKDg+3ag4KClJubW+3YgIAA+fj42I29dOmSjh49WuN6ADgXgQlU4ty5c2rXrp1du5+fn86fP1/rseX9AFyLpXwJsYpEREQY3333XQOWU70zZ86oY8eOzi6jWWkq+/zQ199bv76pa5tqt//Xv/6lzp07q2vXrjbtX3/9tU6dOqV+/fpVOvbw4cMqKyvTDTfcYNP+ww8/6MiRI+rdu7d8fX1t+s6cOaPyn7fi4mKFhIRUWyPqT1N5nbuSxrrPP/nkk92GYURc3V5lYEqqstMZQkNDlZWV5ewympWmss/95223fl3wzPBqt+/cubPuueceJScn27RPnTpVqampOnPmTKVjY2JilJ2drby8PJv2lJQUxcTE6LPPPlNQUFCl41u1amVqJi7qT1N5nbuSRrzPLRU18pYsUImgoCDl5OTYtefm5qpPnz7Vjs3Pz7dbYSg3N1ceHh42E4kAuAYCE6jEyJEj9dFHH+nYsWPWtoKCAn3wwQcaOXJktWMvX76s1NRUa1tJSYk2bdqk3/72t/L09HRY3QAcw+UCc9KkSc4uodlprvv8oYcekr+/v6KiopSWlqb09HRFRUWpW7dumjx5snW748ePy83NTU8++aS1LSQkRDExMYqPj9fq1av17rvvauzYscrPz9cTTzxR7ffu0KGDQ54TKtdcX+fO5Gr7nMBEtZrrPm/VqpX27NmjwMBAjR8/Xvfdd58CAgK0Z88emwk7hmGotLRUZWVlNuNff/11TZw4UQkJCRo+fLhOnDihXbt2qW/f6hdQaIwTIZq65vo6dyZX2+cuN+kHqK2aTvpxpkY8GQJoDlxz0s8LL7ygyMhIdenSRRaLRYsWLarR+P3792vQoEHy9vbWtddeq5kzZ+rixYuOKbaJKCsrU2Jiovz9/eXl5aVbbrlFW7ZsMTU2Li5OFovF7hYfH+/gql0Di7k3vLrs84peyxaLhfWAq3Dy5Ek98sgjGjhwoHx8fGSxWFRQUGBqbF2OPQ2h0QfmqlWrdPr0ad1zzz01Hvvpp5/qrrvuUqdOnbRt2zYtWbJEr7/+uuLi4uq/0CZkwYIFWrRokaZPn66dO3dqwIABGjNmjHbs2GFqfMeOHXXgwAGb24wZMxxcdePHYu4Nr677XLryS+DVr+fAwEAHV+66jh49qpSUFLVr105Dhgyp0di6HnsczjCMqm5OV1paahiGYVy+fNmQZCxcuND02Hvuucfo2bOncenSJWvb2rVrDUnGJ598Ut+lNgnffvut4eHhYTz++OM27eHh4cZNN91U7fgJEyYYXbt2dVR5dXL93G3WmzMkJSUZLVq0MI4cOWJtO3bsmNGyZUtj2bJlNtv269fP5n52drYhyXjttdesbZcvXzYCAwONyMhIxxbuwmqyzysiyfjLX/7iyBKbnPJjtmEYxqpVqwxJRn5+frXj6nrsqWcVZmKjP8Ns0aJ2JV6+fFm7du1SdHS03N3dre3R0dHy8PCodvHs5mr37t26dOmSYmNjbdpjY2N16NAh5efnO6ky18di7g2vLvsctVPbY7YrHHsafWDW1pdffqmioiK7xbO9vLzUo0ePahfPbq5ycnLk6elpd2F9+ao0Zvbb6dOn1aFDB7m5uSkwMFDPPvusSktLHVKvK2Ex94ZXl31ebsWKFfL09JSPj4/Cw8P1/vvv13eZUP0cexytyf49zPLFrStbAJvFryt27tw5tW3bVhaL7SQxs4uGh4SEqF+/fgoKClJRUZG2bt2q+fPn68iRI1q9erXD6nYFLObe8Oqyz6UrZzcjRozQL37xCx0/flzPP/+8wsPD9Y9//EO33367Aypuvup67GkIDRqYGRkZuuuuu6rdLiwsTJmZmXX6Xsb/XS5z9c7/eV9zUNN9bhhGnfbZ1bNhhw0bJl9fXyUlJWnu3Lnq1auXucKbqNru27r+vzRnddlv69ats349ZMgQRUVFKTg4WAkJCdq/f3+91QjXeI03aGAOGjRIn3/+ebXbXf22U21U9VvJ+fPnq1z4uimp6T4v/8376hdv+W/j5fu1JsaNG6ekpCRlZWU168Bs165dpa/His6Cfs7Pz6/CSyHq8v/SHNRln1ekdevWGj58uF599dX6KA8/44hjT31r0MD08fGx+3NHjtKjRw95enraLZ5dVFSkY8eOacyYMQ1Sh7PVdJ8HBQWpuLhYX375pc1nCeWfH1S36HhFqjrbb07qupj71q1bVVhYaPMLJYu5V60u+7wylZ0JoW4cceypb0120o+Hh4ciIiKUkpKikpISa/vmzZtVXFxc7eLZzVVERIQ8PDy0YcMGm/b169crODhYAQEBNX7MjRs3ymKxqH///vVVpktiMfeGV5d9XpEffvhB27dv169//ev6LBNyzLGn3lV2vYnRSK7DPHjwoJGammps2rTJkGSMGTPGSE1NNVJTU40LFy5Yt3vggQeMli1b2oz997//bXh5eRm///3vjYyMDGP16tVGu3btjNGjRzf003Apc+fONTw9PY1ly5YZe/fuNaZMmWJYLBYjPT3dZrvw8HCjR48e1vsFBQXGkCFDjL///e/G7t27jfT0dGPixImGxWIxpkyZ0tBPw46zr8P86aefjB49ehjBwcHGW2+9ZaSlpRk333yzERAQYPz444/W7QoKCgxJxhNPPGEzPiYmxmjbtq2xatUqIyMjw7j33nsNT09PrimuQk32ecuWLW32+fPPP2/88Y9/NDZs2GDs3bvXWLNmjREcHGy4u7sb7733njOejssoP0ZPmTLFkGQsX77cSE1NNTIzM63btGzZ0njggQdsxpk99jSACjOx0QfmhAkTDF1Z09bu9vOLYcu3u9q+ffuMAQMGGJ6enkanTp2MP//5zzZBC3slJSXG4sWLje7duxseHh7GTTfdZKSmptptFxYWZlx//fXW+2fPnjWioqKM7t27G56enoaXl5fxq1/9yvjb3/5mczGzszg7MA3DMI4fP26MGjXKaN26teHr62tERUXZXdSdn59f4SIdhYWFxowZM4zOnTsbnp6exq233mrs3bu3wWp3VbXd5+np6cagQYOM9u3bG25uboafn58RGRlp/POf/2zYJ+CCKjtmh4WF2WwzYcIEm3Fmjz0NoMJMZPF1NBssvg7AJNdcfB0AgMaAwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMoBJlZWVKTEyUv7+/vLy8dMstt2jLli2mxsbFxclisdjd4uPjHVw1AEdxc3YBQGO1YMECLV26VE899ZT69eunN998U2PGjNG2bds0bNiwasd37NhR6enpNm1dunRxVLkAHIzABCpw+vRpLV26VPPmzdPs2bMlSXfccYeOHj2qefPmmQpMDw8PDRgwwNGlAmggvCULVGD37t26dOmSYmNjbdpjY2N16NAh5efnO6kyAM5CYAIVyMnJkaenp3r27GnTHhQUJEnKzc2t9jFOnz6tDh06yM3NTYGBgXr22WdVWlrqkHoBOB5vyQIVOHfunNq2bSuLxWLT7ufnZ+2vSkhIiPr166egoCAVFRVp69atmj9/vo4cOaLVq1c7rG4AjkNgolnIyMiwuX91EJYLCwtTZmamDMOocBvDMEx9v6tnww4bNky+vr5KSkrS3Llz1atXL7sxK1eu1MqVKyVJZ86cMfV9ADQcAhPNwqBBg6SMvdb7n3/+eYXb+fj4SLpyJnn+/Hm74Dx//ry1v6bGjRunpKQkZWVlVRiYkyZN0qRJkyRJoaGhNX58AI5FYKJZKA/CcjfccEOV2wcFBam4uFhffvmlzeeY5Z9d9unTp8Y1lJ+dVnZ2C6BxY9IPUIGIiAh5eHhow4YNNu3r169XcHCwAgICavyYGzdulMViUf/+/eurTAANiDNMoAKdOnXSjBkzlJiYqNatW6tv377atGmT9uzZo7S0NJtthw4dquPHj+vo0aOSpOPHj2v8+PEaO3asevbsqeLiYm3dulVr1qzR5MmT1aNHD2c8JQB1RGAClXjqqafk6+url156SadOnVLv3r2VkpKiyMhIm+1KS0tVUlJivd+6dWv5+fnp2Wef1bfffiuLxaIbb7xRf/3rXzV16tSGfhoA6omlmll/5qYEAi7Af95269cFzwx3YiXVCw0NVVZWlrPLAJqrCica8BkmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCVTihRdeUGRkpLp06SKLxaJFixbVaPz+/fs1aNAgeXt769prr9XMmTN18eJFxxQLwOEITKASq1at0unTp3XPPffUeOynn36qu+66S506ddK2bdu0ZMkSvf7664qLi6v/QgE0CDdnFwA0Vjk5OWrRooVKSkr0yiuv1GjswoULdd111yk1NVXu7u6SJA8PD02YMEFz585V3759HVEyAAfiDBOoRIsWtfvxuHz5snbt2qXo6GhrWEpSdHS0PDw8lJaWVl8lAmhABCZQz7788ksVFRUpODjYpt3Ly0s9evRQbm6ukyoDUBdVviV7++2327XFxcUpLi5O3333nUaPHm3X//DDDysmJkYnTpzQ+PHj7fpnzZqlyMhI5eXlafLkyXb9CQkJuvPOO5Wdna34+HibvoIBc6p7PoApFb22K3vtGYYhSTpx4oQk6cMPP9Rjjz1mNz4pKUkhISHavXu3JCkxMVHJycnW/uTkZPn5+enw4cMVfv9169apW7du2rRpk/Ly8uy22bx5szp06KA1a9ZozZo1duN37NghHx8fLV++XPMRzZkAAALqSURBVCkpKXb9mZmZkqSlS5dq27ZtNn3e3t7auXOnJGnx4sV69913bfrbt2+vLVu2SJLmz5+vAwcO2PRfd911Wr9+vSQpPj5e2dnZNv2BgYFauXKlJGnSpEk6fPiwTX9ISIiSkpIkSbGxsTp58qRN/8CBA5WYmChJuvfee3X27Fmb/qFDh2rBggWSpLvvvttuctWIESM0e/ZsSRX/30dHR2vq1KkqLCzUsGHD7PqdedyTpKefflqDBg2q9rWXkZGhJUuW2PUnJyerd+/eevvtt7Vs2TK7/p+/9lasWGHX39xee+X1Xo3PMNHslBUXat++fXbt+/btU1hYmPWHp7bKA7amfRs2bNDmzZt17tw5Xb58uU41AKh/lqp+gCVV2dnQ/Odtd3YJcHHebhbFhvjp3uC2Ffb7+Pioe/fuNm0lJSVyd3fXwoULTV1a8vnnn6tPnz7auHGjxo0bZ9PXp08fBQUFKTU1tcrHCA0NVVZWVrXfC4BDWCpqdKkzzIJnhju7BKBaPXr0kKenp3Jycmzai4qKdOzYMY0ZM8ZJlQGoCyb9APXMw8NDERERSklJUUlJibV98+bNKi4u1siRI51YHYDacqkzTKAhZWVlqaCgQGVlZZKk3Nxcbd68WZI0bNgw+fj4SJIefPBBrV271iYcFy1apIEDByo6OlrTpk1TQUGB5syZo9GjR6tfv34N/2QA1BmBCVTi5Zdf1tq1a633U1NTrZ895ufny9/fX5JUWlqq0tJSm7Hls2Xnzp2r4cOHq02bNrr//vv19NNPN1j9AOqXS036AZoLJv0ATlXhpB8+wwQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADChuj8gDcAJLBbLLsMwIpxdB4D/j8AEAMAE3pIFAMAEAhMAABMITAAATCAwAQAwgcAEAMCE/weovYTBIgUU4wAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "bipolar_plot()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Schemat perceptronu\n", "\n", "\"Rys." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Zasada działania perceptronu\n", "\n", "1. Ustal wartości początkowe $\\theta$ (wektor 0 lub liczby losowe blisko 0).\n", "1. Dla każdego przykładu $(x^{(i)}, y^{(i)})$, dla $i=1,\\ldots,m$\n", " * Oblicz wartość wyjścia $o^{(i)} = g(\\theta^{T}x^{(i)}) = g(\\sum_{j=0}^{n} \\theta_jx_j^{(i)})$\n", " * Wykonaj aktualizację wag (tzw. *perceptron rule*):\n", " $$ \\theta := \\theta + \\Delta \\theta $$\n", " $$ \\Delta \\theta = \\alpha(y^{(i)}-o^{(i)})x^{(i)} $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "$$\\theta_j := \\theta_j + \\Delta \\theta_j $$\n", "\n", "Jeżeli przykład został sklasyfikowany **poprawnie**:\n", "\n", "* $y^{(i)}=1$ oraz $o^{(i)}=1$ : $$\\Delta\\theta_j = \\alpha(1 - 1)x_j^{(i)} = 0$$\n", "* $y^{(i)}=-1$ oraz $o^{(i)}=-1$ : $$\\Delta\\theta_j = \\alpha(-1 - -1)x_j^{(i)} = 0$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Czyli: jeżeli trafiłeś, to nic nie zmieniaj." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "$$\\theta_j := \\theta_j + \\Delta \\theta_j $$\n", "\n", "Jeżeli przykład został sklasyfikowany **niepoprawnie**:\n", "\n", "* $y^{(i)}=1$ oraz $o^{(i)}=-1$ : $$\\Delta\\theta_j = \\alpha(1 - -1)x_j^{(i)} = 2 \\alpha x_j^{(i)}$$\n", "* $y^{(i)}=-1$ oraz $o^{(i)}=1$ : $$\\Delta\\theta_j = \\alpha(-1 - 1)x_j^{(i)} = -2 \\alpha x_j^{(i)}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Czyli: przesuń wagi w odpowiednią stronę." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Zalety perceptronu\n", "\n", "* intuicyjny i prosty\n", "* łatwy w implementacji\n", "* jeżeli dane można liniowo oddzielić, algorytm jest zbieżny w skończonym czasie" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Wady perceptronu\n", "\n", "* jeżeli danych nie można oddzielić liniowo, algorytm nie jest zbieżny" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [], "source": [ "def plot_perceptron():\n", " plt.figure(figsize=(12,3))\n", "\n", " plt.subplot(131)\n", " plt.ylim(-0.2,1.2)\n", " plt.xlim(-0.2,1.2)\n", "\n", " plt.title('AND')\n", " plt.plot([1,0,0], [0,1,0], 'ro', markersize=10)\n", " plt.plot([1], [1], 'go', markersize=10)\n", "\n", " ax = plt.gca()\n", " ax.spines['right'].set_color('none')\n", " ax.spines['top'].set_color('none')\n", " ax.xaxis.set_ticks_position('none')\n", " ax.spines['bottom'].set_position(('data',0))\n", " ax.yaxis.set_ticks_position('none')\n", " ax.spines['left'].set_position(('data',0))\n", "\n", " plt.xticks(np.arange(0, 2, 1.0))\n", " plt.yticks(np.arange(0, 2, 1.0))\n", "\n", "\n", " plt.subplot(132)\n", " plt.ylim(-0.2,1.2)\n", " plt.xlim(-0.2,1.2)\n", "\n", " plt.plot([1,0,1], [0,1,1], 'go', markersize=10)\n", " plt.plot([0], [0], 'ro', markersize=10)\n", "\n", " ax = plt.gca()\n", " ax.spines['right'].set_color('none')\n", " ax.spines['top'].set_color('none')\n", " ax.xaxis.set_ticks_position('none')\n", " ax.spines['bottom'].set_position(('data',0))\n", " ax.yaxis.set_ticks_position('none')\n", " ax.spines['left'].set_position(('data',0))\n", "\n", " plt.title('OR')\n", " plt.xticks(np.arange(0, 2, 1.0))\n", " plt.yticks(np.arange(0, 2, 1.0))\n", "\n", "\n", " plt.subplot(133)\n", " plt.ylim(-0.2,1.2)\n", " plt.xlim(-0.2,1.2)\n", "\n", " plt.title('XOR')\n", " plt.plot([1,0], [0,1], 'go', markersize=10)\n", " plt.plot([0,1], [0,1], 'ro', markersize=10)\n", "\n", " ax = plt.gca()\n", " ax.spines['right'].set_color('none')\n", " ax.spines['top'].set_color('none')\n", " ax.xaxis.set_ticks_position('none')\n", " ax.spines['bottom'].set_position(('data',0))\n", " ax.yaxis.set_ticks_position('none')\n", " ax.spines['left'].set_position(('data',0))\n", "\n", " plt.xticks(np.arange(0, 2, 1.0))\n", " plt.yticks(np.arange(0, 2, 1.0))\n", "\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqwAAADGCAYAAAAXMtIlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAWVUlEQVR4nO3df5DUd33H8dd7cRO5XJYcaUQn1YEh0Qop2IYqmBACiS1YQ1J6RB1Bo2PUOUwnV5z6MzVJUepUClYnJiZq6FG1sM4kqYBOmhFsMFQ5M57lgt4dXEhik4YcJnCXHyu8+8f3e7C3t7e3y+339nN3z8fMdzb3/X73u5/95vPm+9rvfr+fNXcXAAAAEKpUrRsAAAAAlEJgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWEeJmW03Mzez/ymxjsfTz4ZYfmW8fFPB/HvznutmljOzHjP7VbzsXWY2qdrvCZhozOwdZpY1s6fM7GUzO2JmPzazj5jZq4qs311QmyfM7Fkz22lmy2rxHoDxwMw+GdfU14ZY/iYz6zOzg2Z2TsGyt5lZi5k9bmYvmdlRM/upma01s8lDbO/hIrXcY2a7zOy9SbxHDGT8cEDyzOx1kp5Q9AHBJL3V3X9eZL38/xmN7v79guVXSvqxpK+4+8158++V9AFJd0l6On6djKQ3S1ooabKkn0l6t7t3V+t9ARNFHEbvkvQhSccl/UDSIUnnS1om6fWS9kl6l7s/k/e8bkmvlfSP8ayzFdXlNYrq9APu/q+j8y6A8SM+CbNH0lslLXH3XQXLHpb0tiLLvijp05JelrRT0q8l1Uv6c0kXS+qQ9E537yx4vYclXSZpvaRXJKUlXSTpOklnSfqsu38xgbeKfu7OlPAk6VOSXNKX48evD7GeS/qtpBclPSZpUsHyK+N1NhXMvzee/5Yi2zxfUku8/ICk+lrvDyamsTZJ2hDX0COSXluw7GxJX89b/qq8Zd2Sfldke9fH6x+u9XtjYhqrk6IPfy9JOijpnLz5n4zr66sF6zfnHQtnFixLSbo1Xt4h6dyC5Q/Hy+oL5l8u6YSkXkmTa71PxvPEJQGj4wZJz0v6nKTfSHqPmb16iHX/T9LXJP1R/LwRcffnJL1f0oOS3iTp4yPdJjCRmNkbJd0s6TlJy9396fzl7v6ypCZJP5E0X9G3HcPZpuhM7evN7ILqthiYGNz9MUl/L2mGpC9JkpnNknSbpC5FJ4sUzz9f0j8oCrjXuHtXwbZOuvutkr6r6Mzp35bZhocldUqqU3TcRkIIrAkzs8sUBcWsu78kaYuk8yStKPG09YoC7udLBNuyefQxsP+riutHuj1ggvmAon8rv+HuzxZboaDGPljmdi1+zI2secCEtkHSf0tqMrOrJW1W9BX9h9y9N2+96yWdI2mbu3eU2N4X4scPnUFbqOUEEViT13/waokftyj6WmHIg5q790j6J0XXxa2pUjt+qqiY5ha7OQTAkN4ePz40zHo/kfR7SX9Wxk2O71F08HzM3X83wvYBE5a7n1D0beTLiq4tnyfpX9z9JwWrllXH7r5f0jOS3hDff1KSmS1SdO3rEUWXEiAhBJcExXcmXi/psKKDmdz9kJn9VNISM3uDux8e4umbFH19/2kzu9vdXxhJW9z9FTPrkTRN0lRFlx4AGN5r48cnS63k7i+a2XOKaux8na6xV5vZrfF/ny1plqKbrl5U9T6QAhOWux8wsy2SPqwobH6myGpl1XHeOtMkvU7S/xYs+4yZvaIoP10s6a8knZT08fjyICSEwJqsRknnSvpa/JVhvxZFdxveIOn2Yk90914zW6foetZPKLpOZ6Rs+FUAjEB/jeXX+9mSPl+w3ouSlrn77lFpFTCOmdkMSf1DS01TdDb1P0eyyfix2DBKny74+4Sk97r7thG8HsrAJQHJ6v/af0vB/K2KhsW4wcxKhchvKLr7sdnMXjOShpjZ2YrOrJ6Q1DOSbQETTP9NVn9YaqX4evOpimo7v8aed3dzd5M0RdGB9aSkrWZWcpsASouPod9UdInNzYpuqrrbzOoLVi2rjmMXFjwn37lxLddLeqeimzG/bWZzKm07KkNgTYiZzZR0Rfzn/vwBhxUdzM5SdGfjlUNtw91zis7M1CsaYWAk3q7ojPov3f33I9wWMJE8Ej9eNcx6VyiqsZ/H19UN4u4vuPv3JH1E0msUDYcF4Mw1SVos6R53/4qiY+Z0nR77uF9ZdRyPMjBN0ZBzhZcDnOLuve6+U9JKRWH528OcgMIIEViTc4OirxV+rOjTX+F0f7zecHcUf0dSm6SPKirCisVF1P81xr+fyTaACWyzoq8Gb4yHxhmkoMa+PdwG3f07kvZKepeZLaxWQ4GJJL4U4EuKfphnbTx7g6IfymmKb4jqt1XRpTgr4xNKQ+m//vVb5bQhvrnr+5L+VNHNlEgIgTUBZpZSNBTOCUnvc/cPF06KbsZ6TtJfm1lmqG25+0lJn1V0Rrbis6xmNlXRAfcdin7R446K3xAwgbn7ryV9VdIfSLrfzKblLzezs+LlVyoaXqfcX666LX68tSoNBSaQ+EPitxSd3byx/8bk+NuNDyq6NOebZlYXzz+i6OzrqyX9Rxx287eXMrPPSXqfonFV/7mC5tym6EPtLfHxHwngpqtkXK1oSKrtQ32lEN+1/2+S/kbSuyXdPdTG3P0H8c/CXT7M637MzJ5WdGY3o2gQ40Ua+NOsxyt9MwD0CUXjJ79fUoeZFf406xsktUq6Nr6UZ1ju/kMz+5miEUMujwcgB1CeNYo+JN7j7j/KX+Du7WZ2u6IxVdfp9I8AfFnRB8+/k9RuZjsU/ZhP/0+zvlFRWF3m7sfKbYi7/8rM7lM0YsD1kr43gveFIdjAm9dRDWb2XUVfDax092yJ9d4i6VFJe919QXx96y/d/S1F1r1c0n/Ff37F3W/OW3avBv66zglJxxQNzdEqKStpR3y2FsAZMrO/UHR5znxFB77jii7Z+a6kbxWGVTPrlnSeu583xPb+UtHYkQ+5+9UJNh0YN+Kzo79SdD/IJcWGfYzHG98r6U8kXe7uj+QtW6Bo2MiFiq5XfVHRz7VmJd3h7n1FtvewotF9zi124ifveN4u6Y853lYfgRUAAABB41oLAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0IYLrD6a09KlS0f19ZiYzmAK3ajuD2qWKfApdKO6P6hXpjEwDSmoM6xHjhypdRMAVICaBcYO6hVjWVCBFQAAAChEYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKBVPbA++eSTuummm7RgwQLV1dXJzNTd3V3tlwFQBdQrMHZQr5jIqh5YOzs7tXXrVjU0NGjhwoXDP6GrS2pqkjIZqbU1emxqiuYDRXT1dKlpe5My6zNK3ZZSZn1GTdub1NVDn6lUxfWqgfu/9bet7H+URL1WD/WKUZGfy1KpYHKZuXup5SUXFnPy5EmlUlEOvueee3TjjTfq0KFDmj59+uCVd+6UGhulXE7K5TRP0j5JSqejKZuVli2rtAkYx3Z27FTjtkblTuSUO5k7NT+dSis9Ka3syqyWXZxon7EkN14FFdVsRfWqIvv/LkkfHdX9jzGEeh0W9YqwFOSyU0Yvlw1Zs1U/w9pfTMPq6op2Sl/fwJ0iRX/39UXLOdOKWFdPlxq3Naov1zfg4CdJuZM59eX61LitkTMHFSi7XsX+R2XoL9VHvSJRgeey2t10tWHD4B1SKJeTNm4cnfYgeBse2aDcidJ9Jncip4176TNJYP+jEvSX2mL/o2KB57LaBdYtW8rbMS0to9MeBG9L25ZBZwoK5U7m1NJGn0kC+x+VoL/UFvsfFQs8l9UusB4/Xt31MO4df6W8vlDueqgM+x+VoL/UFvsfFQs8l9UusNbXV3c9jHv1Z5XXF8pdD5Vh/6MS9JfaYv+jYoHnstoF1lWrojvOSkmnpdWrR6c9CN6qOauUTpXuM+lUWqvn0GeSwP5HJegvtcX+R8UCz2W1C6xr15a3Y5qbR6c9CN7aBWuVnjTMP8CT0mqeT59JAvsflaC/1Bb7HxULPJclEliz2ayy2axaW1slSTt37lQ2m9Xu3btPrzRzZjSeV13d4B2UTkfzs9loPUDSzKkzlV2ZVV26btCZg3Qqrbp0nbIrs5o5lT5TibLqVex/VIb+kgzqFYkJPJdV/YcDJMms+LivixYt0q5duwbO7OqKhkhoadG8F17QvkwmOt3c3ExYRVFdPV3auHejWtpadPyV46o/q16r56xW8/zm0fjHd1wNRC5VWK8auP9f+OoLytyUGc39jzGGei2JekV48nKZjh+PrlkdvVw2ZM0mEljP1Lx587Rv377RfEmgUuPuADgS1CwCR73moV4xBozeL10BAAAA1URgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0AisAAAACBqBFQAAAEEjsAIAACBoBFYAAAAEjcAKAACAoBFYAQAAEDQCKwAAAIJGYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0AisAAAACBqBFQAAAEEjsAIAACBoBFYAAAAEjcAKAACAoBFYAQAAEDQCKwAAAIJGYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0AisAAAACBqBFQAAAEEjsAIAACBoBFYAAAAEjcAKAACAoBFYAQAAEDQCKwAAAIJGYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0AisAAAACBqBFQAAAEEjsAIAACBoBFYAAAAEjcAKAACAoBFYAQAAEDQCKwAAAIJGYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABC0qgfWJ554Qo2NjZoyZYoymYxWrFihw4cPV/tlAFQJNQuMHdQrJqqqBta+vj4tWbJEBw4c0ObNm9XS0qKOjg4tXrxYvb29xZ/U1SU1NUmZjNTaGj02NUXzgWLy+0wqRZ8ZAWoWSevq6VLT9iZl1meUui2lzPqMmrY3qauH/lIp6hWjIdiadfdSU0U2bdrkqVTKOzo6Ts07ePCgT5o0yTds2DD4CTt2uNfVuafT7pJfKrlL0d91ddFyIF9Bn/HR7zPD1Uytp4pQs0jSjt/s8Lov1Hn69rTrVp2a0renve4Ldb7jN9RrJahXJC3kmjV3L5lnKwm/V111lV566SXt2bNnwPxFixZJknbv3n16ZleXNGeO1Nd3atY8Sfvyn1hXJ7W1STNnVtIMjFdF+swgyfcZS2rDVULNIghdPV2ac+cc9eWGrte6dJ3aPtammVOp13JQr0hS6DVb1UsC9u/fr0suuWTQ/NmzZ6u9vX3gzA0bpFyu9AZzOWnjxiq2EGMafabqqFkkZcMjG5Q7Ubq/5E7ktHEv/aVc1CuSFHrNVjWw9vT0qKGhYdD8qVOn6ujRowNnbtlSXjG1tFSxhRjT6DNVR80iKVvatih3cpiD38mcWtroL+WiXpGk0Gu25CUBS5cu9SNHjpS9sV/84heaNm2aLrzwwgHzn3rqKT399NO69NJLT89sbR30/MckvbnYhvOfh4mrSJ8ZUkJ9prW19UfuvjSRjVcBNYtQtP62zHo16dLXUa/loF6RpNBrtqrXsE6bNk3XXXed7rrrrgHzm5qatG3bNj377LOnZ2Yy0rFjA9YbdH1N/3rPP19JMzBeFekzQ66XXJ8ZV9fEUbNISmZ9RsdeGb5eM2dn9PynqNdyUK9IUug1W9VLAmbPnq39+/cPmt/e3q5Zs2YNnLlqlZROl95gOi2tXl3FFmJMo89UHTWLpKyas0rpVOn+kk6ltXoO/aVc1CuSFHrNVjWwLl++XHv37tXBgwdPzevu7taePXu0fPnygSuvXVteMTU3V7OJGMvoM1VHzSIpaxesVXrSMAe/SWk1z6e/lIt6RZJCr9mqXhLQ29uruXPnavLkyVq3bp3MTLfccouOHTumtrY21dfXD3zCzp1SY2N04Xcud/rrinQ6mrJZadmyCt8SxrWCPnPK6PWZcfUVIzWLJO3s2KnGbY3KncgNuJkjnUorPSmt7Mqsll1MvZaLekXSgq7ZUoO0nsmIr48//rivWLHCzz33XK+vr/drr73WDx06NPQTOjvd16xxz2SiQY0zmejvzs4zeXlMBHl9xlOp0e4ztR5onJrFmNL5XKev2b7GM+sznrot5Zn1GV+zfY13Pke9nskbol6RtFBrtqpnWEdq3rx52rdv0CXhQEjG1RmbkaJmETjqNQ/1ijFgdG66OhNPPPGEGhsbNWXKFD366KNasWKFDh8+XOtmIWBPPvmkbrrpJi1YsEB1dXUyM3V3d9e6WRMGNYtKUK+1Rb2iEiHXa00Da19fn5YsWaIDBw5o8+bNmjFjhjo6OrR48WL19vbWsmkIWGdnp7Zu3aqGhgYtXLiw1s2ZUKhZVIp6rR3qFZUKuV5rGljvvvtuHTx4UPfdd5+uu+46nXfeeXrggQf0+OOPDxpnDuh3xRVX6JlnntGOHTu0cuXKWjdnQqFmUSnqtXaoV1Qq5HqtaWB94IEHNH/+fF100UWn5s2YMUOXXXaZ7r///hq2DCFLpWp+JcuERc2iUtRr7VCvqFTI9VrTlu3fv1+XXHLJoPmzZ89We3t7DVoEoBRqFhg7qFeMJzUNrD09PWpoaBg0f+rUqTp69GgNWgSgFGoWGDuoV4wnNT/3azZ4BINhhtoCUEPULDB2UK8YL2oaWBsaGtTT0zNo/tGjR4t+KgRQW9QsMHZQrxhPahpYZ8+erf379w+a397erlmzZtWgRQBKoWaBsYN6xXhS08C6fPly7d27VwcPHjw1r7u7W3v27NHy5ctr2DIAxVCzwNhBvWI8qelPs/b29mru3LmaPHmy1q1bp7Vr1+qcc87RsWPH1NbWpvr6+iRfHmNYNpuVJD300EO68847dccdd+iCCy7QBRdcoEWLFiX50hP6px6pWZwJ6nVI1CuCU8N6lUrUbE0DqyQdPnxYzc3NevDBB9Xb26trrrlGmzZt0vTp05N+aYxhxW4kkKRFixZp165dib50khuvAmoWwaFeh0S9Ijg1rFcp5MCab968edq3b99oviRQqQl/AMxHzSJw1Gse6hVjwJA1W/NhrQAAAIBSCKwAAAAIGoEVAAAAQRvuGtZRZWY/dPeltW4HgPJQs8DYQb1iLAsqsAIAAACFuCQAAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQtP8HLYi+hk3C0xEAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plot_perceptron()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Perceptron – funkcje aktywacji\n", "\n", "Zamiast funkcji bipolarnej możemy zastosować funkcję sigmoidalną jako funkcję aktywacji." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [], "source": [ "def plot_activation_functions():\n", " plt.figure(figsize=(16,7))\n", " plt.subplot(121)\n", " x = [-2,-.23,2] \n", " y = [-1, -1, 1]\n", " plt.ylim(-1.2,1.2)\n", " plt.xlim(-2.2,2.2)\n", " plt.plot([-2,2],[1,1], color='black', ls=\"dashed\")\n", " plt.plot([-2,2],[-1,-1], color='black', ls=\"dashed\")\n", " plt.step(x, y, lw=3)\n", " ax = plt.gca()\n", " ax.spines['right'].set_color('none')\n", " ax.spines['top'].set_color('none')\n", " ax.xaxis.set_ticks_position('bottom')\n", " ax.spines['bottom'].set_position(('data',0))\n", " ax.yaxis.set_ticks_position('left')\n", " ax.spines['left'].set_position(('data',0))\n", "\n", " plt.annotate(r'$\\theta_0$',\n", " xy=(-.23,0), xycoords='data',\n", " xytext=(-50, +50), textcoords='offset points', fontsize=26,\n", " arrowprops=dict(arrowstyle=\"->\"))\n", "\n", " plt.subplot(122)\n", " x2 = np.linspace(-2,2,100)\n", " y2 = np.tanh(x2+ 0.23)\n", " plt.ylim(-1.2,1.2)\n", " plt.xlim(-2.2,2.2)\n", " plt.plot([-2,2],[1,1], color='black', ls=\"dashed\")\n", " plt.plot([-2,2],[-1,-1], color='black', ls=\"dashed\")\n", " plt.plot(x2, y2, lw=3)\n", " ax = plt.gca()\n", " ax.spines['right'].set_color('none')\n", " ax.spines['top'].set_color('none')\n", " ax.xaxis.set_ticks_position('bottom')\n", " ax.spines['bottom'].set_position(('data',0))\n", " ax.yaxis.set_ticks_position('left')\n", " ax.spines['left'].set_position(('data',0))\n", "\n", " plt.annotate(r'$\\theta_0$',\n", " xy=(-.23,0), xycoords='data',\n", " xytext=(-50, +50), textcoords='offset points', fontsize=26,\n", " arrowprops=dict(arrowstyle=\"->\"))\n", "\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4sAAAGKCAYAAACl0NTvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3hUZfrG8XvSOxAIEHrokEgNCCKigIIi4KJiWVxAWXApKygqKEVWFHVRwZ+A4NqWoiAugqAioOCKsvSW0CGhk0ACCenJnN8fWQazQwlkMicz8/1cVy7zvmfOzJM45Mmdc857LIZhCAAAAACA3/MyuwAAAAAAQNlDWAQAAAAA2CEsAgAAAADsEBYBAAAAAHYIiwAAAAAAOz7X2c5SqUAxdO/eXd9//73ZZQCuwGJ2AW6A3gwUA70ZKLar9maOLAIOcPbsWbNLAAAAv0NvBkqOsAgAAAAAsENYBAAAAADYISwCAAAAAOwQFgEAAAAAdgiLAAAAAAA7hEUAAAAAgB3CIgAAAADADmERAAAAAGCHsAgAAAAAsENYBAAAAADYISwCAAAAAOwQFgEAAAAAdgiLAAAAAAA7hEUAAAAAgB3CIgAAAADADmERAAAAAGCHsAgAAAAAsENYBAAAAADYISwCAAAAAOwQFgEAAAAAdgiLAAAAAAA7hEUAAAAAgB3CIgAAAADADmERAAAAAGCHsAgAAAAAsENYhNs4fvy4RowYofbt2ysoKEgWi0UJCQnF2tdqtWrKlCmqU6eOAgIC1Lx5c3311VelWzAAAG6O3gy4NsIi3MbBgwe1aNEiVahQQR07dryhfcePH69XXnlFw4cP13fffad27drp4Ycf1rfffltK1QIA4P7ozYBrsxiGca3t19wIlCVWq1VeXoV///jHP/6hP//5zzpy5Ijq1Klzzf2SkpJUs2ZNjRkzRpMmTbLNd+nSRcnJydq5c+d1Xzs2NlabN28uUf2Ah7CYXYAboDfDZdCbAZdw1d7MkUW4jUvN6EatXLlSubm56tevX5H5fv36adeuXTpy5IgjygMAwOPQmwHX5rQji3feeafdXN++fTV06FBlZmbqvvvus9s+YMAADRgwQGfPntVDDz1kt/0vf/mLHnnkER07dkxPPPGE3fbnnntOPXv21L59+zRkyBC77ePGjVPXrl21fft2jRw50m7766+/rttuu02//vqrXnrpJbvt06ZNU4sWLbR69WpNnjzZbvvs2bPVqFEjffPNN3r77bftts+dO1c1a9bUwoULNWvWLLvtixcvVqVKlfTpp5/q008/tdv+7bffKigoSDNnztSiRYvstq9du1aSNHXqVC1fvrzItsDAQH333XeSpFdffVVr1qwpsr1ixYq26wLGjh2r3377rcj2GjVqaN68eZKkkSNHavv27UW2N2zYUHPmzJEkDR48WPv37y+yvUWLFpo2bZqkwh/8x48fL7K9ffv2mjJliiTpwQcf1Llz54ps79Kli8aPHy9Juvfee5WVlVVke0REhBYvXqwjR45owIABtvkLkbE6X6ODDG8/OdKpz0Yqsv80hz4nUBYlvNGjpE/BkcWSozfTm92qN1/y+/de48aNdfz4cd1xxx227QMGDFDTpk116623KiYmRhUrViyy//++97Zs2aLWrVvbtvPeWyvJM997999/v0aPHi3JvX7uXfp/6gAcWQQklUpQBAAAjpWfny8fHx+7+fDwcNv2K5kzZ4569OihLVu2KC8vr1RrBDwB1yzCLV3tuog6Y1aUyutxZBGegiOLZQK9GS7pRq5Z/POf/6zly5fr1KlTReYPHDighg0b6p///OcVj+D8HtcsAsV21d5s/ycbwENc+qX3xRdf1PTp05WVlSWL5fK/lY0bN+rWW2/V8uXL1aPHtX9Bjl09UZtL/ks0AABQ4RHE1NRUGYZRpDenpqbatgPuKCe/QGcv5io5PUdn03N09mKOzmXk6uzFHKVk5ColI1fnLuYqNTNXF3PytXPiPUX+jTgaYREeLzo6Wjk5OTp06JDq169vm4+Pj5ckNW3a1KzSAADwSPRmuBvDMHQ+M08nL2Tp9IVsnbyQrdMXsnQmLUdn0rJ1Ji1bSek5Op95Y6dPZ+YWKNi/9CIdYREer3v37vLz89P8+fM1ceJE2/y8efMUExOjqKgoE6sDAMDz0Jvhii5k5ikxJUPHUrJ0LDVTx1IydTw1SyfOZ+nk+Sxl5hY4/DVTMnIJi0BxLV68WJK0ZcsWSdJ3332niIgIRUREqFOnTlfcp3Llyho1apSmTJmi0NBQtWrVSgsXLtSPP/6opUuXOq12AADcUXF6s4+Pj/r376+PPvpIEr0ZZVdmbr4OJ2fo8NkMHU6+qCNnM5RwLlOJ5zJu+KjglXh7WVQx2E+Vw/xVKaTwo2KInyoF+ys82E/hIX6qGOxX+Hmwn4L8SjfOERbhVh5++OEi46FDh0qSOnXqdM3lhV977TWFhIRo+vTpOn36tBo1aqRFixapZ8+epVkuAABurzi9uaCgQAUFRY+60JthpqzcAu0/k659p9O1/0y6DiRd1MGkizpxPuv6O19FsJ+3IssHKrJcgCLLBahqucLPq4T5q3JogKqEBSg82E/eXmVnLThWQ4VH+f1qqA5Y1dGGFdeAYis7HdB10ZuBYqA3o7iS03O0++QFxZ9MK/w4laaEcxm6dkyy5+/jpdoVg1QrPEg1KgSpZniQalQIVI0KgapePlDlAn1LdTGaEmA1VAAAAACeLT07TzuOXdCO4+e18/h57Tx+QacuZBd7f28vi2pXDFLdSiGqFxGsqEqFH7UrBqtyqL+8ytBRQUcgLAIAAABwO4Zh6GhKpjYeSdHWo6namnhe+5PSi3XE0Msi1akYrEZVQ9WoaqgaVglV/cohqlMxWH4+XqVffBlBWAQAAADg8gzD0JGzGfrt8Dn9duicNiWk6ExaznX3C/D1UtPIMEVXK6foamFqWi1MDauEKsDX2wlVl22ERQAAAAAu6ezFHP1y4Kx+PpCsXw+e0+m0a59S6mWRGlcNU8ta5dW8Rnk1q1lO9SNC5OPtOUcLbwRhEQAAAIBLKLAa2nH8vH7ck6S1+5O0+0TaNR8f6u+j1nUqqE2dcLWqVUHNapQr1fsSuhu+UwAAAADKrKzcAv18IFk/xJ3R2n1JOpeRe9XHhvr76Na6FdW+XkW1qxuuxlXDytStKFwNYREAAABAmXIxJ19r9pzR97tPa+2+ZGXlFVzxcd5eFrWqVV53NIhQx4YRiqkWximlDkRYBAAAAGC67LwCrd2XpGU7TmrNniTl5Fuv+LhKIf7q3DhCnRtXVof6lRQa4OvkSj0HYREAAACAKQzD0KaEVP1r63Gt2HlK6Tn5V3xc/coh6hZdRfc0rapbqpdzu/sZllWERQAAAABOdfpCthZtPqbFW47raErmFR/TuGqoetwSqXtvqar6lUOdXCEkwiIAAAAAJyiwGlq7L0mfbzyqH/cmyWrYP6ZOxSD1al5NPZtXU4MqBESzERYBAAAAlJrzmblauOmY/vlbok6cz7LbHhbgo/ubV9ODrWqoVa3yslg4xbSsICwCAAAAcLiDSen66JcjWrLthLLz7Berua1eRT3atpbuaVpFAb7eJlSI6yEsAgAAAHCISwvWzPn5kFbvSbLbXiHIV31ja+rRtrUUVSnYhApxIwiLAAAAAErEMAz9tC9J7/94UFuPnrfb3jQyTAM61FGv5tU4iuhCCIsAAAAAborVamjVnjP6vx8PaPeJNLvtXZtU0eA76qpNnQpci+iCCIsAAAAAbohhGFq7L1l/X7lP8aeKhkQ/by/1aVVdgzrWVf3KISZVCEcgLAIAAAAoto1HUvT3lXu1KSG1yLy/j5f+eGttDelUV1XCAkyqDo5EWAQAAABwXQeTLmrKt3u0Zm/RhWsCfL30p/Z1NKhjlCqHEhLdCWERAAAAwFWdu5ij6WsOaP5/jqrAatjmfb0terxtLQ3rXJ+Q6KYIiwAAAADs5BdYNXdDot5ZtV/p2fm2eYtF+kOL6hp1d0PVDA8ysUKUNsIiAAAAgCI2JaRo/Ne7tfd0epH5dnXDNa5HU8VUL2dSZXAmwiIAAAAASVJKRq5eW7FHX209XmS+TsUgvdyjqbo2qcwtMDwIYREAAADwcIZhaOn2k/rb8nilZOTa5gN9vTW8c30N6hglfx9vEyuEGQiLAAAAgAc7npqpl5fs1rr9yUXm742pqnH3N1X18oEmVQazERYBAAAAD2QYhhZuOqZXl8crI7fANl+tXIAm/yFGnRtXMbE6lAWERQAAAMDDnEnL1pivduqnfZePJlosUv/2dTS6WyOF+BMTQFgEAAAAPMqKnaf00pJdupCVZ5urWylYf3+4uVrXrmBiZShrCIsAAACAB8jMzdffvonXF5uOFZl/skOUnu/WSIF+LGCDogiLAAAAgJuLP5mmEZ9v1aHkDNtc9fKBmvpwc7WvV9HEylCWERYBAAAAN2UYhhZsPKpJy+KVW2C1zd/fLFKv97lFYQG+JlaHso6wCAAAALihzNx8vbxkt5ZsO2GbC/Lz1iu9ovVw6xqyWCwmVgdXQFgEAAAA3Myh5IsaOm+r9p1Jt801iQzTjMdbqm5EiImVwZUQFgEAAAA38uPeM3rm8+1Kz8m3zT3cuoZefSBGAb4sYoPiIywCAAAAbsAwDH2w7rDeWrlXhlE45+fjpVd7R+uRNrXMLQ4uibAIAAAAuLjsvAK9+NVOLd1+0jZXvXygZj/RWjHVy5lYGVwZYREAAABwYcnpORr0z83acey8ba5tnXDN7NdKlUL8TawMro6wCAAAALiog0npGvDJJh1PzbLNPda2lib1ipafj5eJlcEdEBYBAAAAF/TrobN6eu4WpWUXLmTjZZEm9oxW/9vqmFsY3AZhEQAAAHAxS7ef0OgvdyivoHAlmyA/b/3fYy3VpUkVkyuDOyEsAgAAAC7kk/VHNOmbeNu4cqi/Ph7QhoVs4HCERQAAAMAFGIaht3/Yr/d/Omiba1glRJ8ObKtq5QNNrAzuirAIAAAAlHEFVkPjl+7Wgv8ctc21rl1BH/dvo3JBviZWBndGWAQAAADKsPwCq577ckeReyh2blxZMx5vpUA/bxMrg7sjLAIAAABlVG6+Vc98sU3f7T5tm+vTsrrefKiZfL25NQZKF2ERAAAAKIOy8wo0bP5WrdmbZJvr166W/tYrRl5eFhMrg6cgLAIAAABlTHZegQbP3aKf9yfb5p66PUrjejSRxUJQhHMQFgEAAIAyJCe/QEP+JygOu6ueRt/TiKAIpyIsAgAAAGVEbr5VQ+dt1brfBcWRXRtoZNeGJlYFT8VVsQAAAEAZkFdg1fAFRa9R/Gvn+gRFmIawCAAAAJiswGpo1MLt+iH+jG1u6J31NOpugiLMQ1gEAAAATGQYhsZ9vUvLd56yzQ2+o66e78Y1ijAXYREAAAAwiWEYeuO7vfp84zHbXP/2tTX23sYERZiOsAgAAACYZObaQ5r982HbuE/L6prYM5qgiDKBsAgAAACY4PONR/X3lfts465NqujNh5rJy4ugiLKBsAgAAAA42Zo9Z/Tykl22cfu6FfX+4y3l682v5yg7eDcCAAAATrTtaKqGLdgqq1E4jqkepg/7xyrA19vcwoD/QVgEAAAAnOTI2Qw99dlmZedZJUk1KgTq4wFtFOLvY3JlgD3CIgAAAOAE5y7mqP/HG5WSkStJqhDkq8+ebKvKoQEmVwZcGWERAAAAKGXZeQUaPHeLjqZkSpICfL300YA2qhcRYnJlwNURFgEAAIBSZBiGXvxqp7YkpkqSLBbpvUdbqlWtCiZXBlwbYREAAAAoRe+tOail20/axi/f10T3RFc1sSKgeAiLAAAAQClZtuOk3l293zZ+rG0tPXV7lIkVAcVHWAQAAABKwc7j5/X8lzts49vrV9LfekfLYrGYWBVQfIRFAAAAwMGS0rM1ZO4W5eQX3iKjbkSwZvyxlXy9+fUbroN3KwAAAOBAuflWDZ23VacuZEuSwgJ89FH/NioX6GtyZcCNISzCrRw7dkwPPfSQypUrp7CwMPXp00dHjx4t1r4Wi+WKH9u3by/lqgEAcF+e1psNw9DEZbu1+b8rn3pZpP97vJWiKgWbXBlw43zMLgBwlMzMTHXu3Fn+/v767LPPZLFYNG7cON11113auXOngoOv/0N6wIABGjJkSJG5hg0bllbJAAC4NU/szfP/c1SfbzxmG4+5t7E6NYwwsSLg5hEW4TY+/PBDHT58WPv27VP9+vUlSc2aNVODBg00e/ZsPfvss9d9jurVq6tdu3alXSoAAB7B03rz1qOpmvRNnG38QItq+nPHuiZWBJQMp6HCbSxbtkzt2rWzNSNJioqKUocOHbR06VITKwMAwDN5Um8+ezFHw+ZvVV6BIUmKrhamNx5sxsqncGmERbiNuLg4xcTE2M1HR0crPj6+WM8xa9Ys+fv7KygoSJ07d9a///1vR5cJAIDH8JTenF9g1V8/32Zb0KZcoK8+6NdaAb7eJlcGlAxhEW4jJSVFFSpUsJsPDw9Xamrqdffv16+fZs6cqdWrV2vOnDk6d+6cOnfurLVr117x8XPmzFFsbKxiY2OVnJxc0vIBAHA7ntKb3161X78eOidJslikaY+2UM3wIKe9PlBauGYRbuVKp3oYhlGsfefOnWv7vGPHjurdu7diYmI0btw4/fLLL3aPHzx4sAYPHixJio2NvcmKAQBwb+7em1fFn9GstYds42e6NNBdjSo75bWB0saRRbiNChUqKCUlxW4+NTX1in/VvJ7Q0FD16NFDmzZtckR5AAB4HHfvzcdTM/Xcosu38birUYT+2rmBiRUBjkVYhNuIjo5WXFyc3Xx8fLyaNm16U89pGAYXpgMAcJPcuTfn5ls1fME2pWXnS5Kqlw/Uu4+0kJeX+bUBjkJYhNvo1auXNmzYoMOHD9vmEhIStH79evXq1euGny8tLU0rVqzQrbfe6sgyAQDwGO7cm/++cq+2HzsvSfLxsui9x1qqfJCfyVUBjmW5zjnjxTuhHCgDMjIy1Lx5cwUGBmry5MmyWCwaP3680tPTtXPnToWEhKjOmBW2xz8ZtEUTJkyQJE2dOlX79u3TXXfdpWrVqikxMdE2t2bNGnXs2PGarx0bG6vNmzeX6tcHuAn+5F5y9Ga4jOL0ZklKTExUvXr1NGHCBJfozavjz2jQPy8/90v3NdbgO+qVymsBTnDV3swCN3AbwcHB+vHHHzVq1Cg98cQTMgxDXbp00bRp02zN6PesVqvt80aNGmnJkiVasmSJLly4oLCwMHXo0EEfffSR2rZt68wvAwAAt1Hc3mwYhgoKClyiN588n6XRi3fYxp0bV9ag2+uaVg9QmjiyCI/y+yOLCW/0cNjzcmQRKDaOLJYcvRkohtLozQVWQ49/uEH/OVK4aE9kuQB9+9eOqhDM6adwaVftzVyzCAAAABTDrLUHbUHRyyJNf7QlQRFujbAIAAAAXMeWxFS9u/qAbTyicwO1jQo3sSKg9BEW4RDnz5/Xq6++qhYtWig0NFTh4eHq3Lmzvv32W7NLAwDAI9GbHSctO0/PfLFNBdbCs8Bja1fQiM71Ta4KKH0scIMSW7t2rR5//HGdOnWqyPxPP/2ktWvXatasWRoyZIhJ1QEA4HnozY414evdOp6aJUkKDfDRtEdbyMebYy5wf7zLUSJr167Vfffdp1OnTqlfv37atGmTUlNTtXHjRrVv316GYejZZ5/V8ePHzS4VAACPQG92rGU7Turr7Sdt4zf6NFONCkEmVgQ4D2ERNy0pKUmPPvqosrKy9NZbb2nu3LmKjY1V+fLl1aZNGy1dulQhISHKzMzUggULzC4XAAC3R292rFMXsjRuyS7b+OHWNdSjWaSJFQHORVjETRs9erTOnDmj+++/X88//7zd9oiICHXo0EGStG7dOmeXBwCAx6E3O47Vamj0lzuUlp0vSaoZHqiJvaJNrgpwLsIibsrevXs1f/58WSwWvfXWW1d9XEREhCQpMTHRWaUBAOCR6M2O9emvCVp/8JwkyWKR3unbQiH+LPcBz0JYxE2ZPXu2rFarunbtqiZNmlz1cXl5eUX+CwAASge92XEOnEnXG9/vtY2f7lRPbepwmwx4HsIibpjVatUXX3whSfrjH/94zcempBTeuDYwMLDU6wIAwFPRmx0nr8CqUYu2KzffKklqGhmmUV0bmlwVYA7CIm7Y9u3bdfr0aUnSgAEDZLFYrvqxatUqSVLNmjXNLBkAALdGb3acmT8d0u4TaZIkPx8vTXu0hfx8+JUZnol3Pm7YzVwQ36BBg1KoBAAASPRmR9l94oL+78cDtvFzdzdUwyqhJlYEmIuwiBu2detWSVK3bt2UlZV11Y8lS5bY9mnVqpVZ5QIA4PbozSWXk1+g0V/uUL7VkCS1rl1BgzrWNbkqwFyERdyw/fv3S5Jq1aqlgICAq35s2LDBts8dd9xR5DkWLlyoNm3aKDAwUBUrVlTfvn11+PBhp34dAAC4C3pzyb235oD2nk6XJAX4emnqw83l7WUxuSrAXIRF3LDjx49LksLDr70q2IoVKyRJTZo0Ua1atWzzH3zwgR599FH5+vrq3Xff1bPPPqsff/xR7du319GjR0uvcAAA3BS9uWR2HDuvWWsP2cYvdm+sqErBJlYElA3cLAY3LCsrS5IUEBBw1cfs3btXu3fvliQ98cQTtvmUlBS98MILatGihdatWydfX19J0r333qs2bdropZde0rx580qxegAA3A+9+ebl5Bfo+cU79N+zT9Wubrj6t69jak1AWcGRRdwwPz8/SVJGRsZVHzNz5kxJhctyDxgwwDb/9ddfKz09Xc8884ytGUmF10107txZX331lTIzM0uncAAA3BS9+ebN+OmQ9p+5KEkK9PXWWw82lxennwKSCIu4CbVr15Yk7du374rbjxw5ojlz5kiShg8frsjISNu2jRs3SpI6dOhgt9/tt9+u7Oxs2189AQBA8dCbb078yTTN/OmgbfxC90aqVTHIxIqAsoWwiBvWqVMnSdIPP/ygkydPFtmWkZGhRx55RDk5OWrYsKEmTpxYZPuJEyckSTVq1LB73ktzl667AAAAxUNvvnF5BVY9v/jy6qextStw+inwPwiLuGEDBw6Uj4+PcnJy1KtXL23atEmpqan64Ycf1KFDB23atEnh4eH68ssvFRxc9OLwS6ex+Pv72z1vYGBgkccAAIDioTffuDk/H1bcyTRJkr+Pl956qBmnnwL/g7CIG9akSRPbXyW3bNmitm3bKjw8XN26ddOOHTtUt25drVmzRs2aNbPbNyio8NSOnJwcu23Z2dlFHgMAAIqH3nxjDiZd1PTVB2zjZ+9uqLoRISZWBJRNhEXclHHjxmnhwoVq3769wsLCFBoaqtatW+utt97Srl271KJFiyvuV716dUlXPp3l2LFjkq58GgwAALg2enPxWK2GXvrXLuUWWCVJzWqU01O3R5lcFVA2cesM3LS+ffuqb9++N7RP27ZtNXv2bP32229q0KBBkW3r169XQECAYmJiHFkmAAAeg958fZ9vOqqNCSmSJB8vi958sJl8vDl+AlwJ/zLgVL1791ZISIimT5+u/Px82/y2bdu0Zs0a9enTx+1OdQEAoCzzpN58+kK23vh2r208pFNdNYkMM7EioGzjyCKcqmLFinrjjTc0fPhwderUSX/605907tw5vfvuu6pYsaJee+01s0sEAMCjeFJvnrB0t9JzCgNx3UrBGtG5wXX2ADwbYRFON2zYMIWHh2vq1KkaOXKkAgMD1aVLF02ZMkV16tQxuzwAADyOJ/Tm73ef0g/xZ2zj1/vcogBfbxMrAso+wiJM8dhjj+mxxx4zuwwAAPBf7tyb07LzNGFpnG38WNuaale3ookVAa6BaxYBAADg1qau3Kek9MJbg0SE+mvMvU1MrghwDYRFAAAAuK1tR1M1d0OibTyxZ1OVC/Q1sSLAdRAWAQAA4JbyCqwa+69dMozC8Z2NItTjlkhziwJcCGERAAAAbumT9Ue093S6JCnA10uv9o6RxWIxuSrAdRAWUWbs379fr776qtllAACA/9q7d69ef/11s8u4Kbn5Vr276oBtPLJrQ9UMd4/7RQLOQlhEmREZGak5c+Zo7dq1ZpcCAABU2JtnzJihX375xexSbohhGDp5IUtZeQWSpMZVQ/XU7VEmVwW4HsIiyozQ0FBNnz5dTz/9tHJycswuBwAAj1euXDm9++67evrpp5Wbm2t2OcX2Q/wZpWfnS5IsFum1P9wiX29+7QVuFP9qUKb84Q9/UIMGDfTWW2+ZXQoAAJD08MMPq2bNmnrnnXfMLqVYMnLyNWnZ5XsqPtqmllrXrmBiRYDrIiyiTLFYLHr//fc1bdo0HTx40OxyAADweBaLRTNmzNDUqVN15MgRs8u5rvfWHNDJC9mSpIrBfnqxeyOTKwJcF2ERZU7t2rU1ZswYDR06VMalta4BAIBp6tatq+eee07Dhg0r07153+l0ffTL5UA79r4mKh/kZ2JFgGsjLKJMGjlypE6fPq0vvvjC7FIAAICk5557TomJiVq8eLHZpVyR1Wpo3Ne7lG8tDLPBfj56sFV1k6sCXBthEWWSr6+vZs+ereeee07nz583uxwAADyen5+fZs+erVGjRiktLc3scuz8a9sJbUpIlST5eFlUrXwg91QESoiwiDKrffv26tWrl8aOHWt2KQAAQNLtt9+u7t27a9y4cWaXUsSFzDxN+XaPbTyoY10F+PJrLlBS/CtCmTZlyhQtXbpUGzZsMLsUAAAg6c0339SiRYu0efNms0uxeXvVPp3LKLy1R7VyAfprl/omVwS4B8IiyrQKFSpo6tSpGjJkiPLy8swuBwAAj1exYkW99dZbGjJkiPLz880uR7tPXNC8DYm28fj7myrIz8fEigD3QVhEmffYY4+pcuXKmj59utmlAAAASU888YTKlSunGTNmmFpH4aI2u/XfNW10R8MIdY+pampNgDshLKLMs1gsmjVrlt544w0lJiZefwcAAFCqLvXmV199VcePHzetji+3HNP2Y4UL4fl5e2lSr2gWtQEciLAIl1C/fn0988AjNG8AACAASURBVMwzGjFiRJm+vxMAAJ6iUaNGGjZsmJ555hlTXv98Zq7e+G6vbTz4jrqKqhRsSi2AuyIswmW88MIL2r9/v77++muzSwEAAJLGjh2rXbt2afny5U5/7bd/2K/UzML1DKqXD9Swu1jUBnA0wiJchr+/vz744AP99a9/VXp6utnlAADg8QICAjRz5kwNHz5cGRkZTnvd3ScuaP5/ii5qE+jn7bTXBzwFYREu5c4771SXLl00YcIEs0sBAACSunbtqttvv12TJk1yyusZhqGJy+Jsi9p0bFBJ3aKrOOW1AU9DWITLmTp1qhYsWKCtW7eaXQoAAJD09ttv69NPP9XOnTtL/bX+tfWEtiSmSpJ8vS16hUVtgFJDWITLqVSpkqZMmaIhQ4aooKDA7HIAAPB4VapU0eTJkzVkyBBZrdZSe5207DxN+d2iNk/dXlf1IkJK7fUAT0dYhEsaOHCgAgMDNWvWLLNLAQAAkgYNGiSLxaIPP/yw1F5j+uoDOnsxR5JUNSxAIzqzqA1QmgiLcEkWi0UffPCBJk2apJMnT5pdDgAAHs/Ly0sffPCBxo0bpzNnzjj8+Q+cSddnvybYxi/1aKJgfx+Hvw6AywiLcFlNmzbV4MGDNWrUKLNLAQAAkpo1a6aBAwfq2WefdejzGoahV76JU/5/V7W5NSpcPZtFOvQ1ANgjLMKljRs3Tps3b9b3339vdikAAEDSxIkTtX79eq1atcphz7ky7rTWHzwnSfKyiEVtACchLMKlBQYGaubMmRo6dKgyMzPNLgcAAI8XHBys999/X0OHDlV2dnaJny8rt0CvLt9jGz/RrraaRIaV+HkBXB9hES6vW7duatu2rSZPnmx2KQAAQNL999+vZs2a6fXXXy/xc32w7pBOnM+SJIUH++nZuxuV+DkBFA9hEW7h3Xff1Ycffqi4uDi7bfn5+bpw4YIJVQEA4LmmT5+umTNnau/evXbb8vPzlZaWdt3nOJaSqQ/WHbKNn+/WSOWCfB1aJ4CrIyzCLURGRmrSpEl6+umn7e7v9NNPP2ngwIEmVQYAgGeqUaOGxo8fr6efflqGYRTZtmrVKg0aNOi6z/H6t3uUk1/Y12Oqh6lvbM1SqRXAlREW4TaGDBmi3NxcffLJJ0Xmw8LCdOzYMZOqAgDAcw0fPlzp6en65z//WWS+OL3514Nn9d3u07bxpF7R8vZiURvAmQiLcBve3t6aPXu2XnrpJSUnJ9vmq1atWir3ewIAANd2qTe/+OKLOnfunG3+er05v8CqV765fGlJn5bV1bp2eKnWCsAeYRFupUWLFurXr59Gjx5tm6tSpYrOnDljdwoMAAAofbGxserbt69eeOEF21yVKlV0+vTpq/bmeRsStf/MRUlSkJ+3Xry3sVNqBVAUYRFuZ9KkSVq7dq1++uknSVJAQIACAwOVmppqcmUAAHimyZMna+XKlfr3v/8tSQoJCZGXl5cuXrxo99hzF3P0zqr9tvGIzg1UJSzAabUCuIywCJeWkZGh4cOHKzEx0TYXEhKi9957T08//bRycnIkcSoqAADOkp6erhEjRujo0aO2ubCwME2bNs22voBU2JtPnz5tt//bq/YrLTtfklSnYpCevL2OU+oGYI+wCJcWFBSkyMhItW7dWm+++aby8vIkSb1791aTJk305ptvSrp8ugsAAChdISEhioiIUKtWrTR16lRbb37wwQcVFRWlqVOnSrpyb9594oI+33g5ZI6/v6n8fbydVzyAIgiLcGkWi0Uvv/yyNm7cqHXr1qlly5a2U1z+7//+T++9954OHDhw1b9eAgAAx7JYLJowYYI2bNigVatWqVWrVlq/fr0sFotmzJihd955R4cOHbI768cwDE36Jk6XLmPs1DBCnRtXNumrACARFuEm6tatqxUrVmjSpEl6/PHH9eSTTyowMFAvvfSS/vKXv9gWuQEAAM5Rv359ff/99xo3bpz69u2rQYMGKTQ0VM8//7yGDRtmd2Txm52ntCmhcH0BHy+LJvRsKouFW2UAZiIswm1YLBa1bdtWrVq10vz581W5cmXNnTtXJ0+e1OnTp697ZDE7O1vPP/+8IiMjFRgYqPbt2+vnn392UvUAALgfi8Wi2267TbGxsZo7d64iIiL0xRdf6MiRIzp79qztD7mZufl6fcUe234DO9RRvYgQejNgMh+zCwAcJTMzU507d5a/v78WLlyoI0eO6OWXX1Z+fr4SExPVu3dvqdbV93/qqae0YsUK/f3vf1fdunU1Y8YMdevWTb/99ptatGjhvC8EAAA38b+9+fDhwxo3bpwKCgp0/PhxBQYGSpJmrT2k02nZkqRKIX4a0aWBJHozYDbCItzGhx9+qMOHD2vfvn2qX7++JKlXr15q0KCBrFarfvvtN6nWH6+4744dO7RgwQJ9/PHHGjhwoCSpU6dOio6O1oQJE7Rs2TKnfR0AALiLq/Xmhg0bqqCgQL/++quOpWRq9s+Hbfu80K2xwgJ86c1AGcBpqHAby5YtU7t27WzNSJLq1aunjh07qlWrVnrttdeuua+vr68eeeQR25yPj48effRRrVy50nYLDgAAUHxX6s3169dXx44d1bJlS02ePFmvrdij3HyrJKlZjXJ6qHUN2770ZsBchEW4jbi4OMXExNjNR0dH68iRI3r88cevuW9UVJSCgoLs9s3NzdXBgwcdXi8AAO7uWr05ISFBNVt30fdxl9cUmNgzWl5eFtu+9GbAXIRFuI2UlBRVqFDBbj48PFypqak3ve+l7QAA4MZcszefv6BJ38Tb5v7Qsrpa165QrH0vbQdQuizGpZvZXEH37t2Ns2fPOuSFkpOTFRER4ZDnwvV54vd769atqlKliqpXr15k/sSJEzp9+rRat26tXScu2OZvqV7O9vn+/ftltVrVuHHjIvumpaXpwIEDatSokUJCQopsS05O1qV/Hzk5OVxo70Se+P42m6O+51u2bFlpGEZ3B5TksejNrssTv9/X6s1JaVnyCSv8fnhZLGpYJVS+3pdvlUFvdi2e+P42mzN68zXDoqRrbrwRsbGx2rx5s6OeDtfhid/vKlWq6IEHHtDs2bOLzA8dOlRffvmlkpOTVWfMCtt8whs9bJ8/8sgj2r59u/bt21dk30WLFumRRx7R7t27FR0dfdXXDg4OVkZGhoO+ElyPJ76/zebA7zk3TSs5erOL8sTv99V686Bhz2iV322y+BeGvee7NdKwu+oXeQy92bV44vvbbM7ozZyGCrcRHR2tuLg4u/n4+Hg1bdr0uvseOXJEmZmZdvv6+fkVuTAfAAAUz9V6838yImxBsVZ4kJ66PeqK+9KbAXMRFuE2evXqpQ0bNujw4cvLbyckJGj9+vXq1avXdffNy8vTl19+aZvLz8/XwoULdc8998jf37/U6gYAwF1dqTev2bJXaZWb2cbjejRRgK/3FfelNwPm8n7llVeutf2aG29U69atHfl0uA5P+37fcsst+uKLL7R48WJVr15d+/fv1+DBgxUQEKCPPvpIfn5+mrb6gO3xab9+oU6dOkmSqlatqr1792rGjBmqVKmSUlNTNWbMGG3cuFHz5s1TZGTkNV97zpw5GjVqVKl+fSjK097fZYGDvueTHPEkHu4VRz4Z/5acy9O+3//bm/ft26/hC7bKCK4oSerYoJJG39NIR48eVaVKlSSJ3uzCPO39XRaUdm922jWLgDMcPXpUo0aN0qpVq2QYhrp06aJp06apTp06klTkmsUBAZv0+z+WZGVl6eWXX9aCBQt0/vx5NW/eXG+++abuvPPO674u5+kDxcY1iyVHb4ZL+X1v9q3bVqHdCwOct5dF3z/TUQ2qhCohIUFRUVGaOHEivRlwvqv2ZsIiPMrVFrgpKRoSUGyExZKjN8MlZeUWqOs763TifJYk6ckOUZrQ89prCpQEvRkotrKzwM3+/fv1zDPPqFmzZgoJCVFkZKR69eqlHTt2OLsUj/HOO++oZ8+eioyMlMVi0XVOPUYxHTt2TA899JDKlSunbdu2qU+fPjp69KjZZbmt48ePa8SIEWrfvr2CgoJksViUkJBgdllua/HixXrwwQdVu3ZtBQYGqlGjRho7dqzS09PNLg2lgN7sfJ7Ym2f/fMgWFCsG++mZrg0c/hr0ZueiNzuXGb3Z6WHxhx9+0E8//aT+/fvrm2++0cyZM5WcnKxbb71VW7ZscXY5HuHDDz9UUlKSHnjgAbNLcRuZmZnq3Lmz9u7dq88++0xRUVE6cOCA7rrrLpbpLiUHDx7UokWLVKFCBXXs2NHsctze1KlT5e3trddff13ff/+9/vKXv2jWrFm6++67ZbVazS4PDkZvdj5P683HUzM1a+0h23h0t0YqF+jr0NegNzsfvdm5TOnNhmFc68PhkpOTDavVWmTu/PnzRvny5Y0nnniiNF7S4xUUFBiGYRh5eXmGJGPixInmFmSi2i8ut32UxLRp0wwvLy/jwIEDhmEYRuvWrY3Dhw8b3t7exttvv+2IUvE/Lr2PDcMwPvzwQ0OSceTIEfMKcnNJSUl2c5999pkhyVizZk1Jnvp6fYeP6384HL3Z+TytNw+dt8XWf3u897ORX2C9/k43iN7sfPRm5zKjNzv9yGKlSpVksRQ9LbZcuXJq2LChTpw44exyPIKXF3dIcbRly5apXbt2Re7xFBUVpQ4dOmjp0qUmVua+eB87V0REhN1cmzZtJImf1W6I3ux8nvQz7bdD57Ri1ynb+JWe0fL2cvzly/Rm5/Ok93FZYEZvLhP/h1NSUrR79241adLE7FKAYomLi1NMTIzdfHR0tOLj402oCCh969atkyR+VnsIejMcIb/AqleWxdnGvVtUU2yd8FJ5LXozPFFp9+YyERZHjBghwzA0cuRIs0sBiiUlJUUVKlSwmw8PD1dqaqoJFQGl68SJE5owYYK6du2q2NhYs8uBE9Cb4Qjz/3NU+84ULr4R5OetsfeW3h8f6M3wNM7ozSUOi6tXr5bFYrnux9XuhzNlyhQtWLBA77//fpHTBnBlJf1+w3H+95QtqfAaYMDdXLx4Ub1795aPj48++eQTs8tBMdCbnYvefGUpGbl6+4d9tvGwu+qrarmAUn1NejM8hbN6s09Jn+C2227Tnj17rvu4oKAgu7kPPvhAL730kiZPnqwnn3yypKV4hJJ8v+E4FSpUUEpKit18amrqFf+qCbiq7Oxs9erVS4cPH9a6detUo0YNs0tCMdCbnYvefGVTf9intOx8SVLtikEa1DGqVF+P3gxP4czeXOKwGBQUpMaNG9/wfnPnztXQoUP13HPP6eWXXy5pGR7jZr/fcKzo6GjFxcXZzcfHx6tp09K7wTDgTHl5eXrwwQe1ceNGrV69WrfccovZJaGY6M3ORW+2t/vEBX2+8fL9DSfc31T+Pt6l+pr0ZngCZ/dmU65ZXLJkiQYOHKhBgwZp6tSpZpQAlEivXr20YcMGHT582DaXkJCg9evXq1evXiZWBjiG1WrVH//4R61Zs0ZLly5Vu3btzC4JpYzeDEcxDEOvLIvTpbM/72wUoc6NK5f669Kb4e7M6M0lPrJ4o37++Wc99thjatasmQYMGKANGzbYtvn7+6tly5bOLsntbd68WQkJCbabdcbHx2vx4sWSpPvuu8/jTotxhD//+c96//331bt3b02ePFnnz59X7969VbNmTQ0ZMsTs8tzWpfftpZuEf/fdd4qIiFBERIQ6depkZmluZ9iwYfryyy/18ssvKzg4uMjP6ho1anA6qpuhNzufO/fmr7ef0ObEwgVlfL0tmnB/0yteS+ho9GZz0Judx5TefK2bMJbkzo5XM3HiREPSFT9q165dGi/p8fr373/V77mn3Tj10g2Ba7+4vMTPlZiYaPTp08cIDQ01vLy8jN69e3vc99PZrvY+7tSpk9mluZ3atWtf9ftdwpuHm31De3f4cDh6s/O5a29Oy8o1YievsvXa17+Nd+rr05udj97sPGb0Zotx7RWiWD4KbqXOmBW2zxPe6OGw542NjdXmzZsd9nyAGyv9wwvuj96MMuv1b/dozs+Fp4FWCfPXmufuVIi/009kk0RvBm7AVXtzmbjPIgAAAFzbwaR0ffzLEdv4pfuamBYUATgGYREAAAAlYhiGXlkWr3xr4YHvtnXC1at5NZOrAlBShEUAAACUyMq40/rl4FlJkpdFmtQ72imL2gAoXYRFAAAA3LSs3AK9unyPbfxEu9pqEhlmYkUAHIWwCAAAgJs246eDOnE+S5IUHuynZ+9uZHJFAByFsAgAAICbcuRshm31U0kac29jlQvyNbEiAI5EWAQAAMANMwxDE5fFKbfAKklqWau8HmpVCjcFB2AawiIAAABu2Mq4M/p5f7IkyWKRXu0dIy8vFrUB3AlhEQAAADekcFGbeNu43621FVO9nIkVASgNhEUAAADckPd/OlBkUZvR97CoDeCOCIsAAAAotkPJF4ssavNi90YsagO4KcIiAAAAisUwDE1Yult5BYYkqVWt8nq4dU2TqwJQWgiLAAAAKJZvdp7S+oPnJEleFmnyA7ewqA3gxgiLAAAAuK607Lwii9oMuC1KTauFmVgRgNJGWAQAAMB1vbtqv5LTcyRJlUP9NeruBiZXBKC0ERYBAABwTbtPXNBnvybYxuPvb6rQABa1AdwdYREAAABXVWA19PKSXbIWrmmjDvUr6v5mkeYWBcApCIsAAAC4qvn/SdSO4xckSX7eXnq1d4wsFha1ATwBYREAAABXdCYtW3//fp9tPPSueqobEWJiRQCcibAIAACAK3p1ebzSc/IlSVGVgvV0p3omVwTAmQiLAAAAsLNuf7KW7zxlG7/2QIwCfL1NrAiAsxEWAQAAUERWboHGf73bNv5Dy+q6rX4lEysCYAbCIgAAAIqYtma/jqZkSpLCAnz0co8mJlcEwAyERQAAANjEnbygf/z7iG38co8mqhTib2JFAMxCWAQAAICkwnsqjv3XLhX896aKt0aFq29sTZOrAmAWwiIAAAAkSZ/+mqCdl+6p6OOlKX1u4Z6KgAcjLAIAAEDHUzP19g+X76k44q763FMR8HCERQAAAA9nGIbGfb1bmbkFkqSGVUI0hHsqAh6PsAgAAODhvt5+Qmv3JUuSLBZpSp9b5OfDr4mAp+OnAAAAgAc7ezFHk76Jt437t6+j1rXDTawIQFlBWAQAAPBgE5fF6XxmniSpevlAPd+tkckVASgrCIsAAAAeamXcaa3Yeco2ntLnFgX7+5hYEYCyhLAIAADggS5k5Wn817tt44da19AdDSNMrAhAWUNYBAAA8ECvLo9XUnqOJCki1F/jezQ1uSIAZQ1hEQAAwMP8uPeMFm85bhu/2jta5YJ8TawIQFlEWAQAAPAgFzLzNPZfu2zjns2rqXtMpIkVASirCIsAAAAe5G/L43UmrfD000ohfprUK9rkigCUVYRFAAAAD/Hj3jP6auvl008nP3CLwoP9TKwIQFlGWAQAAPAA5zNzi5x+2qt5NXWPqWpiRQDKOsIiAACAB5iwNO53p5/6c/opgOsiLAIAALi5b3ac1LIdJ23jKX1uUQVOPwVwHYRFAAAAN3YmLVvjvt5tG/eNraG7m1YxsSIAroKwCAAA4KYMw9ALi3fqQlaeJKl6+UCNv7+pyVUBcBWERQAAADc1/z9HtW5/siTJYpHe7ttcoQG+JlcFwFUQFgEAANzQwaSLmrwi3jZ+qkOU2tWtaGJFAFwNYREAAMDN5OZbNXLhNmXnWSVJDSqHaHS3RiZXBcDVEBYBAADczDur9mv3iTRJkp+3l6Y/2lIBvt4mVwXA1RAWAQAA3Mivh85q9s+HbOMXujdS02phJlYEwFURFgEAANzE+cxcPbtwhwyjcNyxQSU92SHK3KIAuCzCIgAAgBswDEMvfrVTp9OyJUkVgnz19sPN5eVlMbkyAK6KsAgAAOAG5m5I1Mq4M7bxmw82U+WwABMrAuDqCIsAAAAuLu7kBU1evsc27t++tu6JrmpiRQDcAWERAADAhWXk5GvEgm3KLSi8TUbTyDCNva+JyVUBcAeERQAAABc2/uvdOnw2Q5IU5Oet9x/nNhkAHIOwCLdhtVo1ZcoU1alTRwEBAWrevLm++uqrYu07YMAAWSwWu4+RI0eWctUAANy8RZuO6V/bTtjGr/0hRnUjQkysqCh6M+DafMwuAHCU8ePHa+rUqXrttdfUunVrffHFF3r44Ye1fPly3XfffdfdPyIiQsuWLSsyFxkZWVrlAgBQInEnL2j80t228UOta+gPLWuYWJE9ejPg2giLcAtJSUmaOnWqxowZo9GjR0uS7rrrLh08eFBjxowpVkPy8/NTu3btSrtUAABKLC07T0Pnb1VOfuF1io2qhOpvvaNNrqooejPg+jgNFW5h5cqVys3NVb9+/YrM9+vXT7t27dKRI0dMqgwAAMcyDEOjF+1Q4rlMSVKwn7dm9mulIL+ydQyA3gy4PsIi3EJcXJz8/f1Vv379IvPR0YV/ZY2Pj7/ucyQlJalSpUry8fFRw4YN9eabb6qgoKBU6gUA4Gb9499H9EP87+6n+FAz1StD1yleQm8GXF/Z+hMUcJNSUlJUvnx5WSyWIvPh4eG27dfSokULtW7dWtHR0crOztaSJUs0duxYHThwQP/4xz+uuM+cOXM0Z84cSVJycrIDvgoAAK7t10Nn9cb3e23jAbfV0f3NqplY0dXRmwHXR1hEmbR69Wrdfffd131cp06dtHbtWhmGYdeMpMJTdYrjf1dWu++++xQSEqJp06bpxRdfVIMGDez2GTx4sAYPHixJio2NLdbrAABws46nZmr4gm0qsBb2tpa1yuslJ95Pkd4MeB7CIsqk2267TXv27Lnu44KCgiQV/pUyNTXVrjGlpqbatt+oxx57TNOmTdPmzZuv2JAAAHCW7LwCPT1vi1IyciVJlUL8NeuPreXn47wriujNgOchLKJMCgoKUuPGjYv9+OjoaOXk5OjQoUNFro24dD1E06ZNb7iGS3/5vNJfRQEAcBbDMPTSv3Zp94k0SZKPl0Wz+rVS1XIBTq2D3gx4Hha4gVvo3r27/Pz8NH/+/CLz8+bNU0xMjKKiom74ORcsWCCLxaI2bdo4qkwAAG7YJ+sT9K9tJ2zjib2i1abOjR+VczZ6M+D6OLIIt1C5cmWNGjVKU6ZMUWhoqFq1aqWFCxfqxx9/1NKlS6+5b2Jiop544gk9+uijql+/vnJycrRkyRJ9+umnGjJkiOrVq+ekrwIAgKLW7kvS5BWXVw3tG1tD/W6tZWJFxXcjvblLly5KTEzUwYMHJdGbgbKCsAi38dprrykkJETTp0/X6dOn1ahRIy1atEg9e/a85n6hoaEKDw/Xm2++qTNnzshisahJkyZ67733NHToUCdVDwBAUQeT0jViwTb9dz0btahZXn/rHeNSp2AWtzcXFBQoPz/fNqY3A2WD5TorUhVvuSrARdQZs8L2ecIbPRz2vLGxsdq8ebPDng9wY67zW27ZRW/2AKkZuXpg5nolnsuUJEWWC9DS4R1UOdS51ym6MnozUGxX7c1cswgAAFCG5OZb9Zf5W2xBMdDXWx/+KZagCMDpCIsAAABlhGEYemnJLm04fPmG9e8+0lwx1cuZWBUAT0VYBAAAKCOmrzmgxVuO28aj72mo7jGRJlYEwJMRFgEAAMqAxVuOa9rqA7bxQ61raNhd9a+xBwCULsIiAACAyX45cFZjvtppG3dsUElT+tziUiufAnA/hEUAAAATxZ28oL/M26L8/94jo3HVUM38Yyv5evNrGgBz8VMIAADAJInnMtT/401Kzym8x2DVsAB9MrCNQgN8Ta4MAAiLAAAApkhKz9YTH23U2Ys5kqTQAB99MrCNIssFmlwZABQiLAIAADhZWnae+n+8SUdTCu+l6O/jpY/6t1GTyDCTKwOAywiLAAAATpSZm69Bn27WnlNpkiRvL4tmPN5KbaPCTa4MAIoiLAIAADhJdl6BBv9zizYmpNjm3uhzi7o2rWJiVQBwZYRFAAAAJ8jNt2ro/K365eBZ29y4Hk30cGxNE6sCgKsjLAIAAJSy/AKrnvlim37cm2Sbe+7uhhrUsa6JVQHAtREWAQAASlF+gVWjFu3Qd7tP2+aG3VVPI7o0MLEqALg+H7MLAAAAcFd5BVaN/GK7Vuw6ZZt7skOURt/TyMSqAKB4CIsAAAClIK/Aqr9+vq3IEcU/ta+t8fc3kcViMbEyACgewiIAAICD5eZbNeLzrVoZd8Y2N+C2OprYsylBEYDLICwCAAA4UGZuvobM3aJ/H7i86ulTt0dpXA+OKAJwLYRFAAAAB7mQlacnP92kLYmptrnBd9TV2HsbExQBuBzCIgAAgAMkp+foTx9v1J5Taba5kV0b6JkuDQiKAFwSYREAAKCEEs9lqP/HG5VwLtM2N+H+pnry9igTqwKAkiEsAgAAlMCOY+f15KebdC4jV5LkZZHeeqi5Hmpdw+TKAKBkCIsAAAA3ac2eMxq+YJuy8gokSf4+Xpr+aEt1j6lqcmUAUHKERQAAgJsw/z+JGv/1blmNwnH5IF/940+xiq0Tbm5hAOAghEUAAIAbkF9g1Wvf7tEn6xNsczUqBOrTgW1Vv3KIeYUBgIMRFgEAAIopLTtPIxZs07r9yba5mOph+nhAG1UODTCxMgBwPMIiAABAMSSczdCgf27WwaSLtrnu0VX1ziPNFeTHr1QA3A8/2QAAAK5jzZ4zGrlwu9Kz821zw++qr2fvbigvL+6hCMA9ERYBAACuwmo1NG3NAb235oBtzs/HS39/qJl6t6huYmUAUPoIiwAAAFeQmpGrUYu2a+2+y9cnVi8fqFn9WqlZjfImVgYAzkFYBAAA+B+bElL+v727D6qqzuM4/rnI40VFEFAUCEUBn0YWzUm0pJRSkHJTaXfGihlnNW2mMdO1Wk1mZK3ZSddm3TTbHXVqZ0fBKZNVbHXXhronKQAACypJREFUWM0pwVDB5weSBx8BEbtCXO7dP1pZr8dSE7nce9+vf8Tf9xzvd4Yz53c/nnN+R6/8/Rudq29sHRvVr7v+9OskhQT6OrEzAGg/hEUAAID/sdnsWvXFKS3/53G13HiBoqSZY/pq/pPx8u7k5cTuAKB9ERYBAAAknau/rnm5B7TnZE3rWDezj5ZNHaqxA3o4sTMAcA7CIgAA8HifHajWwk8O6epNq50+HBOs9371C/XqFuDEzgDAeQiLAADAY9VbmvXWZ6XaXFLdOmYySbNTYvXquDhuOwXg0QiLAADAI31edl4LPy3VxYam1rHI4AAtz0zUiD4hTuwMADoGwiIAAPAoNdealL3lsLYcqHYYnzIsUoszBqqLv4+TOgOAjoWwCAAAPILdbtem/VVauvWIar/7vnU8tLOfciYN1vjBPZ3YHQB0PIRFAADg9k5caNDvPi3V12dqHcYnJ0Vq0cQB6mbm3YkAcCvCIgAAcFvXmqxa+a+T+st/Tst603sTewX56/fPDtHj8eFO7A4AOjbCIgAAcDs2m12b9lfqD9uP6dJNC9h4e5k0fXQfvTK2vwL9+BoEAD+FsyQAAHAr+8prtST/sA5W1juMPxwTrJxJQxTfs4uTOgMA10JYBAAAbuHY+Qb9oeCodh696DDeo6ufXp+QoGeG9paXl8lJ3QGA6yEsAgAAl1ZRa9F7O09o0/5K2f//WKL8vL0047G+emlMLLecAsDPwJkTAAC4pMo6i/7875PKLap0WLzGZJJ+mdhbr6bGKSrE7MQOAcC1ERYBAIBLOVtj0erCU8otqlBzi92h9nh8mH47PkEDIro6qTsAcB+ERQAA4BIOV1/V6i9OKf9gtWyOGVEj+oRobmqcHunb3TnNAYAbIiwCAIAOy263q/DEZf119xkVHr9kqD8cE6xXx8VpZGx3mUwsXgMAbYmwCAAAOhzL91Z98k2V1u4p18mL1wz1R/uHataYWEIiADxAhEUAANBhnLjQoL99dVab9leqodHqUDOZpLTBEXppTKyGRAY5qUMA8ByERQAA4FSW763adui8NhRV6OsztYZ6Zz9vZQ6PUlZyjKK7s7opALQXwiIAAGh3drtdxd/WKa+4UvkHz+lak9WwTZ/QQE175CFlDo9UF38fJ3QJAJ6NsAgAANrN8QsN2lxSpc0l1aqsu26od/Iy6cmBPTTtkYeUzPOIAOBUhEUAAPBAnbjQoK2Hzmtb6TkdPd9w2236hgVq6rAoPZvUWz26+rdzhwCA2yEsAgCANmWz2XWg8op2HLmgz8su6MRtVjOVpK7+3kobEqGpwyOVFB3MVUQA6GAIiwAA4L41NDZrz8nL2nXsknYevahLDU233c7P20tjB4TrmcTeSokPk593p3buFABwtwiLAADgnrXY7CqtqteeU5dVePySisrrZLXZb7utv4+XnkgI14TBEXoiIVyBfnz9AABXwNkaAADckc1m17ELDfrqdI32nq7R3lM1utpoXMH0hpBAXz2REK7UgT30aP9QmX35ygEAroYzNwAAMGhsblFpVb2Kvq1TUXmdir6t1RVL80/uM6hXV6XEhyklPlxJ0cHq5MUziADgygiLAAB4OJvNrvKa73Sg8ooOVNSrpOKKyqrr1dxy+9tKbwjv4qfR/UKV3C9Uj8WFKrwLq5gCgDshLAIA4EFsNrtOX76m0qqrOlRVr9Kqeh2uvqqGph+/pfSG0M6+GtEnRCNiQjSqX6j6hXdmBVMAcGOERQAAPEijtUVP/rFQP7IWjYO+YYEaFh2sYQ8Fa3hMiGLDAgmHAOBBCIsAAHgQs6+3YsM6G959GGz20dCobhoa2U2J0T/8GRLo66QuAQAdAWERAAAP82j/MEWFmDW4V1cN7h2kwb2DFBHkz1VDAIADwiIAAB7mrYyBzm4BAOACvJzdAAAAAACg4yEswm0sX75cGRkZioiIkMlkUnZ29j3tv3v3biUnJysgIEA9e/bU3Llzdf369QfTLAAAHoC5GXBthEW4jQ8//FAXL17UpEmT7nnfgwcPKjU1VeHh4crPz1dOTo7Wrl2rrKystm8UAAAPwdwMuDaeWYTbKCsrk5eXl6xWq1avXn1P+y5evFiRkZHKzc2Vj4+PJMnX11cvvviiFixYoKSkpAfRMgAAbo25GXBtXFmE2/Dy+nmHc3NzswoKCpSZmdk6GUlSZmamfH19tXnz5rZqEQAAj8LcDLg2k93+k2/lvYtX9t5ZzOv/aIt/BmhT5e+ky2KxKCUlRfv27dOAAQMUHh4uScrKylJWVpbi4uJUV1enQYMGOew7a9YsPffcc6qoqNDzzz+v4uJiDRs2rLX+2muvKSMjQ8eOHdPMmTMNn71w4UKNGzdOJSUlmjNnjqG+dOlSJScn68svv9Sbb75pqK9YsUKJiYnasWOHcnJyDPUPPvhA8fHx2rJli5YtW2aof/TRR4qKitKGDRu0atUqQz0vL0+hoaFat26d1q1bZ6hv3bpVZrNZ77//vjZu3Gio79q1S5L07rvvKj8/36EWEBCgbdu2SZKWLFminTt3OtS7d++uTZs2SZLeeOMN7d2716EeGRmpjz/+WJI0Z84clZSUONTj4uK0Zs0aSdKMGTN0/Phxh3piYqJWrFghSZo2bZoqKysd6iNHjtTbb78tSZo8ebJqamoc6mPHjtWiRYskSRMmTDA8OzNx4kTNmzdPkpSSkqJbZWZmavbs2bJYLEpLSzPUbxx7ly9f1pQpUwz1W4+9Wz2oY+/G77QN8G6G+9cmc7PkXsfoDZwfO+b50W63q7CwUIsXL1Z2dvYdjz3mZo69mzE3O29u5soiPJKPydb6s9VqlSR5exvvyg4ODm6t32rNmjVKT09XcXGxmpubH0yjAAB4IOZmoGPgyiI8TqBvJ80ZF6ffPNZXkrRnzx6NHj1aBQUFeuqppxy2HTVqlPz9/Q3/w3ar4cOHq6io6IH1DLgRrizevza7sgjcix07dig1NfWO240ZM8ZwxcNqtcrHx6f1yuKdMDcD7epH5+Z2WeCm/J309vgYuBGLxaKzZ8/ecTuz2azo6GiHsXudkEJCQiRJtbW1htrtbnMBAMATJScn68iRI3fczmw23/dnMTcDHQOroaJDMpvNSkhIaJfPio2NlZ+fn8rKyhzGGxsbdfr0aU2dOrVd+gAAoCNjbgY8D88swuP5+vpq/Pjx2rhxo8MzEHl5eWpqatLTTz/txO4AAPA8zM1Ax8CVRbiNoqIilZeXy2b7YfGaw4cPKy8vT5KUlpbWelvM9OnTtX79eofJJzs7WyNHjlRmZqZefvlllZeXa/78+ZoyZYrDSmoAAODuMTcDro2wCLexcuVKrV+/vvXvubm5ys3NlSSdOXNGMTExkqSWlha1tLQ47JuYmKjt27drwYIFSk9PV1BQkF544QUtXbq03foHAMDdMDcDrq1dVkMF3B0rrgF3jdVQ7x9zM3AXmJuBu8Z7FgEAAAAAd4+wCAAAAAAwICwCAAAAAAwIiwAAAAAAA8IiAAAAAMCAsAgAAAAAMCAsAgAAAAAMCIsAAAAAAAPCIgAAAADAgLAIAAAAADAgLAIAAAAADAiLAAAAAAADwiIAAAAAwICwCAAAAAAwICwCAAAAAAwIiwAAAAAAA8IiAAAAAMCAsAgAAAAAMCAsAgAAAAAMCIsAAAAAAAPCIgAAAADAgLAIAAAAADAgLAIAAAAADAiLAAAAAAADk91ud3YPgMszmUwFdrt9vLP7AAAAP2BuBu4fYREAAAAAYMBtqAAAAAAAA8IiAAAAAMCAsAgAAAAAMCAsAgAAAAAMCIsAAAAAAIP/Ao9YXFZFum1OAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plot_activation_functions()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Perceptron a regresja liniowa\n", "\n", "\"Rys." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Uczenie regresji liniowej:\n", "* Model: $$h_{\\theta}(x) = \\sum_{i=0}^n \\theta_ix_i$$\n", "* Funkcja kosztu (błąd średniokwadratowy): $$J(\\theta) = \\frac{1}{m} \\sum_{i=1}^{m} (h_{\\theta}(x^{(i)}) - y^{(i)})^2$$\n", "* Po obliczeniu $\\nabla J(\\theta)$ - zwykły SGD." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Perceptron a dwuklasowa regresja logistyczna\n", "\n", "\"Rys." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Uczenie dwuklasowej regresji logistycznej:\n", "* Model: $h_{\\theta}(x) = \\sigma(\\sum_{i=0}^n \\theta_ix_i) = P(1|x,\\theta)$\n", "* Funkcja kosztu (entropia krzyżowa): $$\\begin{eqnarray} J(\\theta) &=& -\\frac{1}{m} \\sum_{i=1}^{m} \\big( y^{(i)}\\log P(1|x^{(i)},\\theta) \\\\ && + (1-y^{(i)})\\log(1-P(1|x^{(i)},\\theta)) \\big) \\end{eqnarray}$$\n", "* Po obliczeniu $\\nabla J(\\theta)$ - zwykły SGD." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Perceptron a wieloklasowa regresja logistyczna\n", "\n", "\"Rys." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Wieloklasowa regresja logistyczna\n", "* Model (dla $c$ klasyfikatorów binarnych): \n", "$$\\begin{eqnarray}\n", "h_{(\\theta^{(1)},\\dots,\\theta^{(c)})}(x) &=& \\mathrm{softmax}(\\sum_{i=0}^n \\theta_{i}^{(1)}x_i, \\ldots, \\sum_{i=0}^n \\theta_i^{(c)}x_i) \\\\ \n", "&=& \\left[ P(k|x,\\theta^{(1)},\\dots,\\theta^{(c)}) \\right]_{k=1,\\dots,c} \n", "\\end{eqnarray}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Funkcja kosztu (**przymując model regresji binarnej**): $$\\begin{eqnarray} J(\\theta^{(k)}) &=& -\\frac{1}{m} \\sum_{i=1}^{m} \\big( y^{(i)}\\log P(k|x^{(i)},\\theta^{(k)}) \\\\ && + (1-y^{(i)})\\log P(\\neg k|x^{(i)},\\theta^{(k)}) \\big) \\end{eqnarray}$$\n", "* Po obliczeniu $\\nabla J(\\theta)$, **c-krotne** uruchomienie SGD, zastosowanie $\\mathrm{softmax}(X)$ do niezależnie uzyskanych klasyfikatorów binarnych." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Przyjmijmy: \n", "$$ \\Theta = (\\theta^{(1)},\\dots,\\theta^{(c)}) $$\n", "\n", "$$h_{\\Theta}(x) = \\left[ P(k|x,\\Theta) \\right]_{k=1,\\dots,c}$$\n", "\n", "$$\\delta(x,y) = \\left\\{\\begin{array}{cl} 1 & \\textrm{gdy } x=y \\\\ 0 & \\textrm{wpp.}\\end{array}\\right.$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Wieloklasowa funkcja kosztu $J(\\Theta)$ (kategorialna entropia krzyżowa):\n", "$$ J(\\Theta) = -\\frac{1}{m}\\sum_{i=1}^{m}\\sum_{k=1}^{c} \\delta({y^{(i)},k}) \\log P(k|x^{(i)},\\Theta) $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Gradient $\\nabla J(\\Theta)$:\n", "$$ \\dfrac{\\partial J(\\Theta)}{\\partial \\Theta_{j,k}} = -\\frac{1}{m}\\sum_{i = 1}^{m} (\\delta({y^{(i)},k}) - P(k|x^{(i)}, \\Theta)) x^{(i)}_j \n", "$$\n", "\n", "* Liczymy wszystkie wagi jednym uruchomieniem SGD" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Perceptron – podsumowanie\n", "\n", "* W przypadku jednowarstowej sieci neuronowej wystarczy znać gradient funkcji kosztu.\n", "* Wtedy liczymy tak samo jak w przypadku regresji liniowej, logistycznej, wieloklasowej logistycznej itp. (wymienione modele to szczególne przypadki jednowarstwowych sieci neuronowych).\n", "* Regresja liniowa i binarna regresja logistyczna to jeden neuron.\n", "* Wieloklasowa regresja logistyczna to tyle neuronów, ile klas." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Funkcja aktywacji i funkcja kosztu są **dobierane do problemu**." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## 10.2. Funkcje aktywacji" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Operacje, które każdy neuron wykonuje, to dodawanie i mnożenie. Są to **funkcje liniowe**." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Złożenie funkcji liniowych jest funkcją liniową." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Gdyby nie było funkcji aktywacji, cała sieć neuronowa działałaby jak regresja liniowa." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **Najważniejszym celem** stosowania funkcji aktywacji jest **wprowadzenie nieliniowości**, dzięki której sieć może uczyć się zależności bardziej skomplikowanych niż tylko liniowe." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Jakie cechy powinna mieć dobra funkcja aktywacji?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **różniczkowalna** (co najmniej **przedziałami różniczkowalna**), aby można było obliczyć jej gradient" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **prosta obliczeniowo**, ponieważ podczas uczenia sieci jej wartość obliczana jest wiele razy" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* odporna na **problem zanikającego gradientu** (wyjaśnienie kilka slajdów dalej)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/home/pawel/.local/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n", " from ._conv import register_converters as _register_converters\n", "Using TensorFlow backend.\n" ] } ], "source": [ "%matplotlib inline\n", "\n", "import math\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import random\n", "\n", "import keras\n", "from keras.datasets import mnist\n", "from keras.models import Sequential\n", "from keras.layers import Dense, Dropout, SimpleRNN, LSTM\n", "from keras.optimizers import Adagrad, Adam, RMSprop, SGD\n", "\n", "from IPython.display import YouTubeVideo" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [], "source": [ "def plot(fun):\n", " x = np.arange(-3.0, 3.0, 0.01)\n", " y = [fun(x_i) for x_i in x]\n", " fig = plt.figure(figsize=(14, 7))\n", " ax = fig.add_subplot(111)\n", " fig.subplots_adjust(left=0.1, right=0.9, bottom=0.1, top=0.9)\n", " ax.set_xlim(-3.0, 3.0)\n", " ax.set_ylim(-1.5, 1.5)\n", " ax.grid()\n", " ax.plot(x, y)\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Funkcja logistyczna" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$$ g(x) = \\frac{1}{1 + e^{-x}} $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Funkcja logistyczna – wykres" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1cAAAG2CAYAAACTRXz+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XmsXudh3/nfw7vvGy/3VRu1Wou12HAWOk4cx0jjpq1bezpN0mXUdhq0M0Axk04GCaaDAl2AWVNMa6RB00GRNJhOWs/UbdI0ZdJ2YFmWLVnWZslcxEUSl7vxbrzbmT/el1ekxCvJvke8l+TnA7x43/O+5/I8NB6R/Pqc87ylqqoAAACwPls2egAAAAA3A3EFAABQA3EFAABQA3EFAABQA3EFAABQA3EFAABQg1riqpTya6WUs6WUb6/x+eFSymQp5dnm45fqOC4AAMBm0VrTr/OPk/xKkn/yHvv8h6qqfrKm4wEAAGwqtZy5qqrqD5OM1fFrAQAA3Iiu5z1XHy+lPFdK+dellPuu43EBAAA+dHVdFvh+vpFkf1VV06WUzyb5F0nuvNaOpZQnkzyZJJ2dnR/dt2/fdRoiN4qVlZVs2WItFq5mXrAWc4NrMS9Yi7nBtXznO985X1XV6PvtV6qqquWApZQDSf7fqqru/wD7Hk/yaFVV599rv0OHDlWvvPJKLePj5nHkyJEcPnx4o4fBJmNesBZzg2sxL1iLucG1lFKeqarq0ffb77pkeSllRymlNF8/3jzuhetxbAAAgOuhlssCSym/keRwkq2llFNJfjlJW5JUVfUPkvyJJH+5lLKUZC7JF6q6TpkBAABsArXEVVVVX3yfz38ljaXaAQAAbkru1gMAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKhBLXFVSvm1UsrZUsq31/i8lFL+t1LKa6WUb5VSHqnjuAAAAJtFXWeu/nGSz7zH5z+R5M7m48kk/0dNxwUAANgUaomrqqr+MMnYe+zyuST/pGr4apLBUsrOOo4NAACwGVyve652Jzl5xfap5nsAAAA3hdaNHsA7lVKeTOPSwYyOjubIkSMbOyA2nenpafOCdzEvWIu5wbWYF6zF3GA9rldcnU6y94rtPc333qWqqi8l+VKSHDp0qDp8+PCHPjhuLEeOHIl5wTuZF6zF3OBazAvWYm6wHtfrssAvJ/mZ5qqBH0syWVXVG9fp2AAAAB+6Ws5clVJ+I8nhJFtLKaeS/HKStiSpquofJPlKks8meS3JbJI/W8dxAQAANota4qqqqi++z+dVkr9Sx7EAAAA2o023oAUAAMCHpaqqXFpaycylpcwuLGdmYSkzl5Yyc2k5swuN55mFt7d//L4dH/jXFlcAAMCmd2lpOdPzS5m+tJSLzefp+aXMLFy9/fbni1e9N7uwvPq8vFJ9oGOWkuwd7v7AYxRXAADAh+bymaKpucVMzi1man4xU3NLmZpvbs8t5uIVETQ9v3T1dvP1wvLK+x6rZUtJb0drejta09fZeB7sbs+e4e70tLeku73xXndHS3raW9Pd3tLcbl39vKejJT0drelpb01n25aUUvKnPuDvVVwBAADvaXG5EUdT80vvGUlT80tXvG5+Prf4vmHU0bplNYZ6m8+7Brve9d7qo7M1fVe+39mavo621RjaKOIKAABuESsrVS7OL2VibiHjs4sZn13IZPN5fHYxE7MLmWhuT1zx+cVLS+/567ZuKRnoakv/5Udna3YPdTXe62xLf1dr+jvb3t6ns3X1dV9nazpaW67T/wIfLnEFAAA3oKqqMjW/lAvTlzI2s5ALMwu5ML2QsZlLzTBqxNL47EIm5hYz0dxe63ajUpL+zrYMdbdloLs9I73tuWNbbwa72zLY1Z7B7mtFUuP1Rp8x2izEFQAAbAKXY2lsZiEXpi/lwszCNV5fjqhLGZ9dyOLytUupq60lQ91tGexuz1BPW3YOdGWwuy1D3Y1IGuxuf/vz5vNAV1tatgik9RBXAADwIamqKhOzizk3fSnnLjYeZy/Or76+MLOQ882zTWMza8dSb0drRnrbM9zTnt2DnfnI7oEM97ZnpKe9+X7H6uuh7vZ0tt0cl9ndaMQVAAB8j+YXl5uh1Iym6Us5NzV/VURdfv9awdTZtiWjfR3Z2tuR3YOdeWB3f0Z6G4E03NO++los3VjEFQAANC2tVDk9MZc3J+fz1tT828/N142IunTNBR5KSUZ6OjLa15FtfR25c3tfRvs6Mtr79nujzUdvR6t7lG5C4goAgJteVVW5eGkpb02+HUpvR9OlvDU1nzcm53Nh+lKq3/39q362vWVLtg90ZEd/Z+7Z0Z8fuvPtSLocT9v6OzLc3Z7Wli0b9DtkMxBXAADc8OYXl3NmYi5nJuZzZmIupyfmGtuTc3ljohFRswvL7/q5we627OjvzPb+zty7sz/z42/mYw/evfrejoHODHW3OcvEByKuAADY1FZWqlyYWXg7mK6Mp2ZMXZhZuOpnSkm29XVk50BX7t7Zlx8+NJodzVi6/Ly9v/Nd9zIdOTKWw4/vu56/PW4i4goAgA21vFLlran5nBybzcnxuZwcm70qpM5MzmdhaeWqn+lub8nuwa7sGuzK/bsHsnuwM7ua27sHu7K9vzPtrS7R4/oSVwAAfKguL0d+cnw2J8fm8vrYbPP1bE6Nz+X0+FwWlt+Op1KS7X2d2T3UlQf2DObH7+9shNTA2/HU32VBCDYfcQUAwLrNLy7n5NhsI5yuOAP1ejOgpt+xut5Qd1v2Dnfn3p39+fR927NvuDt7h7qzd7g7uwY709Fq6XFuPOIKAIAPZG5hOSfGZnL8/GyOX5jJiQszOXZ+JicuzOaNyfmr9u1s27IaS08cHM7e4cbrxntd6ets26DfBXx4xBUAAKtmF5Zy4sJsjp+fyfELs1cF1JtTVwfUcE97Dox05+O3jeTA1p7sH+nOnmY8jfZ2uGyPW464AgC4xSwtr+T1sdkcPTeT756bztFzMzl2YSbHz8/k7MVLV+27tbc9+0d68ok7tubASHf2b+3JwZGe7BvpzkCXs09wJXEFAHCTmpxdzGvnpnP03HS+e26m+Tyd18dms7hcre63tbc9B7f25IfuGs2Bke4c2NqTAyONM1Eu34MPTlwBANzAlpZXcmp8bvUM1JXPV373U1tLyf6RntyxrTefvm9Hbh/tzW2jPbl9a28GugUU1EFcAQDcAJaWV3L8wmxefetivvPWdF49ezGvvjWdY+dnrlrGfKSnPbeN9uTH7t3eiKfR3tw22pu9Q11pbfG9T/BhElcAAJvI5Yh67Wwjor7zViOijp6fvupSvr3DXblrW18O3z2a20d7m4+eDHa3b+Do4dYmrgAANsDS8kpOjL19Juo7b13Ma2cbl/RdeSbqckR98u5tuXNbb+7a3pfbt/Wku90/42Cz8V8lAMCH7Pz0pbz8xsW8/OZUXmo+v3p2OgtL746oHz40mru29YkouAH5rxUAoCbzi8t57ex0Xn7zYl5+Y6rx/OZUzk+/vbDEtr6O3L2zP5+4Y2sObe/Lndt7c8e2XhEFNwH/FQMAfI+qqsqZyfkrAqoRU0fPz2R5pXFfVEfrlhza0ZdPHtqWu3f2554dfTm0oy8jvR0bPHrgwyKuAADew9LySl47N50XTk/lhTNTeeHMZF56YypT80ur++wZ6srdO/rzmft35O4d/bl7Z18OjPSkZUvZwJED15u4AgBoml9czktvXI6oqbx4ZjIvv3kxl5r3RnW2bcndO/rzkw/uyj3Ns1F37ehLvy/aBSKuAIBb1OTsYl54YzIvnnn7jNRrZ6fTvKov/Z2tuW/XQP7Mx/bn/t0DuW9Xfw5u7fFdUcCaxBUAcNM7d/FSnj89sXpp3zNHZ3Pu3/zu6uc7+jtz367+fOa+Hbl3VyOk9gx1pRSX9QEfnLgCAG4qE7MLef70ZL51ajLfOjWR509N5szk/OrnB7f25ODAlvzZH74j9zVDaqtFJoAaiCsA4IY1fWkp3z7diKhvnZrM86cnc+LC7OrnB7f25LGDw3lg90A+smcw9+7qT29Ha44cOZLDh+/YwJEDNyNxBQDcEOYWlvPiG40zUs+fmsxzpyZy9PxMquY9UrsHu/Lg3oF84bF9eXDPQO7bPZCBLgtNANePuAIANp2l5ZW8/ObFPHtyYjWkXj07vfodUtv7O/LA7sF87qHd+ciegTywe8D3RwEbTlwBABvuzcn5fPP18Tx7ciLffH0iz5+ezNzicpJkqLstH9kzmE/fuz0P7BnMR/YMZHt/5waPGODdxBUAcF3NLizl+VOTqyH17MmJvDnVWHCivWVL7tvdny88vjcP7xvKw3sHrdoH3DDEFQDwoVlZqXL0/MxVZ6Veeevi6uV9+0e688Rtw3l472Ae2jeUe3b2paO1ZYNHDfD9EVcAQG3GZhby7MnxPPv6RL55snFW6uL8UpKkr6M1D+0bzH95z+15eN9gHtwz6D4p4KYirgCA78vKSpXvnpvO10+M55kT4/nGifEcPT+TJNlSkkM7+vNHHtyVh/YO5pF9g7lta2+2bHF5H3DzElcAwAcyu7CUZ09O5BuXY+r1iUzOLSZJhnva88i+oXz+0b15eN9gHtg9kJ4O/8wAbi3+1AMArunMxFyeaYbUMyfG8+IbU6v3St25rTeffWBHHtk3lI/uH8rBrT0WnQBueeIKAMjS8kpeeuNivn5ibPUSvzOTjRX8utpa8tDewfzlH749H90/lEf2DWWg25fzAryTuAKAW9Dk3GK+8fp4njneOCv17MmJ1e+V2jXQmUf2D+W/2D+UR/cP5+6dfWlr2bLBIwbY/MQVANwC3pycz9eOj+XpY2N5+vhYXnnrYqoqadlScu/O/vypx/bmo/sbl/jtGuza6OEC3JDEFQDcZKqqyrHzM3n6+Fi+dmw8Tx8fy+tjs0mS7vaWfHT/UD77wM48emAoD+0dTHe7fw4A1MGfpgBwg1teqfLSG1P5WvOs1NPHx3J+eiFJYxW/xw4M5Wc+vj+PHxzOvTv70+oSP4APhbgCgBvM/OJynjs50Tgzdbyx+MT0pcYX9e4Z6soP3Tmaxw4O57EDw7l91Cp+ANeLuAKATW5qfjHPHB9fvWfqW6cms7C8kiS5a3tvPvfQrjx+cDiPHxzOzgH3SwFsFHEFAJvM2am3F5/42vHxvPzmVKoqad1Scv/ugfzcJw7ksQPDeXT/UIZ62jd6uAA0iSsA2GBvTM7lqaNjeerYhXz16FiOnZ9J0vh+qUf2D+avferOPH5gOA/ts/gEwGbmT2gAuM5OT8zlq9+9kKeOXchTx8Zy4kJjJb++ztY8fmA4X3x8bx4/OJL7dvX7fimAG4i4AoAPUVVVOTU+l68ebZyVeurYhZwan0uSDHS15fGDw/mZjx/IEweHc8/O/rRssfgEwI1KXAFAjaqqyutjs/nq0QvNS/3GcnqiEVND3W154uBI/vwPHMwTB0dy946+bBFTADcNcQUA63D5C3ufOja2GlRvTs0nSUZ62vPEbcP5iz98W544OJI7t/WKKYCbmLgCgO9BVVX57rnp5iV+Y3nq6IWcvXgpSTLa15EnDg7nidtG8vHbhnP7aK/vmAK4hYgrAHgPVVXl1bPTV1zmdyHnpxeSJNv7O/Lx20fyxMGRPHHbcG7b6gt7AW5l4goArrCyUuWVty7mqaONlfyeOjaWsZlGTO0a6MwP3jmaj902nCcOjmT/SLeYAmCVuALglrayUuWlN6cal/kdvZCvHR/LxOxikmTPUFc+eWhbnrhtOB+/bSR7hrrEFABrqiWuSimfSfK/JmlJ8qtVVf3td3z+c0n+XpLTzbd+paqqX63j2ADwvVheqfLimanmF/ZeyNeOjWVqfilJsm+4O5++d/vqZX57hro3eLQA3EjWHVellJYkfz/JjyU5leTpUsqXq6p68R27/rOqqn5+vccDgO/F8kqV505ONO6ZOjaWp4+N5eKlRkwd3NqTzz6wMx+7rRFTOwe6Nni0ANzI6jhz9XiS16qqOpokpZTfTPK5JO+MKwD40C0tr+T505OrS6N/9bXZzP/uf0qS3Dbakz/y0K48cXA4H7ttJNv7Ozd4tADcTOqIq91JTl6xfSrJE9fY74+XUn4oyXeS/NdVVZ28xj4ppTyZ5MkkGR0dzZEjR2oYIjeT6elp84J3MS9uXUsrVY5PreTlseW8MraSV8eXM7/c+GxXT8ljo1Ue2N6Zu4a3ZLAjSS4kExfy0jdezUsbOXA2lD8zWIu5wXpcrwUt/p8kv1FV1aVSyl9M8utJfuRaO1ZV9aUkX0qSQ4cOVYcPH75OQ+RGceTIkZgXvJN5cetYXF7Jt05NNs5KHb2QZ06MZ3ahUVN3be/N5x8bycduG8njB4cz2tdhbnBN5gVrMTdYjzri6nSSvVds78nbC1ckSaqqunDF5q8m+bs1HBeAW8DC0kq+dWpi9TK/rx8fz9xiI6YObe/L5z+6ZzWmRno7Nni0ANzK6oirp5PcWUo5mEZUfSHJf3blDqWUnVVVvdHc/KnElRgAXNulpeXGmanvXshXjzXOTM0vriRJ7t7Rlz/12N587LbhPH5wJMM97Rs8WgB427rjqqqqpVLKzyf5nTSWYv+1qqpeKKX8zSRfr6rqy0n+ainlp5IsJRlL8nPrPS4AN4dLS8t59vW3z0w9c2I8l5ZWUkpy947+fPHxfY2l0Q8OZ0hMAbCJ1XLPVVVVX0nylXe890tXvP4bSf5GHccC4MY2v7icZ5tLo3/16IV88/WJ1Zi6Z0d//vQT+5tnpoYz2C2mALhxXK8FLQC4Rc0vLucbr4/nqaONM1PfPDmRhWZM3berP//5x/Y37pk6MJyB7raNHi4AfN/EFQC1mltYzjdfH2+emRrLsycnsrC8ki0luW/XQH72442YevTAcAa6xBQANw9xBcC6TF9ayjMnxvO1YxfytWONmFpcrrKlJA/sHsjPfeJAPnbbcB49MJz+TjEFwM1LXAHwPRmfWcjTx8fytWNj+drxsbxwZirLK1VatpTcv3sgf+4HDuZjB0fy6IGh9IkpAG4h4gqA9/TW1HyeOjaWp481guqVty4mSdpbt+ThvYP5K4dvz+MHR/LwvsH0dPhrBYBbl78FAVhVVVVOjs3lqeYlfl87PpYTF2aTJD3tLfnogeH81EO78vjB4Xxkz0A6Wls2eMQAsHmIK4Bb2MpKldfOTV91ZurNqfkkyVB3Wx47MJw/87H9eeLgSO7Z2ZfWli0bPGIA2LzEFcAtZGl5JS+9cXH1zNTTx8cyPruYJNne35EnDo7ksYPDeeLgcO4Y7c2WLWWDRwwANw5xBXATu/yFvY3V/MbyzInxTF9aSpLsH+nOj96zPY8fHM4TB0eyd7grpYgpAPh+iSuAm8j56Uv5+vHxPHNiLE8fH88LZyazuFwlSe7a3puffnh3Hjs4nMcPDGfHQOcGjxYAbi7iCuAGVVVVjp6fydePj+Xrx8fz9RPjOXZ+JkljJb8H9wzkL/zgbXnswFAe2TeUwe72DR4xANzcxBXADeLS0nK+fXqqEVMnxvPMifGMzSwkaSw+8dH9w/nCY3vz6IGh3L/bSn4AcL2JK4BNanJ2Mc+83ri875nj43n21EQWllaSJAe39uRH7t6Wxw4M5aP7h3P7aI/7pQBgg4krgE3g8iV+3zgxnm+8PpFnTozlO29NJ0lat5Tct3sgP/Ox/Xm0GVOjfR0bPGIA4J3EFcAGmL60lOdOTjRjajzfPDmRieaS6H0drXlk/1B+6sFd+ej+4Ty0dzBd7S7xA4DNTlwBfMjeeVbqm6+P55W3LqZqLOKXO7b15tP3bs8j+4byyP4h3y8FADcocQVQs4vzi3nu5GS++fo1zkp1tuahvYP58ft25JH9Q3lo72AGuto2eMQAQB3EFcA6rKxUOXZh7bNSd27rzY/fuyOP7B/MI/uGcruzUgBw0xJXAN+Ds1Pzee7UZJ47OZHnTk3kuZMTmZpfStI4K/XwvqF85v4deXifs1IAcKsRVwBruDi/mOdPT+a5k42Y+tapiZyZnE+StGwpuXtHX37ywV15cM+As1IAgLgCSJKFpZW8/OZU84xUI6ZeOze9ennf/pHuPHpgOA/uHcxDewdy784BK/gBAFcRV8At5/J9Ut86NZHnTk7m2ZMTefHMVBaWG1/QO9LTngf3DuYnP7IrD+4dyIN7BjPU077BowYANjtxBdzUVlYay6B/+/Rknm8+XjwzlelLjfukutpa8sCegfzcJw7kwT2DeXDvQHYPdqUUl/cBAN8bcQXcNJZXqhw9N70aUS+cnsoLZyYzs7CcJGlv3ZJ7dvbnjz68Kw/sHsiDewdzx2hvWlu2bPDIAYCbgbgCbkhLyyv57rmZPH96Mt8+PZn/9OJcTv/+72S2GVKdbVty787+/PGP7sn9uwfywO6B3LGtN21CCgD4kIgrYNO7tLScV9+azktvTK1e3vfiG1OZX2zcI9XV1pI9PcmffHTvakjdPtrjjBQAcF2JK2BTGZtZyEtvTOWlN6by4pmpvPjGVF47O52llcayfT3tLblv10C++Pi+PNAMqdtGe/Mf/vAPcvjwfRs8egDgViaugA2xslLlxNhsXjzTDKlmTL05Nb+6z/b+jtyzsz8/cve23LurP/fs7M+BkZ60+C4pAGATElfAh252YSmvvHkxL15xRurlNy+u3h/VsqXkjtHefPz2kdyzsy/37hzIPTv7MtLbscEjBwD44MQVUJvF5ZUcOz+Tl9+8mO+8ebHx/NbFvD42u7pPX0dr7tnVnz/56N7cu7M/9+7qzx3betPZ5gt5AYAbm7gCvmcrK1VOT8zllTcv5pW3LuaVZkR999x0Fpcb90a1bCk5uLUnD+wZyJ/46J4c2tGXe3f2Z8+Q75ACAG5O4gpYU1VVuTCzcNVZqJffvJhX37q4+t1RSbJ7sCuHdvTl8KFtuXtHX+7a3pfbt/Wko9XZKADg1iGugKysVDkzOZfXzk5f/Tg3nYnZxdX9hrrbcmhHXz7/6N7ctb0vh3b05a7tvenrbNvA0QMAbA7iCm4hi8srOXFhNq+dnc53zzUC6tWzF/PdszOZW3z7TNRQd1vu3NaXn7h/Z+7Y1ptD2/ty147ejPZ2uKQPAGAN4gpuQtOXlnL8/MxqQF1+HL8ws3pPVJLsHOjMHdt684XHh3PHtt7cMdqbO7b1WqUPAOD7IK7gBrWwtJKT47M5dm4mx87P5Oj5mRw9N51j52dy9uKl1f22lGT/SE9uH+3Np+7Znju29ebObb25fVtvejv8EQAAUBf/soJNbGWlyptT86vx1AipRkCdHJ/L8srbZ6GGe9pzcGtPfuiu0Rzc2pPbtvbk4GhPDoz0WOYcAOA6EFewwZZXqpyZmMvrY7M5cWE2J8ZmcnJsNkfPzeT4hZnML66s7tvV1pKDW3ty3+6B/JEHd+Xg1p7Vx2B3+wb+LgAAEFdwHcwtLDfjaeaKiJrNybHZnBqfveo+qLaWkr1D3TmwtSefuGNr4yzUaE9u29qb7f0WlAAA2KzEFdSgqqqcn17IyfHZvH7h7TNQrzcj6twV90AlSV9na/aPdOfenf35zP07sn+4O/tGurNvuDs7B7rSskVAAQDcaMQVfAArK1XOT1/KyfG5nBqfzemJuZwabzxON7evvHwvaazEt3e4O4fvGs3+ke7sG+nJ/uHu7B/pzkBXmzNQAAA3GXEFadz3dPbifDOYZnN6/Ip4mpjL6fG5LCxfHU/DPe3ZPdiVu7b35ZOHtmXPUFf2NuNpz1C3RSQAAG4x4oqbXlVVGZ9dzBuTc3ljYr7xPDnffMzlzMR8zkzMZemKlfeSZGtvR3YPdeXeXf359H3bs2ewK3uGurN7qCu7B7vSYxlzAACu4F+H3NCqqsrE7GLOTM7lzcn5nJmcz5urEfV2SF1auvqsU+uWku39ndk12JmH9g7mJz+yM7uHGvG0pxlPzjwBAPC9EFdsWnMLyzl7cT5vTV266vlbr87nS69+dTWe3nmvU8uWkh39ndkx0Jn7dw/k0/ftyI5mSO0Y6Mqugc6M9HZYNAIAgFqJK667K6Ppran5nL14KWebz29d8XxxfuldP9vesiV9bVX2b1vOvbv686m7t2XnYFd2DnRm50Bndg12ZatwAgBgA4granFpaTkXphdyfvpS89F8fXEhF2Yu5dwV4bRWNG3r78j2/s7cua03P3DH1oz2Nba3XfE82N2WP/iDP8jhw5/YgN8lAACsTVxxTVVVZXZheTWWzjUj6fzFtwPqckydm752MCVJT3tLRno7MtrXkUM7+vKDd45mW39HtvV1ZvsVz5YmBwDgRieubhGLyyuZmF3M+OxCxmYWMj6zkLHZhYxNN54b24uN55lGSL3zXqbLBrrasrW3PVt7O3LPrv78YE/j9da+jmzt7chIb3tGexuvu9otCgEAwK1BXN2ALi0tZ3JuMVNzi5mYXWzE0uxCxmbeHU+XY2lqjTNLSdLb0ZqhnrYMd7dnpLc9d27rzXBP+zVjabinPe2tW67j7xYAAG4M4mqDLK9UmZpbzOTcYiaaz5Nzi5mcXWi8N/v2exNXhNTk3GLmFpfX/HU727ZkuLs9Qz3tGe5pz96h7gz3tGeouz3DPW2N96/4fLC7LR2tzi4BAMB6iavv08LSSi7OL+bi/FLzsZip5vOV712cX8rFS1drvUgqAAAO+0lEQVTEUzOS1rpH6bLu9pYMdLWtPvaPdOcje9oy2N2ega629He1ZbD52XBP+2o0uQwPAAA2xi0XV0vLK5lZWM7MpaXGo/n67Ti6MpDeHUqXA+qdX0p7LV1tLenrbE1fZ2sGutqyra8zd27rWw2mwe6rnxuPRjy59A4AAG4smz6uFpZWmhG0lJlLy5m+tJTZhUYYTV9azuzCUqYvh9Kl5av2bezT2J5t/uwHiaKkceaoEUZtjTjqbs+e4e70X36vo/Wqzy8/9zefeztb09YikAAA4FaxqePqxNRK7vrv//UH2ndLSXo6WtPb0Zqejtb0tLekp6M1e7q709vReN14vzU9V2z3drSku7316jDqaE2rMAIAAL4Hmzqu+ttL/vqn77oiiq4Io/ZGBHV3tKS3ozUdrVt8TxIAALBhNnVcDXWW/PyP3LnRwwAAAHhfrn0DAACoQS1xVUr5TCnllVLKa6WUX7jG5x2llH/W/PypUsqBOo4LAACwWaw7rkopLUn+fpKfSHJvki+WUu59x25/Psl4VVV3JPmfk/yd9R4XAABgM6njzNXjSV6rqupoVVULSX4zyefesc/nkvx68/X/leRTxeoTAADATaSOBS12Jzl5xfapJE+stU9VVUullMkkI0nOv/MXK6U8meTJJBkdHc2RI0dqGCI3k+npafOCdzEvWIu5wbWYF6zF3GA9Nt1qgVVVfSnJl5Lk0KFD1eHDhzd2QGw6R44ciXnBO5kXrMXc4FrMC9ZibrAedVwWeDrJ3iu29zTfu+Y+pZTWJANJLtRwbAAAgE2hjrh6OsmdpZSDpZT2JF9I8uV37PPlJD/bfP0nkvx+VVVVDccGAADYFNZ9WWDzHqqfT/I7SVqS/FpVVS+UUv5mkq9XVfXlJP8oyf9ZSnktyVgaAQYAAHDTqOWeq6qqvpLkK+9475eueD2f5PN1HAsAAGAzquVLhAEAAG514goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG4goAAKAG64qrUspwKeXfllJebT4PrbHfcinl2ebjy+s5JgAAwGa03jNXv5Dk31VVdWeSf9fcvpa5qqoeaj5+ap3HBAAA2HTWG1efS/Lrzde/nuSPrvPXAwAAuCGVqqq+/x8uZaKqqsHm65Jk/PL2O/ZbSvJskqUkf7uqqn/xHr/mk0meTJLR0dGP/tZv/db3PT5uTtPT0+nt7d3oYbDJmBesxdzgWswL1mJucC2f/OQnn6mq6tH32+9946qU8ntJdlzjo19M8utXxlQpZbyqqnfdd1VK2V1V1elSym1Jfj/Jp6qq+u77De7QoUPVK6+88n67cYs5cuRIDh8+vNHDYJMxL1iLucG1mBesxdzgWkopHyiuWt9vh6qqfvQ9DvJWKWVnVVVvlFJ2Jjm7xq9xuvl8tJRyJMnDSd43rgAAAG4U673n6stJfrb5+meT/Mt37lBKGSqldDRfb03yiSQvrvO4AAAAm8p64+pvJ/mxUsqrSX60uZ1SyqOllF9t7nNPkq+XUp5L8u/TuOdKXAEAADeV970s8L1UVXUhyaeu8f7Xk/yF5uv/L8kD6zkOAADAZrfeM1cAAABEXAEAANRCXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRAXAEAANRgXXFVSvl8KeWFUspKKeXR99jvM6WUV0opr5VSfmE9xwQAANiM1nvm6ttJ/liSP1xrh1JKS5K/n+Qnktyb5IullHvXeVwAAIBNpXU9P1xV1UtJUkp5r90eT/JaVVVHm/v+ZpLPJXlxPccGAADYTK7HPVe7k5y8YvtU8z0AAICbxvueuSql/F6SHdf46BerqvqXdQ+olPJkkieTZHR0NEeOHKn7ENzgpqenzQvexbxgLeYG12JesBZzg/V437iqqupH13mM00n2XrG9p/neWsf7UpIvJcmhQ4eqw4cPr/Pw3GyOHDkS84J3Mi9Yi7nBtZgXrMXcYD2ux2WBTye5s5RysJTSnuQLSb58HY4LAABw3ax3KfafLqWcSvLxJP+qlPI7zfd3lVK+kiRVVS0l+fkkv5PkpSS/VVXVC+sbNgAAwOay3tUCfzvJb1/j/TNJPnvF9leSfGU9xwIAANjMrsdlgQAAADc9cQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFADcQUAAFCDdcVVKeXzpZQXSikrpZRH32O/46WU50spz5ZSvr6eYwIAAGxGrev8+W8n+WNJ/uEH2PeTVVWdX+fxAAAANqV1xVVVVS8lSSmlntEAAADcoK7XPVdVkt8tpTxTSnnyOh0TAADgunnfM1ellN9LsuMaH/1iVVX/8gMe5weqqjpdStmW5N+WUl6uquoP1zjek0kuB9ilUsq3P+AxuHVsTeISU97JvGAt5gbXYl6wFnODazn0QXZ637iqqupH1zuSqqpON5/PllJ+O8njSa4ZV1VVfSnJl5KklPL1qqrWXCiDW5N5wbWYF6zF3OBazAvWYm5wLR90Ub4P/bLAUkpPKaXv8uskn05jIQwAAICbxnqXYv/pUsqpJB9P8q9KKb/TfH9XKeUrzd22J/mPpZTnknwtyb+qqurfrOe4AAAAm816Vwv87SS/fY33zyT5bPP10SQPfp+H+NL3PzpuYuYF12JesBZzg2sxL1iLucG1fKB5Uaqq+rAHAgAAcNO7XkuxAwAA3NQ2dVyVUv7HUsq3SinPllJ+t5Sya6PHxOZQSvl7pZSXm/Pjt0spgxs9JjZeKeXzpZQXSikrpRQrPd3iSimfKaW8Ukp5rZTyCxs9HjaHUsqvlVLO+qoXrlRK2VtK+fellBebf4/8tY0eE5tDKaWzlPK1UspzzbnxP7zn/pv5ssBSSn9VVVPN1381yb1VVf2lDR4Wm0Ap5dNJfr+qqqVSyt9Jkqqq/tsNHhYbrJRyT5KVJP8wyV+vquoDLZvKzaeU0pLkO0l+LMmpJE8n+WJVVS9u6MDYcKWUH0oyneSfVFV1/0aPh82hlLIzyc6qqr7RXOX6mSR/1J8ZlFJKkp6qqqZLKW1J/mOSv1ZV1Vevtf+mPnN1OayaepJs3hLkuqqq6nerqlpqbn41yZ6NHA+bQ1VVL1VV9cpGj4NN4fEkr1VVdbSqqoUkv5nkcxs8JjaBqqr+MMnYRo+DzaWqqjeqqvpG8/XFJC8l2b2xo2IzqBqmm5ttzceaTbKp4ypJSil/q5RyMsmfTvJLGz0eNqU/l+Rfb/QggE1ld5KTV2yfin8oAR9AKeVAkoeTPLWxI2GzKKW0lFKeTXI2yb+tqmrNubHhcVVK+b1Syrev8fhcklRV9YtVVe1N8k+T/PzGjpbr6f3mRnOfX0yylMb84BbwQeYFAHw/Sim9Sf55kv/qHVdQcQurqmq5qqqH0rhS6vFSypqXFK/re67qUFXVj37AXf9pkq8k+eUPcThsIu83N0opP5fkJ5N8qtrMNw9Sq+/hzwxubaeT7L1ie0/zPYBrat5P88+T/NOqqv7vjR4Pm09VVROllH+f5DNJrrkozoafuXovpZQ7r9j8XJKXN2osbC6llM8k+W+S/FRVVbMbPR5g03k6yZ2llIOllPYkX0jy5Q0eE7BJNRct+EdJXqqq6n/a6PGweZRSRi+vSl1K6UpjoaQ1m2Szrxb4z5McSmP1rxNJ/lJVVf6fR1JKeS1JR5ILzbe+aiVJSik/neR/TzKaZCLJs1VV/fjGjoqNUkr5bJL/JUlLkl+rqupvbfCQ2ARKKb+R5HCSrUneSvLLVVX9ow0dFBuulPIDSf5DkufT+Hdnkvx3VVV9ZeNGxWZQSvlIkl9P4++SLUl+q6qqv7nm/ps5rgAAAG4Um/qyQAAAgBuFuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKiBuAIAAKjB/w8cV2m05IFPywAAAABJRU5ErkJggg==", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot(lambda x: 1 / (1 + math.exp(-x)))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Funkcja logistyczna przyjmuje wartości z przedziału $(0, 1)$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Tangens hiperboliczny" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$$ g(x) = \\tanh x = \\frac{e^{x} - e^{-x}}{e^{x} + e^{-x}} $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Tangens hiperboliczny – wykres" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1cAAAG2CAYAAACTRXz+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd0HOW9h/Hvu7vq1bKKZVtyt9xxw6ZjUw0pQCgBQjc4cOEGktz0nnADaeSShCQQejVOaCZ0AqLjinuVu+Si3tuW9/6htRFGsmVrpFlJz+ecPTuzO9r5kTNH8pPZnTXWWgEAAAAAOsfj9gAAAAAA0BsQVwAAAADgAOIKAAAAABxAXAEAAACAA4grAAAAAHAAcQUAAAAADnAkrowxDxpjio0xa9p5fpYxpsoYsyJ8+6kT+wUAAACASOFz6HUelvQXSY8eYpv3rLVfdGh/AAAAABBRHDlzZa19V1K5E68FAAAAAD1Rd37m6nhjzEpjzCvGmPHduF8AAAAA6HJOvS3wcJZLGmKtrTXGnCvpeUmj2trQGDNP0jxJio2NnZabm9tNI6KnCIVC8ni4Fgs+i+MC7eHYQFs4LtAejg20ZdOmTaXW2ozDbWestY7s0BgzVNK/rbUTOrDtdknTrbWlh9ouLy/Pbty40ZH50Hvk5+dr1qxZbo+BCMNxgfZwbKAtHBdoD8cG2mKMWWatnX647boly40xA4wxJrw8I7zfsu7YNwAAAAB0B0feFmiMeUrSLEnpxphCST+TFCVJ1tq/S7pI0k3GmICkBkmXWqdOmQEAAABABHAkrqy1lx3m+b+o5VLtAAAAANAr8Wk9AAAAAHAAcQUAAAAADiCuAAAAAMABxBUAAAAAOIC4AgAAAAAHEFcAAAAA4ADiCgAAAAAcQFwBAAAAgAOIKwAAAABwAHEFAAAAAA4grgAAAADAAcQVAAAAADiAuAIAAAAABxBXAAAAAOAA4goAAAAAHEBcAQAAAIADiCsAAAAAcABxBQAAAAAOIK4AAAAAwAHEFQAAAAA4gLgCAAAAAAcQVwAAAADgAOIKAAAAABxAXAEAAACAA4grAAAAAHAAcQUAAAAADiCuAAAAAMABxBUAAAAAOIC4AgAAAAAHEFcAAAAA4ADiCgAAAAAcQFwBAAAAgAOIKwAAAABwAHEFAAAAAA4grgAAAADAAcQVAAAAADiAuAIAAAAABxBXAAAAAOAA4goAAAAAHEBcAQAAAIADiCsAAAAAcABxBQAAAAAOIK4AAAAAwAHEFQAAAAA4gLgCAAAAAAcQVwAAAADgAOIKAAAAABxAXAEAAACAA4grAAAAAHAAcQUAAAAADiCuAAAAAMABxBUAAAAAOIC4AgAAAAAHEFcAAAAA4ADiCgAAAAAcQFwBAAAAgAOIKwAAAABwAHEFAAAAAA5wJK6MMQ8aY4qNMWvaed4YY/5kjCkwxqwyxkx1Yr8AAAAAECmcOnP1sKQ5h3j+HEmjwrd5kv7m0H4BAAAAICI4ElfW2ncllR9ik/MkPWpbfCwp1RiT7cS+AQAAACAS+LppP4Mk7Wq1Xhh+bE837R8AAABAL2StVchKwZBVyFoFQ1ZBaxUKhZfD68GQVSikT5ftp8+HPvPYZ19rdFZSh2fprrjqMGPMPLW8dVAZGRnKz893dyBEnNraWo4LfA7HBdrDsYG2cFygPX3x2LDWKmilYEjyh6SAtQqE1OrWsu4Ph4m/jecCISlgJX8oHCe25bHQ/tcOv37IhuMmvL7/uQNRc2Bd4SjSZ7YNHfSzofDzXWnuhOgOb9tdcVUkKafV+uDwY59jrb1P0n2SlJeXZ2fNmtXlw6Fnyc/PF8cFDsZxgfZwbKAtHBdoTyQcG6GQVYM/qPrmoOqbA6prCqoxEFSjP6gmf0iN/v3r4WV/SA3+oJr8wQPr+7c/sE0g9Jnn/cGQmgMhNQVblq3DgeLzGPm8Rj6PJ3zfsuz1GEV5Tfi+Zd3n9SjGYz77Mwf9vNdjFOXxyOs1ivIYeT2eA6/jC697PZLHY+Q1LY97wvcHbsa0PO/Rp8/tf2z/zxx4TAceG5qeoJ929L/b2f8Z27VQ0i3GmPmSZkqqstbylkAAAAD0eNZa1TcHVdMYUE2jX9WNAVU3+lXTGFB9U0B1zcFP78OxVN/c/uP1zcGjmiPa51Gsz6PYKK9io7yKi/IqNsqjmCivUuKiFJsUo9gor2J8HsVEeRTl9Sja51G0t+UWtX+51X3UgXujaJ9HMZ95rO3to7xGxhiH/1fuGRyJK2PMU5JmSUo3xhRK+pmkKEmy1v5d0suSzpVUIKle0rVO7BcAAABwgj8YUkV9s4pqQlq0tUwV9X5VNTSruuHzwVTT6G95vGn/ekDB0OFP/UR7PYqP8Soh2qf4aK/iY3xKiPYqNT5aCTFexUe3rO9/fP99XDiWYsOx9Jlln/dAMHk8fTNoIokjcWWtvewwz1tJNzuxLwAAAOBQgiGr8rpmldQ0qbS2SRX1zaqoa1ZFvV+V9S33FfXNqmx1X9sU+PQFPvj4M69njJQY41NybJSSYlvus1NiNTo2UclxLY8lxX56n9zqPiHGp4Ron+KivYr2OfUtSIhUEXdBCwAAAOBgoZBVWatg2n9rWW9utdyk8rpmtXciKTnWp34J0UqNj1b/xGiNzExUanyU+sVHq198lHZvL9CJ0ye3PJYQ3RJI0T7OCqFDiCsAAAC4yh8MqbimSXurGrW3qlF7qhpa7qsbta+qUXuqGrWvulGBNoopxudRRlKM0hNjNLhfvKbkpiojMUbp4cfSE2OUltASTilxUfJ5D332KL9pu04ald5V/6no5YgrAAAAdKlGf1CFFfXaVdGgwvJ6FVY0aFdFvYoqGrSnqlEltU2fu1pdbJRH2SlxGpAcq5nD0jQgJVZZybHKSIo5EFPpidFKjPH12YsnIPIQVwAAAOgUa61Kapq0tbRO20vrtLNVQBVWNKikpukz20f7PBrcL06DUuM0ZkCyBqTEHrhlp8QqOzlOyXFEE3oe4goAAAAdUtXg17bSOm0rrdW2kjptLa3TtnBQ1bW6fLjXYzQwNVY5/eI1Oy9DOf3ilZMWr5y0OA3uF6+MxBg+w4ReibgCAADAZ5TWNmnT3hpt2lejTcW12ryvRltL6lRW13xgG4+RBveL17D0BB07NE3DMxI0tH+ChqUnKDsl9rCfbQJ6I+IKAACgj6pu9GvDnnBEHbjVqrxVRKXERWl0VqLOGp+lYekJGpaeqGHpLWeiYnxeF6cHIg9xBQAA0AeU1DRp7e4qrd1dfeB+R1n9gecTY3walZWos8ZlaVRWkvKykjQ6K1EZSTF89gnoIOIKAACglymuadSKnZVaXfRpTO2r/vSiErlp8Ro/MFkXTxus8QNTNHpAkgamxBJRQCcRVwAAAD1YUyCodbur9cnOSn2yq1Kf7KxQYUWDpJbPRY3MTNSJI9I1bmCyxg9M0biByUqJi3J5aqB3Iq4AAAB6kH3VjVq8rVzLd1bok52VWre7Ws3BkCRpYEqspuT20zUnDNWU3FSNH5ii2Cg+FwV0F+IKAAAgQllrVVjRoEXbyrV4W5kWbSs/8Dmp2CiPJg1O1bUnDdWUnH6akpuqrORYlycG+jbiCgAAIILsKq/XBwWlWrStXIu2lml3VaMkKTU+SscOTdOVxw3RjGFpGpudrCgudw5EFOIKAADARdWNfn1YUKb3C0r0/uZSbQ+fmUpPjNbMYf114/A0zRiWptGZSXzxLhDhiCsAAIBuFAiGtLKwUu9uKtV7m0u0srBKwZBVfLRXxw/vr6tPGKqTR6VrREYiV+8DehjiCgAAoItVN/r1zsYSvbWhWG9vLFZlvV/GSJMGp+qmU0fo5FHpmpLbT9E+3uYH9GTEFQAAQBfYUVanN9cX6z/r92nxtnIFQlb94qN0Wl6mThubqZNGpis1PtrtMQE4iLgCAABwgLVW6/ZU6+XVe/Tqmr3aUlInqeV7puaePExnjM3S1Nx+8vK5KaDXIq4AAACOkrVWa3e3BNXLq/doe1m9PEaaOay/vjZziE4fm6kh/RPcHhNANyGuAAAAjoC1VmuKqvXS6j16Zc0e7Sirl9djdPzw/pp3ygidPT5L/RNj3B4TgAuIKwAAgA4orKjX858U6dnlRdpaWievx+iEEf1146kjdPb4AUpL4PNTQF9HXAEAALSjutGvV1bv0bPLi7RoW7kkacawNN1wynDNGT9A/QgqAK0QVwAAAK0EgiG9t7lUzywv1Bvr9qkpENLw9AR9+8zROn/KIOWkxbs9IoAIRVwBAABI2lVerwVLd2nB0l3aV92k1PgoffXYHF0wZZAm56Tyhb4ADou4AgAAfZY/GNJ/1hfrqcU79e7mEknSrNEZ+sWXc3XamEy+1BfAESGuAABAn1PWENLvXtugBUsLVVLTpAHJsfrv00bpkumDNbgfb/sDcHSIKwAA0CdYa7V4W7ke+Wi7Xl3TIGmLZudl6rIZuZqVlyGfl7NUADqHuAIAAL1aoz+ohSt266EPt2v9nmqlxEVpztAo/fCSkzhLBcBRxBUAAOiVSmqa9MiH2/XEoh2qqPcrLytJd3xlos6fPEiLPnyPsALgOOIKAAD0KttK63Tfu1v1zPJC+YMhnTE2S9eeOFTHD+/PFf8AdCniCgAA9Aord1Xq7+9s0atr9yrK69GFUwdr3inDNSw9we3RAPQRxBUAAOixrLV6v6BU97xdoI+3lisp1qebTh2ha04cqsykWLfHA9DHEFcAAKDHsdbq3c2luvvNTVq+s1IDkmP1o3PH6rKZuUqM4Z83ANzBbx8AANBjWGv1zqYS/d+bm7ViV6UGpsTqfy+YoIumDVaMz+v2eAD6OOIKAABEPGut8sNRtXJXpQalxunXF0zURdMGK9rH91MBiAzEFQAAiGiLt5XrN69u0LIdFRrcL053fGWiLpxKVAGIPMQVAACISOv3VOt3r23UWxuKlZUcw5kqABGPuAIAABFlV3m97npjk55fUaSkGJ++f84YXX38UMVF85kqAJGNuAIAABGhvK5Zf/rPZj2xaIe8HqMbTx2hG08ZoZT4KLdHA4AOIa4AAICrmgMhPfbxDt395ibVNQd1yfQc3XbGKGUl8z1VAHoW4goAALjCWqu3NhTrf19ar62ldTpldIZ+8oWxGpWV5PZoAHBUiCsAANDtNu6t0e0vrdN7m0s1PCNBD11zrGblZcgY4/ZoAHDUiCsAANBtqur9+v3rG/XEoh1Kio3Sz740TlccN0RRXq4ACKDnI64AAECXs9bqmeVFuuPl9aqob9aVxw3RbWeMVr+EaLdHAwDHEFcAAKBLbdxbo588v0aLt5drSm6qHp07Q+MHprg9FgA4jrgCAABdorYpoLvf3KQHP9iupFiffnPhRF08LUceD5+rAtA7EVcAAMBxr67Zo58tXKt91U269NgcfXfOGKXxFkAAvRxxBQAAHFNc3aifvrBWr67dq7HZyfrbFdM0Nbef22MBQLcgrgAAQKdZa/XPpYW6/aV1agyE9N05ebrh5OFcBRBAn0JcAQCATtlZVq8fPrda7xeUasbQNN1x4USNyEh0eywA6HbEFQAAOCrBkNVDH2zTH17fJK/H6PbzJ+jyGblcsAJAn0VcAQCAI7ajrE7fXrBSS3dU6LQxmbr9/AkamBrn9lgA4CriCgAAdJi1Vk8s2qlfv7xeXo/RXZccowumDJIxnK0CAOIKAAB0yN6qRn3vmVV6Z1OJThqZrt9eNImzVQDQCnEFAAAOyVqrhSt36yfPr1FzMKRfnjdeV8wcwmerAOAgxBUAAGhXVb1fP3x+tV5atUdTc1P1h0sma1h6gttjAUBEIq4AAECbFm8r123zP1FxTZO+c3aebjx1hLycrQKAdhFXAADgMwLBkP70VoH+8tZm5abF69n/OkGTBqe6PRYARDziCgAAHFBYUa/b5q/Q0h0VunDqYP3ivPFKjOGfCwDQER4nXsQYM8cYs9EYU2CM+X4bz19jjCkxxqwI3653Yr8AAMA5/161W+fc/Z427K3R3ZdO1h8uOYawAoAj0OnfmMYYr6R7JJ0pqVDSEmPMQmvtuoM2fdpae0tn9wcAAJzV6A/q5wvXav6SXZqck6o/XTpFuf3j3R4LAHocJ/7vqBmSCqy1WyXJGDNf0nmSDo4rAAAQYbaV1um/nliu9XuqddOsEfrWmaMV5XXkjS0A0OcYa23nXsCYiyTNsdZeH16/UtLM1mepjDHXSLpDUomkTZK+aa3d1c7rzZM0T5IyMjKmLViwoFPzofepra1VYmKi22MgwnBcoD0cG+1bsjegB1Y3yeuRvj4pRpMy+s5bADku0B6ODbRl9uzZy6y10w+3XXf9Fn1R0lPW2iZjzNclPSLptLY2tNbeJ+k+ScrLy7OzZs3qphHRU+Tn54vjAgfjuEB7ODY+rzkQ0q9fXq+HV2zX5JxU3fO1qRqUGuf2WN2K4wLt4dhAZzgRV0WSclqtDw4/doC1tqzV6v2SfuvAfgEAwBEqqmzQzU8s14pdlbr2xKH6wTljFe3jbYAA4AQn4mqJpFHGmGFqiapLJV3eegNjTLa1dk949cuS1juwXwAAcATe2VSiW+d/okDQ6q9fm6pzJ2a7PRIA9CqdjitrbcAYc4uk1yR5JT1orV1rjPmlpKXW2oWSvmGM+bKkgKRySdd0dr8AAKBjrLX6a/4W/f71jcrLStLfrpimYekJbo8FAL2OI5+5sta+LOnlgx77aavlH0j6gRP7AgAAHVfbFNB3/rlSr6zZqy8dM1C/uXCi4qP7zoUrAKA78dsVAIBealtpneY9ulRbSmr14y+M1dyThskY4/ZYANBrEVcAAPRCb23Yp1vnr5DPY/TY3Jk6cWS62yMBQK9HXAEA0IuEQlb3vF2gu97cpHHZyfr7FdOUkxbv9lgA0CcQVwAA9BINzUH9z79W6qVVe3TBlEG64ysTFRvldXssAOgziCsAAHqBfdWNuuHRpVpdVKUfnjtGN5w8nM9XAUA3I64AAOjhVhdW6fpHl6i2MaB/XDldZ4zLcnskAOiTiCsAAHqwl1fv0bcWrFD/hBj966YTNDY72e2RAKDPIq4AAOiBrLX681sFuuuNTZo2pJ/uvXKa0hNj3B4LAPo04goAgB6m0R/Ud/61Si+u3K2vTBmkX3PhCgCICMQVAAA9SElNk254dKlWFlbqe3PG6MZTuXAFAEQK4goAgB6ioLhW1zy0WGW1zfr7FdN09vgBbo8EAGiFuAIAoAdYtLVM8x5bpiiv0dNfP06TBqe6PRIA4CDEFQAAEW7hyt36nwUrlZMWp4evnaGctHi3RwIAtIG4AgAgQllrde+7W3XnKxs0Y1ia7rtymlLjo90eCwDQDuIKAIAIFAiG9PMX1+rxj3fqS8cM1O8vnqQYH1cEBIBIRlwBABBh6psD+u8nP9F/NhTrxlNH6Ltn58nj4YqAABDpiCsAACJISU2T5j6yRGuKqnT7+RN0xXFD3B4JANBBxBUAABFiZ1m9rnxwkYqrm/SPq6br9LFZbo8EADgCxBUAABFg7e4qXfPQEvmDIT15w0xNye3n9kgAgCNEXAEA4LKPtpRp3qNLlRTr01M3HK+RmUlujwQAOArEFQAALnp1zR59Y/4KDUmL16NzZyg7Jc7tkQAAR4m4AgDAJU8u2qkfP79aU3L76YGrp/MdVgDQwxFXAAB0M2ut/vxWge56Y5NOG5Opey6fqrhovsMKAHo64goAgG4UDFn94sW1evSjHbpw6mDdeeFERXk9bo8FAHAAcQUAQDfxB0P69oKVWrhyt75+ynB9/5wxMoYvBwaA3oK4AgCgGzT6g7r5ieX6z4ZifW/OGN00a4TbIwEAHEZcAQDQxWqbArrhkaX6eFuZfnX+BF153BC3RwIAdAHiCgCALlRZ36yrH1qiNUVV+uMlk3X+lEFujwQA6CLEFQAAXaS4plFX3r9Y28rq9PcrpunMcVlujwQA6ELEFQAAXaCwol5X3L9IxTVNeuiaY3XiyHS3RwIAdDHiCgAAh20pqdUV9y9SXVNAj82dqWlD+rk9EgCgGxBXAAA4aO3uKl31wGIZI82fd7zGDUx2eyQAQDchrgAAcMiyHeW65qElSorx6fHrZ2p4RqLbIwEAuhFxBQCAAxZtLdO1Dy9RVnKsHr9+pgalxrk9EgCgmxFXAAB00kdbynTdw0s0MDVWT91wnDKTY90eCQDgAuIKAIBO+LCgVNc9skQ5/eL15A3HKSMpxu2RAAAuIa4AADhKHxSUau4jS5Sb1hJW6YmEFQD0ZR63BwAAoCd6d1OJrnt4iYb2T9BThBUAQJy5AgDgiOVvLNa8x5ZpREainrh+ptISot0eCQAQAThzBQDAEXh7Q7HmPbpMozIT9SRhBQBohTNXAAB00H/W79NNjy/X6AGJenzuTKXGE1YAgE9x5goAgA54Y90+3fj4Mo3JTtITc48jrAAAn8OZKwAADuO1tXt1y5PLNW5gih69boZS4qLcHgkAEIE4cwUAwCG8snqPbn5iuSYMStFjcwkrAED7iCsAANrx0qo9uuWpT3RMTqoevW6GkmMJKwBA+3hbIAAAbXhx5W7d9vQKTclJ1cPXzVBiDH8yAQCHxpkrAAAO8sKKIt06/xNNy+1HWAEAOoy/FgAAtPLcJ4X69oKVOnZomh685lglEFYAgA7iLwYAAGHPLCvU//xrpY4b1l8PXDNd8dH8mQQAdBx/NQAAkLRg6S5975lVOnFEuv5x1XTFRXvdHgkA0MMQVwCAPu/pJTv1/WdX66SRLWEVG0VYAQCOHBe0AAD0aU8t3qnvPbNaJ4/KIKwAAJ1CXAEA+qzHP96hHzy7WrPzMnTfldMIKwBAp/C2QABAn/ToR9v10xfW6vQxmfrrFVMV4yOsAACdQ1wBAPqchz/Ypp+/uE5njM3SPV+bQlgBABxBXAEA+pQH3t+mX/17nc4en6U/XzZV0T7eIQ8AcAZxBQDoM+5/b6tuf2m9zpkwQH+6bIqivIQVAMA5xBUAoE+4950tuuOVDfrCxGz936WTCSsAgOMc+ctijJljjNlojCkwxny/jedjjDFPh59fZIwZ6sR+AQDoiL/mF+iOVzboi5OydTdhBQDoIp3+62KM8Uq6R9I5ksZJuswYM+6gzeZKqrDWjpT0R0m/6ex+AQDoiIVbmvXbVzfqvMkD9X9fnSwfYQUA6CJO/IWZIanAWrvVWtssab6k8w7a5jxJj4SX/yXpdGOMcWDfAAC06+43N+vZzX5dMGWQ7rqEsAIAdC0nPnM1SNKuVuuFkma2t421NmCMqZLUX1LpwS9mjJknaZ4kZWRkKD8/34ER0ZvU1tZyXOBzOC5wsOc2N+uFLX7NyLT6UmaF3nv3HbdHQgThdwbaw7GBzoi4C1pYa++TdJ8k5eXl2VmzZrk7ECJOfn6+OC5wMI4L7Get1R/f2KQXthToommDdW56uU6bPdvtsRBh+J2B9nBsoDOceH9EkaScVuuDw4+1uY0xxicpRVKZA/sGAOAAa61+//pG/emtAn11eo5+e+EkeXgXOgCgmzgRV0skjTLGDDPGREu6VNLCg7ZZKOnq8PJFkt6y1loH9g0AgKSWsPrNqxt1z9tbdNmMXN3xlYnyeAgrAED36fTbAsOfobpF0muSvJIetNauNcb8UtJSa+1CSQ9IeswYUyCpXC0BBgCAI6y1uuOVDbrv3a264rhc/fLLEwgrAEC3c+QzV9balyW9fNBjP2213CjpYif2BQBAa9Za/e9L63X/+9t01fFD9IsvjxcXpAUAuCHiLmgBAEBHWWv1y3+v00MfbNc1JwzVz740jrACALiGuAIA9EjWWv3ixXV6+MPtuu7EYfrJF8cSVgAAVxFXAIAeJxSy+tnCtXrs4x264eRh+uG5hBUAwH3EFQCgRwmFrH78who9uWinvn7qcH1/zhjCCgAQEYgrAECPEQpZ/ej51Xpq8S7dNGuEvnt2HmEFAIgYxBUAoEcIhax+8OxqPb10l26ZPVLfPms0YQUAiCjEFQAg4gVDVt/510o9u7xI3zhtpL55JmEFAIg8xBUAIKIFgiF9+58r9cKK3frmGaN16xmj3B4JAIA2EVcAgIjlD4Z029Mr9NKqPfrO2Xm6efZIt0cCAKBdxBUAICI1B0K6df4nemXNXv3w3DGad8oIt0cCAOCQiCsAQMRpCgR1y5Of6I11+/STL47T3JOGuT0SAACHRVwBACJKoz+o/3piud7aUKxfnjdeVx0/1O2RAADoEOIKABAxGv1BzXtsmd7dVKJfXzBRl8/MdXskAAA6jLgCAESEhuagbnh0qT7YUqrfXjhJlxyb4/ZIAAAcEeIKAOC6uqaA5j6yRIu3lev3Fx2jC6cNdnskAACOGHEFAHBVbVNA1z60WMt2VOiPX52s8yYPcnskAACOCnEFAHBNdaNf1zy4WCsLq/Sny6boi5MGuj0SAABHjbgCALiiqt6vqx5arLVFVfrLZVN0zsRst0cCAKBTiCsAQLcrq23SlQ8sVkFxrf76tak6a/wAt0cCAKDTiCsAQLfaW9Wor93/sYoqG/SPq6fr1NEZbo8EAIAjiCsAQLfZVV6vr92/SGW1TXrk2hmaOby/2yMBAOAY4goA0C22ltTqa/cvUl1TQE/ccJwm56S6PRIAAI4irgAAXW7D3mpdcf9iWWs1f97xGjcw2e2RAABwHHEFAOhSqworddWDixXj8+iJ64/XyMxEt0cCAKBLEFcAgC6zZHu5rn1oiVLjo/Tk9ccpt3+82yMBANBliCsAQJd4f3Opbnh0qbJTYvXEDTOVnRLn9kgAAHQp4goA4Lg31u3TzU8s1/CMBD02d6YykmLcHgkAgC5HXAEAHPXMskJ995lVmjAwWY9cN0Op8dFujwQAQLcgrgAAjrn/va26/aX1OnFkf9175XQlxvBnBgDQd/BXDwDQadZa/eH1TfrL2wU6Z8IA/d+lkxXj87o9FgAA3Yq4AgB0SjBk9ZMX1ujJRTt12Ywc3X7+RHk9xu2xAADodsQVAOCoNQdC+uaCFXpp1R7dNGuEvnt2nowhrAAAfRNxBQA4KnVNAd34+DK9t7lUPzp3rG44ZbjbIwEA4CriCgBwxCrqmnXtw0tUfiR5AAAZp0lEQVS0qrBSv71oki6ZnuP2SAAAuI64AgAckb1VjbrygUXaUV6vv10xTWePH+D2SAAARATiCgDQYQXFtbr6wcWqavDr4WuP1Qkj0t0eCQCAiEFcAQA6ZOn2cl3/6FL5PB49dcNxmjg4xe2RAACIKMQVAOCwXl2zR9+Yv0KDU+P0yHUzlJMW7/ZIAABEHOIKAHBID3+wTb/49zpNyUnV/Vcfq7SEaLdHAgAgIhFXAIA2hUJWv3l1g+59d6vOGpeluy+dorhor9tjAQAQsYgrAMDnNAWC+u6/VumFFbt15XFD9PMvj5fXw5cDAwBwKMQVAOAzqhv9+vqjy/TR1jJ9d06ebjp1hIwhrAAAOBziCgBwQFFlg+Y+vEQFxbW665Jj9JWpg90eCQCAHoO4AgBIklbuqtTcR5aqyR/UQ9ceq5NHZbg9EgAAPQpxBQDQy6v36FsLVig9MUZP3TBTo7KS3B4JAIAeh7gCgD7MWqu/5m/R717bqKm5qbrvqulKT4xxeywAAHok4goA+qjmQEg/eHa1nlleqC8fM1C/vWiSYqO41DoAAEeLuAKAPqiirllff3yZFm8r121njNKtp4/iioAAAHQScQUAfczWklpd9/AS7a5s1N2XTtZ5kwe5PRIAAL0CcQUAfcj7m0t185PL5fMYPTVvpqYNSXN7JAAAeg3iCgD6AGutHnh/m3798nqNykzSP66artz+8W6PBQBAr0JcAUAv1+gP6ofPrdazy4t09vgs3XXJZCXE8OsfAACn8dcVAHqxvVWN+vpjS7WysErfPGO0/vu0kfJ4uHAFAABdgbgCgF5q2Y4K3fj4MtU3BXTvldN09vgBbo8EAECvRlwBQC+0YMku/fj5NRqQEqvH585U3oAkt0cCAKDXI64AoBdp9Af184VrNX/JLp00Ml1/uXyKUuOj3R4LAIA+gbgCgF5iV3m9bnpimdYUVevm2SP0rTPz5OXzVQAAdBviCgB6gbc3Fuu2+SsUslb/uGq6zhyX5fZIAAD0OZ2KK2NMmqSnJQ2VtF3SJdbaija2C0paHV7daa39cmf2CwBoEQxZ3f2fzfrzW5s1ZkCy/n7FVA3pn+D2WAAA9EmdPXP1fUn/sdbeaYz5fnj9e21s12CtndzJfQEAWimva9at8z/Re5tLddG0wbr9/AmKjfK6PRYAAH1WZ+PqPEmzwsuPSMpX23EFAHDQ4m3l+sZTn6i8rll3fGWiLj02R8bw+SoAANxkrLVH/8PGVFprU8PLRlLF/vWDtgtIWiEpIOlOa+3zh3jNeZLmSVJGRsa0BQsWHPV86J1qa2uVmJjo9hiIMH3luAhZqxe3+PV8gV+Z8UY3HROjoSmcrTqUvnJs4MhwXKA9HBtoy+zZs5dZa6cfbrvDnrkyxrwpqa1vnvxR6xVrrTXGtFdqQ6y1RcaY4ZLeMsasttZuaWtDa+19ku6TpLy8PDtr1qzDjYg+Jj8/XxwXOFhfOC6Kqxt129Mr9OGWep03eaD+94KJSozhukSH0xeODRw5jgu0h2MDnXHYv8rW2jPae84Ys88Yk22t3WOMyZZU3M5rFIXvtxpj8iVNkdRmXAEAPu+dTSX61tMrVN8c1G8vmqSLpw3mbYAAAEQYTyd/fqGkq8PLV0t64eANjDH9jDEx4eV0SSdKWtfJ/QJAn+APhnTnKxt09YOLlZ4Yo4W3nKhLpvP5KgAAIlFn309yp6QFxpi5knZIukSSjDHTJd1orb1e0lhJ9xpjQmqJuTuttcQVABxGQXGtvvn0Cq0uqtJlM3L1sy+N42qAAABEsE7FlbW2TNLpbTy+VNL14eUPJU3szH4AoC+x1uqxj3fo1y+vV1yUV3+/YqrmTMh2eywAAHAYfBIaACJIcXWjvvOvVXpnU4lOHZ2h3100SZnJsW6PBQAAOoC4AoAI8eqavfrBs6tU3xzUL88bryuPG8JnqwAA6EGIKwBwWXWjX796cZ3+uaxQEwel6I9fnayRmXzHCgAAPQ1xBQAuentDsX7w7GoV1zTqltkj9Y3TRyna19kLuQIAADcQVwDggqp6v37573V6ZnmhRmcl6t4rT9QxOalujwUAADqBuAKAbvbGun360XOrVVbXrP8+baRuOW2kYnxcYh0AgJ6OuAKAblJR16xfvLhWz6/YrTEDkvTgNcdqwqAUt8cCAAAOIa4AoItZa/X8iiLd/u/1qmrw69bTR+nm2SP5bBUAAL0McQUAXWhrSa1+8sIafVBQpsk5qXrsgokaNzDZ7bEAAEAXIK4AoAs0BYL6e/5W3ZNfoBifR786f4Iun5Err4fvrQIAoLcirgDAYR9tKdOPnl+trSV1+uKkbP30i+OUmRzr9lgAAKCLEVcA4JB91Y2685UNeu6TIuWkxenha4/VrLxMt8cCAADdhLgCgE5qCgT14Pvb9ee3NisQtLp59gjdMnuU4qK5vDoAAH0JcQUAnfDWhn365YvrtL2sXmeOy9KPvzBWQ/onuD0WAABwAXEFAEdha0mtfvXvdXp7Y4mGZyToketm6NTRGW6PBQAAXERcAcARKK9r1p/f2qzHP96hGJ9XP/7CWF11/FC+swoAABBXANARjf6gHvxgm/729hbVNQf01WNz9M0zRysziasAAgCAFsQVABxCMGT17PJC3fXGJu2patQZYzP1vTljNCorye3RAABAhCGuAKAN1lq9s6lEd76yQRv21uiYwSn641cn67jh/d0eDQAARCjiCgAO8uGWUt31+iYt3VGh3LR4/eXyKfrCxGwZY9weDQAARDDiCgDClm4v1x9e36SPtpZpQHKsfnX+BH11eg4XqwAAAB1CXAHo81bsqtRdb2zSu5tKlJ4Yo59+cZwun5mr2Ci+BBgAAHQccQWgz1q6vVz3vF2gtzeWqF98lH5wzhhddfxQxUUTVQAA4MgRVwD6FGut3t1cqnveLtDibeVKS4jWd87O09UnDFViDL8SAQDA0eNfEgD6hFDI6rW1e3VPfoHWFFUrOyVWP/vSOF16bC5nqgAAgCOIKwC9WqM/qOc+KdL9723VlpI6DUtP0G8vnKTzpwziQhUAAMBRxBWAXqm4plGPf7RDjy/aqfK6Zo3LTtZfLp+icyZky+vhkuoAAMB5xBWAXmX9nmo98P42LVyxW/5QSKePydLck4bpuOFpfE8VAADoUsQVgB7PHwxpyd6A7vvHx/pwS5niory6dEaOrj1xmIalJ7g9HgAA6COIKwA91p6qBj21eJfmL96p4pomDUr16HtzxujyGblKiY9yezwAANDHEFcAepRQyOr9glI9/vEO/WdDsULW6tTRGbosoVrfuGg2n6cCAACuIa4A9AhFlQ16Zlmh/rWsUDvL65WWEK0bTh6ur83MVU5avPLz8wkrAADgKuIKQMRq9Af12tq9+ufSQn2wpVTWSieM6K9vnzVacyYMUIyP76cCAACRg7gCEFGstVq+s0LPLi/SwpW7VdMY0OB+cbr19FG6cOpg5aTFuz0iAABAm4grAK6z1mr9nhotXLlbL67craLKBsVGeXTOhGxdPG2wjhveXx7e8gcAACIccQXANdtL67Rw5W4tXLlbBcW18nqMTh6Vrm+fNVpnjstSUixX/AMAAD0HcQWgW20vrdNra/fq5dV7tLKwSpI0Y1iabj9/gs6dmK20hGiXJwQAADg6xBWALmWt1Zqiar2+bq9eW7tXm/bVSpImDkrRj84dqy8ek63slDiXpwQAAOg84gqA4wLBkBZvL9fra/fpjXX7VFTZII9pOUP1sy+N01njB2hQKkEFAAB6F+IKgCOKqxuVv6lE72ws0XubS1TdGFCMz6OTR2XotjNG6fSxWbzlDwAA9GrEFYCjEgiGtHxnpfI3Fit/Y4nW7amWJGUmxWjOhAGanZepU0ZnKCGGXzMAAKBv4F89ADrEWqstJbX6cEuZPiwo04dbSlXdGJDXYzQtt5++c3aeZudlamx2kozhsukAAKDvIa4AtMlaq13lDfpwS6k+2lqmD7eUqaSmSZI0KDVOcyYM0Ky8TJ04Ml0pcVwyHQAAgLgCIEkKhVrOTC3dUaGl2yv08dYyFVU2SJIykmJ0/PD+OmFEf50wIl05aXGcnQIAADgIcQX0UY3+oFYXVWnp9got3V6uZTsrVFnvlySlJUTr2KH99PVTh+uEEf01IiORmAIAADgM4groA0Ihq62ldVpVWKlVhVVaWViptUXVag6GJEnDMxJ01rgsTR+apulD+mlYegIxBQAAcISIK6CXsdaqqLLhQESt2lWlNUVVqmkKSJLio72aMDBF15w4VNOH9NO0If3UPzHG5akBAAB6PuIK6MGaAyEVFNdqw95qrd9TrfV7arR+T7XK6polSVFeo7HZyTpvykBNGpyqYwanamRmorwezkoBAAA4jbgCegBrrYprmrRpX81nImpLSa38QStJivZ5NDorUaeNydSkwSmaNDhVY7KTFOPzujw9AABA30BcAREkEAxpV0WDCoprD9y2lNRqS3Htgbf1SVJWcozGZidr9phMjRmQpHHZyRqWniCf1+Pi9AAAAH0bcQV0M2utSmubtbO8TttL67WjrE4FJbXaUlynbaV1By4yIUmZSTEakZGo86cM0sjMRI3KTNSY7GSlJUS7+F8AAACAthBXQBcIBEPaXdmoHeV12lFWr53lLRG1f7m+OXhgW4+RctPiNTIzUbPyMjQiM1EjMxM1IiORL+cFAADoQYgr4CjUNPq1u7JRuysbVFTZoN0Hbo0qqmzQ3upGBUP2wPbRPo9y+sVpaP8EHT+iv4akxWtI/wTl9o/X4H5xfC4KAACgFyCugFZCIauyumbtq25USU2TimsaVVzdpL3VjdpT9WlM1TQGPvNzPo/RgJRYDUyN04xhaRqYGqvctHjlpiVoSP94DUiOlYcr9AEAAPRqxBV6PWutqhsCKqtrUnlds8rqmlVW26zimkbtq25SSU2jimuatK+6UaW1zZ8547RfanyUBqbEaXC/eM0clqaBqXEHboNS45SRFMPlzQEAAPo44go9irVWjYGWL8mtqverqsGvivqWYCqvbVZ5XVPLcvhWVtesirpmBdoIJknqnxCtzORYZSbFKC8rSZnJMcoKr2ck7b+PUWwUb9sDAADAoRFX6FbWWjUFQqptCqi2MdBy3xRQTWNAlfXNqmrwq7qhJZoqw/dVDf4DIVXV4G8JpTffavP1k2N96p8Yo7SEaOWkxWtyTqrSEqKVlhCt/onR6p8Qc2A5PTFGUVy6HAAAAA4hrnBIwZBVfXNADf6gGptDqvcH1NAcbLn5g6pvDqq+uSWO9gdT3UHr+wNq/3p7Z5H2M0ZKivEpNT5aKXFRSomL0sDUuAPLZXt2aur4MQfWU+NbYqlffLSifcQSAAAA3NGpuDLGXCzp55LGSpphrV3aznZzJN0tySvpfmvtnZ3Zb19krVUgZNUcCKkpEFJTIHhguTm83uQPqSkYUpM/pOZgSE3+YKvnP/2ZBv9n46gxfN/Qejm8TevvXOqI+GivEmN8LbfYlvuchHglhdcTws8lxfqUEN3y2P7nUuKilBoXrcRY3yE/v5Sfv1ezZuR29n9SAAAAwFGdPXO1RtJXJN3b3gbGGK+keySdKalQ0hJjzEJr7bpO7rtN1lqFbMsZl1A4SIIhq1CoZTlkW9YP3Oynz7X+mdBB2wSCVv5gSP6gVSAUvg+GPveYPxhq2TYU+uzPBEMtcRQMtSwHrfyhz79GINiyTXOrKNofT4c54dMhUV6j2Civ4qO9iovyKi7ap7goj+KjfeoXH624aK/io7yKiw7fovZv13IfH+1VbKvluChvSzCFY4mLOgAAAKCv6lRcWWvXS5Ixh/wH9QxJBdbareFt50s6T9Jh46qoNqTT/pDfEjrWKhgM34ekYCgUjiEpEAopFFL4OQcKpJM8RvJ5PYryGEX5PPJ5PIryGvm8RlFej6I8Hvm8Rj6vR9FeI5/Ho9io8HPhx2O8HsVEeRTt9SgmyqsY3/5lj2J8XkX7PIrxHbzsCS97W/2sRzHeT9e5HDgAAADQNbrjM1eDJO1qtV4oaWZHfjDKYzQ2O1leY+TzGHk8Rl5j5PWG7z2f3jzGyOuRvB5P+LnwskfyhH/e2/o1PAfdTMtzvna28Xn2x09LGO0PpNZBtH8bzt4AAAAAfc9h48oY86akAW089SNr7QtOD2SMmSdpniRlZGTo4oHVnXvB/R8ZCh75j4Uk+Tu3d3SB2tpa5efnuz0GIgzHBdrDsYG2cFygPRwb6IzDxpW19oxO7qNIUk6r9cHhx9rb332S7pOkvLw8O2vWrE7uHr1Nfn6+OC5wMI4LtIdjA23huEB7ODbQGd1x3eolkkYZY4YZY6IlXSppYTfsFwAAAAC6TafiyhhzgTGmUNLxkl4yxrwWfnygMeZlSbLWBiTdIuk1SeslLbDWru3c2AAAAAAQWTp7tcDnJD3XxuO7JZ3bav1lSS93Zl8AAAAAEMm6422BAAAAANDrEVcAAAAA4ADiCgAAAAAcQFwBAAAAgAOIKwAAAABwAHEFAAAAAA4grgAAAADAAcQVAAAAADiAuAIAAAAABxBXAAAAAOAA4goAAAAAHEBcAQAAAIADiCsAAAAAcABxBQAAAAAOIK4AAAAAwAHEFQAAAAA4gLgCAAAAAAcQVwAAAADgAOIKAAAAABxAXAEAAACAA4grAAAAAHAAcQUAAAAADiCuAAAAAMABxBUAAAAAOIC4AgAAAAAHEFcAAAAA4ADiCgAAAAAcQFwBAAAAgAOIKwAAAABwAHEFAAAAAA4grgAAAADAAcQVAAAAADiAuAIAAAAABxBXAAAAAOAA4goAAAAAHEBcAQAAAIADiCsAAAAAcABxBQAAAAAOIK4AAAAAwAHEFQAAAAA4gLgCAAAAAAcQVwAAAADgAOIKAAAAABxAXAEAAACAA4grAAAAAHAAcQUAAAAADiCuAAAAAMABxBUAAAAAOIC4AgAAAAAHEFcAAAAA4ADiCgAAAAAcQFwBAAAAgAOIKwAAAABwAHEFAAAAAA4grgAAAADAAcQVAAAAADiAuAIAAAAABxBXAAAAAOCATsWVMeZiY8xaY0zIGDP9ENttN8asNsasMMYs7cw+AQAAACAS+Tr582skfUXSvR3Ydra1trST+wMAAACAiNSpuLLWrpckY4wz0wAAAABAD9Vdn7mykl43xiwzxszrpn0CAAAAQLc57JkrY8ybkga08dSPrLUvdHA/J1lri4wxmZLeMMZssNa+287+5knaH2BNxpg1HdwH+o50SbzFFAfjuEB7ODbQFo4LtIdjA23J68hGh40ra+0ZnZ3EWlsUvi82xjwnaYakNuPKWnufpPskyRiz1Frb7oUy0DdxXKAtHBdoD8cG2sJxgfZwbKAtHb0oX5e/LdAYk2CMSdq/LOkstVwIAwAAAAB6jc5eiv0CY0yhpOMlvWSMeS38+EBjzMvhzbIkvW+MWSlpsaSXrLWvdma/AAAAABBpOnu1wOckPdfG47slnRte3irpmKPcxX1HPx16MY4LtIXjAu3h2EBbOC7QHo4NtKVDx4Wx1nb1IAAAAADQ63XXpdgBAAAAoFeL6LgyxvzKGLPKGLPCGPO6MWag2zMhMhhjfmeM2RA+Pp4zxqS6PRPcZ4y52Biz1hgTMsZwpac+zhgzxxiz0RhTYIz5vtvzIDIYYx40xhTzVS9ozRiTY4x52xizLvx35Fa3Z0JkMMbEGmMWG2NWho+NXxxy+0h+W6AxJtlaWx1e/oakcdbaG10eCxHAGHOWpLestQFjzG8kyVr7PZfHgsuMMWMlhSTdK+l/rLUdumwqeh9jjFfSJklnSiqUtETSZdbada4OBtcZY06RVCvpUWvtBLfnQWQwxmRLyrbWLg9f5XqZpPP5nQFjjJGUYK2tNcZESXpf0q3W2o/b2j6iz1ztD6uwBEmRW4LoVtba1621gfDqx5IGuzkPIoO1dr21dqPbcyAizJBUYK3daq1tljRf0nkuz4QIYK19V1K523Mgslhr91hrl4eXayStlzTI3akQCez/t3fHoDqFcRzHvz83pIxM3GKQTSwmg6LcJDebsshkMJgMblHqrlLmqww3pa7BcA2UwUIWRTFYdBkodQeZ6G94X3XTfd2Lw3N0v586w3N6ht/w9L7nf85z/mfg03C4fniMrEl6XVwBJJlOsgCcAi61zqNeOgPcax1CUq9sAxaWjN/ihZKkVUiyA9gHPGmbRH2RZCzJM+ADcL+qRq6N5sVVkgdJXixzTAJU1VRVjQOzwLm2afUvrbQ2hnOmgC8M1ofWgNWsC0mSfkeSzcAccP6HHVRaw6rqa1XtZbBTan+SkVuK/+g7V12oqsOrnDoLzAOX/2Ic9chKayPJaeAYcKj6/PKgOvULvxla294B40vG24fnJGlZw/dp5oDZqrrTOo/6p6oWkzwEJoBlm+I0f3L1M0l2LRlOAq9aZVG/JJkALgDHq+pz6zySeucpsCvJziQbgJPA3caZJPXUsGnBDPCyqq62zqP+SLL1e1fqJJsYNEoaWZP0vVvgHLCbQfevN8DZqvLOo0jyGtgIfByeemwnSSU5AVwHtgKLwLOqOtI2lVpJchS4BowBN6pqunEk9UCSW8BBYAvwHrhcVTNNQ6m5JAeAR8BzBtedABerar5dKvVBkj3ATQb/JeuA21V1ZeT8PhdXkiRJkvS/6PW2QEmSJEn6X1hcSZIkSVIHLK4kSZIkqQMWV5IkSZLUAYsrSZIkSeqAxZUkSZIkdcDiSpIkSZI6YHElSZIkSR34Bikt02UAw3ESAAAAAElFTkSuQmCC", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot(lambda x: math.tanh(x))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Tangens hiperboliczny przyjmuje wartości z przedziału $(-1, 1)$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Powstaje z funkcji logistytcznej przez przeskalowanie i przesunięcie." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Daje szybszą zbieżność niż funkcja logistyczna dzięki temu, że przedział wartości jest symetryczny względem zera." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Ze względu na ograniczony przedział wartości, obie funkcje (logistyczna i $\\tanh$) są podatne na problem zanikającego gradientu." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Problem zanikającego gradientu (*vanishing gradient problem*)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Sigmoidalne funkcje aktywacji ograniczają wartości na wyjściach neuronów do niewielkich przedziałów ($(-1, 1)$, $(0, 1)$ itp.).\n", "* Jeżeli sieć ma wiele warstw, to podczas propagacji wstecznej mnożymy przez siebie wiele małych wartości → obliczony gradient jest mały.\n", "* Im więcej warstw, tym silniejszy efekt zanikania." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Sposoby na zanikający gradient\n", "\n", "* Modyfikacja algorytmu optymalizacji (*RProp*, *RMSProp*)\n", "* Użycie innej funckji aktywacji (ReLU, softplus)\n", "* Dodanie warstw *dropout*\n", "* Nowe architektury (LSTM itp.)\n", "* Więcej danych, zwiększenie mocy obliczeniowej" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### ReLU (*Rectifier Linear Unit*)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$$ g(x) = \\max(0, x) $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### ReLU – wykres" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1cAAAG2CAYAAACTRXz+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAGJhJREFUeJzt3X+sZOd91/HPN/6RRQk0AS+NYzvEpdalpkBLLNcRVbkmTnFMFTelkWwhkQDVtgirBQmBi0UCgUitkABBItpVY9VJo6QWqZul2eIkda7dCNzaiZzWXmfdxQR2t4lN7CT0KmnM1g9/7Gx7vZ659+7Oc2fOzLxe0tXOj7PzPH98vfbb58zZaq0FAACA6bxk3hsAAABYBuIKAACgA3EFAADQgbgCAADoQFwBAAB0IK4AAAA66BJXVXVnVT1dVY9OeH+9qr5WVY+Mft7RY10AAIChuLDT5/x8kvckef82x/x6a+0HOq0HAAAwKF3OXLXWHkjybI/PAgAAWESz/M7V66vqc1X1q1X152e4LgAAwJ7rdVngTj6b5M+01jar6qYkv5zkqnEHVtWBJAeSZN++fa97zWteM6Mtsiief/75vOQl7sXCC5kLJjEbjGMukme+0fJ7/6/lWy6uvHJfzXs7g2E2GOeJJ574cmtt/07HVWuty4JV9dokv9Ja+85dHPuFJNe01r683XFra2vt6NGjXfbH8tjY2Mj6+vq8t8HAmAsmMRuMs8pz0VrLP//oo/mFB/93fvSvfltuv/HPpUpcnbHKs8FkVfWZ1to1Ox03kyyvqlfV6J/aqrp2tO4zs1gbAIDThBXsrS6XBVbVh5KsJ7mkqk4keWeSi5KktfYzSX44yd+vqlNJvpHkltbrlBkAADsSVrD3usRVa+3WHd5/T07fqh0AgBkTVjAbvq0HALDEhBXMjrgCAFhSwgpmS1wBACwhYQWzJ64AAJaMsIL5EFcAAEtEWMH8iCsAgCUhrGC+xBUAwBIQVjB/4goAYMEJKxgGcQUAsMCEFQyHuAIAWFDCCoZFXAEALCBhBcMjrgAAFoywgmESVwAAC0RYwXCJKwCABSGsYNjEFQDAAhBWMHziCgBg4IQVLAZxBQAwYMIKFoe4AgAYKGEFi0VcAQAMkLCCxSOuAAAGRljBYhJXAAADIqxgcYkrAICBEFaw2MQVAMAACCtYfOIKAGDOhBUsB3EFADBHwgqWh7gCAJgTYQXLRVwBAMyBsILlI64AAGZMWMFyElcAADMkrGB5iSsAgBkRVrDcxBUAwAwIK1h+4goAYI8JK1gN4goAYA8JK1gd4goAYI8IK1gt4goAYA8IK1g94goAoDNhBatJXAEAdCSsYHWJKwCAToQVrDZxBQDQgbACxBUAwJSEFZCIKwCAqQgr4AxxBQBwnoQVsJW4AgA4D8IKOJu4AgA4R8IKGEdcAQCcA2EFTCKuAAB2SVgB2xFXAAC7IKyAnYgrAIAdCCtgN8QVAMA2hBWwW+IKAGACYQWcC3EFADCGsALOlbgCADiLsALOh7gCANhCWAHnS1wBAIwIK2Aa4goAIMIKmJ64AgBWnrACehBXAMBKE1ZAL+IKAFhZwgroqUtcVdWdVfV0VT064f2qqv9QVceq6req6i/3WBcA4HwJK6C3Xmeufj7Jjdu8/6YkV41+DiT5T53WBQA4Z8IK2AsX9viQ1toDVfXabQ65Ocn7W2styYNV9YqqurS19sUe6wMA7FZrLR848lzuOy6sgL5m9Z2ry5Ic3/L8xOg1AICZOXPG6r7jp4QV0F2XM1c9VdWBnL50MPv378/GxsZ8N8TgbG5umgtexFwwidngjD86Y3UqN1zWct2+L+X++5+a97YYGH9mMI1ZxdXJJFdseX756LUXaa0dTHIwSdbW1tr6+vqeb47FsrGxEXPB2cwFk5gNkq1nrE5fCnjdvi/l+uuvn/e2GCB/ZjCNWV0WeCjJ3x7dNfC6JF/zfSsAYBbcvAKYlS5nrqrqQ0nWk1xSVSeSvDPJRUnSWvuZJIeT3JTkWJKvJ/k7PdYFANiOsAJmqdfdAm/d4f2W5B/0WAsAYDeEFTBrs7osEABgZoQVMA/iCgBYKsIKmBdxBQAsDWEFzJO4AgCWgrAC5k1cAQALT1gBQyCuAICFJqyAoRBXAMDCElbAkIgrAGAhCStgaMQVALBwhBUwROIKAFgowgoYKnEFACwMYQUMmbgCABaCsAKGTlwBAIMnrIBFIK4AgEETVsCiEFcAwGAJK2CRiCsAYJCEFbBoxBUAMDjCClhE4goAGBRhBSwqcQUADIawAhaZuAIABkFYAYtOXAEAcyesgGUgrgCAuRJWwLIQVwDA3AgrYJmIKwBgLoQVsGzEFQAwc8IKWEbiCgCYKWEFLCtxBQDMjLAClpm4AgBmQlgBy05cAQB7TlgBq0BcAQB7SlgBq0JcAQB7RlgBq0RcAQB7QlgBq0ZcAQDdCStgFYkrAKArYQWsKnEFAHQjrIBVJq4AgC6EFbDqxBUAMDVhBSCuAIApCSuA08QVAHDehBXAHxFXAMB5EVYALySuAIBzJqwAXkxcAQDnRFgBjCeuAIBdE1YAk4krAGBXhBXA9sQVALAjYQWwM3EFAGxLWAHsjrgCACYSVgC7J64AgLGEFcC5EVcAwIsIK4BzJ64AgBcQVgDnR1wBAH9IWAGcP3EFACQRVgDTElcAgLAC6EBcAcCKE1YAfYgrAFhhwgqgH3EFACtKWAH0Ja4AYAUJK4D+xBUArBhhBbA3xBUArBBhBbB3usRVVd1YVUer6lhV3T7m/bdX1f+pqkdGPz/SY10AYPeEFcDeunDaD6iqC5K8N8kbk5xI8lBVHWqtHTnr0F9srd027XoAwLkTVgB7r8eZq2uTHGutPdlaey7Jh5Pc3OFzAYAOhBXAbEx95irJZUmOb3l+Isn3jDnub1bV9yV5Isk/aq0dH3NMqupAkgNJsn///mxsbHTYIstkc3PTXPAi5oJJVn02Wmv5wJHnct/xU7npyoty3b4v5f77n5r3tuZu1eeCycwG0+gRV7vxX5J8qLX2zar60SR3Jflr4w5srR1McjBJ1tbW2vr6+oy2yKLY2NiIueBs5oJJVnk2zpyxuu+4M1ZnW+W5YHtmg2n0uCzwZJIrtjy/fPTaH2qtPdNa++bo6c8leV2HdQGACVwKCDB7PeLqoSRXVdWVVXVxkluSHNp6QFVduuXpm5M83mFdAGAMYQUwH1NfFthaO1VVtyW5N8kFSe5srT1WVe9K8nBr7VCSH6+qNyc5leTZJG+fdl0A4MWEFcD8dPnOVWvtcJLDZ732ji2PfzLJT/ZYCwAYT1gBzFeXv0QYAJgvYQUwf+IKABacsAIYBnEFAAtMWAEMh7gCgAUlrACGRVwBwAISVgDDI64AYMEIK4BhElcAsECEFcBwiSsAWBDCCmDYxBUALABhBTB84goABk5YASwGcQUAAyasABaHuAKAgRJWAItFXAHAAAkrgMUjrgBgYIQVwGISVwAwIMIKYHGJKwAYCGEFsNjEFQAMgLACWHziCgDmTFgBLAdxBQBzJKwAloe4AoA5EVYAy0VcAcAcCCuA5SOuAGDGhBXAchJXADBDwgpgeYkrAJgRYQWw3MQVAMyAsAJYfuIKAPaYsAJYDeIKAPaQsAJYHeIKAPaIsAJYLeIKAPaAsAJYPeIKADoTVgCrSVwBQEfCCmB1iSsA6ERYAaw2cQUAHQgrAMQVAExJWAGQiCsAmIqwAuAMcQUA50lYAbCVuAKA8yCsADibuAKAcySsABhHXAHAORBWAEwirgBgl4QVANsRVwCwC8IKgJ2IKwDYgbACYDfEFQBsQ1gBsFviCgAmEFYAnAtxBQBjCCsAzpW4AoCzCCsAzoe4AoAthBUA50tcAcCIsAJgGuIKACKsAJieuAJg5QkrAHoQVwCsNGEFQC/iCoCVJawA6ElcAbCShBUAvYkrAFaOsAJgL4grAFaKsAJgr4grAFaGsAJgL4krAFaCsAJgr4krAJaesAJgFrrEVVXdWFVHq+pYVd0+5v2XVtUvjt7/jap6bY91AWAnwgqAWZk6rqrqgiTvTfKmJFcnubWqrj7rsL+X5CuttW9P8u+S/PS06wLATlpr+cCR54QVADNxYYfPuDbJsdbak0lSVR9OcnOSI1uOuTnJvxg9/s9J3lNV1Vpr233ws7/f8q9/5ch2h7CCjp/4Zj69aS54IXPBOF945uu57/gpYQXATPSIq8uSHN/y/ESS75l0TGvtVFV9LcmfSvLlsz+sqg4kOZAkF3/rn80v/Pf/2WGLLJOWljpuLnghc8E4VcmNV7Rct+9Luf/+p+a9HQZkc3MzGxsb894GA2Q2mEaPuOqqtXYwycEkWVtba59/99+Y844Ymo2Njayvr897GwyMuWASs8E45oJJzAbT6HFDi5NJrtjy/PLRa2OPqaoLk3xLkmc6rA0AADAIPeLqoSRXVdWVVXVxkluSHDrrmENJ3jZ6/MNJ7tvp+1YAAACLZOrLAkffobotyb1JLkhyZ2vtsap6V5KHW2uHkrwvyQeq6liSZ3M6wAAAAJZGl+9ctdYOJzl81mvv2PL495O8tcdaAAAAQ9TlLxEGAABYdeIKAACgA3EFAADQgbgCAADoQFwBAAB0IK4AAAA6EFcAAAAdiCsAAIAOxBUAAEAH4goAAKADcQUAANCBuAIAAOhAXAEAAHQgrgAAADoQVwAAAB2IKwAAgA7EFQAAQAfiCgAAoANxBQAA0IG4AgAA6EBcAQAAdCCuAAAAOhBXAAAAHYgrAACADsQVAABAB+IKAACgA3EFAADQgbgCAADoQFwBAAB0IK4AAAA6EFcAAAAdiCsAAIAOxBUAAEAH4goAAKADcQUAANCBuAIAAOhAXAEAAHQgrgAAADoQVwAAAB2IKwAAgA7EFQAAQAfiCgAAoANxBQAA0IG4AgAA6EBcAQAAdCCuAAAAOhBXAAAAHYgrAACADsQVAABAB+IKAACgA3EFAADQgbgCAADoQFwBAAB0IK4AAAA6EFcAAAAdiCsAAIAOxBUAAEAH4goAAKCDqeKqqv5kVX2iqn5n9OsrJxz3B1X1yOjn0DRrAgAADNG0Z65uT/JrrbWrkvza6Pk432itfdfo581TrgkAADA408bVzUnuGj2+K8kPTvl5AAAAC6laa+f/m6u+2lp7xehxJfnKmednHXcqySNJTiX5qdbaL2/zmQeSHEiS/fv3v+7uu+8+7/2xnDY3N/Pyl7983ttgYMwFk5gNxjEXTGI2GOf666//TGvtmp2O2zGuquqTSV415q07kty1Naaq6iuttRd976qqLmutnayqb0tyX5I3tNb+x06bW1tba0ePHt3pMFbMxsZG1tfX570NBsZcMInZYBxzwSRmg3GqaldxdeFOB7TWbthmkaeq6tLW2her6tIkT0/4jJOjX5+sqo0k351kx7gCAABYFNN+5+pQkreNHr8tyUfPPqCqXllVLx09viTJX0lyZMp1AQAABmXauPqpJG+sqt9JcsPoearqmqr6udEx35Hk4ar6XJJP5fR3rsQVAACwVHa8LHA7rbVnkrxhzOsPJ/mR0eP/luQvTLMOAADA0E175goAAICIKwAAgC7EFQAAQAfiCgAAoANxBQAA0IG4AgAA6EBcAQAAdCCuAAAAOhBXAAAAHYgrAACADsQVAABAB+IKAACgA3EFAADQgbgCAADoQFwBAAB0IK4AAAA6EFcAAAAdiCsAAIAOxBUAAEAH4goAAKADcQUAANCBuAIAAOhAXAEAAHQgrgAAADoQVwAAAB2IKwAAgA7EFQAAQAfiCgAAoANxBQAA0IG4AgAA6EBcAQAAdCCuAAAAOhBXAAAAHYgrAACADsQVAABAB+IKAACgA3EFAADQgbgCAADoQFwBAAB0IK4AAAA6EFcAAAAdiCsAAIAOxBUAAEAH4goAAKADcQUAANCBuAIAAOhAXAEAAHQgrgAAADoQVwAAAB2IKwAAgA7EFQAAQAfiCgAAoANxBQAA0IG4AgAA6EBcAQAAdCCuAAAAOhBXAAAAHYgrAACADqaKq6p6a1U9VlXPV9U12xx3Y1UdrapjVXX7NGsCAAAM0bRnrh5N8kNJHph0QFVdkOS9Sd6U5Ookt1bV1VOuCwAAMCgXTvObW2uPJ0lVbXfYtUmOtdaeHB374SQ3JzkyzdoAAABDMovvXF2W5PiW5ydGrwEAACyNHc9cVdUnk7xqzFt3tNY+2ntDVXUgyYEk2b9/fzY2NnovwYLb3Nw0F7yIuWASs8E45oJJzAbT2DGuWms3TLnGySRXbHl++ei1SesdTHIwSdbW1tr6+vqUy7NsNjY2Yi44m7lgErPBOOaCScwG05jFZYEPJbmqqq6sqouT3JLk0AzWBQAAmJlpb8X+lqo6keT1ST5WVfeOXn91VR1OktbaqSS3Jbk3yeNJ7m6tPTbdtgEAAIZl2rsF3pPknjGv/26Sm7Y8P5zk8DRrAQAADNksLgsEAABYeuIKAACgA3EFAADQgbgCAADoQFwBAAB0IK4AAAA6EFcAAAAdiCsAAIAOxBUAAEAH4goAAKADcQUAANCBuAIAAOhAXAEAAHQgrgAAADoQVwAAAB2IKwAAgA7EFQAAQAfiCgAAoANxBQAA0IG4AgAA6EBcAQAAdCCuAAAAOhBXAAAAHYgrAACADsQVAABAB+IKAACgA3EFAADQgbgCAADoQFwBAAB0IK4AAAA6EFcAAAAdiCsAAIAOxBUAAEAH4goAAKADcQUAANCBuAIAAOhAXAEAAHQgrgAAADoQVwAAAB2IKwAAgA7EFQAAQAfiCgAAoANxBQAA0IG4AgAA6EBcAQAAdCCuAAAAOhBXAAAAHYgrAACADsQVAABAB+IKAACgA3EFAADQgbgCAADoQFwBAAB0IK4AAAA6EFcAAAAdiCsAAIAOxBUAAEAH4goAAKCDqeKqqt5aVY9V1fNVdc02x32hqn67qh6pqoenWRMAAGCILpzy9z+a5IeS/Owujr2+tfblKdcDAAAYpKniqrX2eJJUVZ/dAAAALKhZfeeqJfl4VX2mqg7MaE0AAICZ2fHMVVV9Msmrxrx1R2vto7tc53tbayer6k8n+URVfb619sCE9Q4kORNg36yqR3e5BqvjkiQuMeVs5oJJzAbjmAsmMRuMs7abg3aMq9baDdPupLV2cvTr01V1T5Jrk4yNq9bawSQHk6SqHm6tTbxRBqvJXDCOuWASs8E45oJJzAbj7PamfHt+WWBVvayq/viZx0m+P6dvhAEAALA0pr0V+1uq6kSS1yf5WFXdO3r91VV1eHTYtyb5dFV9LslvJvlYa+2/TrMuAADA0Ex7t8B7ktwz5vXfTXLT6PGTSf7SeS5x8Px3xxIzF4xjLpjEbDCOuWASs8E4u5qLaq3t9UYAAACW3qxuxQ4AALDUBh1XVfWvquq3quqRqvp4Vb163ntiGKrq31TV50fzcU9VvWLee2L+quqtVfVYVT1fVe70tOKq6saqOlpVx6rq9nnvh2Goqjur6ml/1QtbVdUVVfWpqjoy+vfIT8x7TwxDVe2rqt+sqs+NZuNfbnv8kC8LrKo/0Vr7v6PHP57k6tbaj815WwxAVX1/kvtaa6eq6qeTpLX2T+e8Leasqr4jyfNJfjbJP26t7eq2qSyfqrogyRNJ3pjkRJKHktzaWjsy140xd1X1fUk2k7y/tfad894Pw1BVlya5tLX22dFdrj+T5Af9mUFVVZKXtdY2q+qiJJ9O8hOttQfHHT/oM1dnwmrkZUmGW4LMVGvt4621U6OnDya5fJ77YRhaa4+31o7Oex8MwrVJjrXWnmytPZfkw0lunvOeGIDW2gNJnp33PhiW1toXW2ufHT3+vSSPJ7lsvrtiCNppm6OnF41+JjbJoOMqSarq3VV1PMnfSvKOee+HQfq7SX513psABuWyJMe3PD8R/6EE7EJVvTbJdyf5jfnuhKGoqguq6pEkTyf5RGtt4mzMPa6q6pNV9eiYn5uTpLV2R2vtiiQfTHLbfHfLLO00G6Nj7khyKqfngxWwm7kAgPNRVS9P8pEk//CsK6hYYa21P2itfVdOXyl1bVVNvKR4qr/nqofW2g27PPSDSQ4neecebocB2Wk2qurtSX4gyRvakL88SFfn8GcGq+1kkiu2PL989BrAWKPv03wkyQdba7807/0wPK21r1bVp5LcmGTsTXHmfuZqO1V11ZanNyf5/Lz2wrBU1Y1J/kmSN7fWvj7v/QCD81CSq6rqyqq6OMktSQ7NeU/AQI1uWvC+JI+31v7tvPfDcFTV/jN3pa6qP5bTN0qa2CRDv1vgR5Ks5fTdv/5Xkh9rrfk/j6SqjiV5aZJnRi896E6SVNVbkvzHJPuTfDXJI621vz7fXTEvVXVTkn+f5IIkd7bW3j3nLTEAVfWhJOtJLknyVJJ3ttbeN9dNMXdV9b1Jfj3Jb+f0f3cmyT9rrR2e364Ygqr6i0nuyul/l7wkyd2ttXdNPH7IcQUAALAoBn1ZIAAAwKIQVwAAAB2IKwAAgA7EFQAAQAfiCgAAoANxBQAA0IG4AgAA6EBcAQAAdPD/AVNAgTNkBNHAAAAAAElFTkSuQmCC", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot(lambda x: max(0, x))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### ReLU – zalety\n", "* Mniej podatna na problem zanikającego gradientu (*vanishing gradient*) niż funkcje sigmoidalne, dzięki czemu SGD jest szybciej zbieżna.\n", "* Prostsze obliczanie gradientu.\n", "* Dzięki zerowaniu ujemnych wartości, wygasza neurony, „rozrzedzając” sieć (*sparsity*), co przyspiesza obliczenia." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### ReLU – wady\n", "* Dla dużych wartości gradient może „eksplodować”.\n", "* Za dużo „wygaszonych” neuronów może mieć negatywny wpływ na przepływ gradientu do poprzedzających warstw." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Inne funkcje aktywacji\n", "\n", "W literaturze zaproponowano też inne funkcje aktywacji, które są lepsze od ReLU pod niektórymi względami:\n", "* softplus\n", "* ELU\n", "* leaky ReLU\n", "* SiLU / swish\n", "\n", "W praktyce jednak ReLU pozostaje najpopularniejszą funkcją aktywacji ze względu na jej prostotę, skuteczność i szybkość obliczania." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## 10.3. Wielowarstwowe sieci neuronowe\n", "\n", "czyli _Artificial Neural Networks_ (ANN) lub _Multi-Layer Perceptrons_ (MLP)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "![Rys. 9.5. Wielowarstwowa sieć neuronowa](nn1.png \"Rys. 9.5. Wielowarstwowa sieć neuronowa\")" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Architektura sieci\n", "\n", "* Sieć neuronowa jako graf neuronów. \n", "* Organizacja sieci przez warstwy.\n", "* Najczęściej stosowane są sieci jednokierunkowe i gęste.\n", "* $n$-warstwowa sieć neuronowa ma $n+1$ warstw (nie liczymy wejścia).\n", "* Rozmiary sieci określane poprzez liczbę neuronów lub parametrów." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Sieć neuronowa jednokierunkowa (*feedforward*)\n", "\n", "* Mając daną $n$-warstwową sieć neuronową oraz jej parametry $\\Theta^{(1)}, \\ldots, \\Theta^{(L)} $ oraz $\\beta^{(1)}, \\ldots, \\beta^{(L)} $, liczymy:

\n", "$$a^{(l)} = g^{(l)}\\left( a^{(l-1)} \\Theta^{(l)} + \\beta^{(l)} \\right). $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "![Rys. 11.6. Wielowarstwowa sieć neuronowa - feedforward](nn2.png \"Rys. 11.6. Wielowarstwowa sieć neuronowa - feedforward\")" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Funkcje $g^{(l)}$ to tzw. **funkcje aktywacji**.
\n", "Dla $i = 0$ przyjmujemy $a^{(0)} = \\mathrm{x}$ (wektor wierszowy cech) oraz $g^{(0)}(x) = x$ (identyczność)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Parametry $\\Theta$ to wagi na połączeniach miedzy neuronami dwóch warstw.
\n", "Rozmiar macierzy $\\Theta^{(l)}$, czyli macierzy wag na połączeniach warstw $a^{(l-1)}$ i $a^{(l)}$, to $\\dim(a^{(l-1)}) \\times \\dim(a^{(l)})$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Parametry $\\beta$ zastępują tutaj dodawanie kolumny z jedynkami do macierzy cech.
Macierz $\\beta^{(l)}$ ma rozmiar równy liczbie neuronów w odpowiedniej warstwie, czyli $1 \\times \\dim(a^{(l)})$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **Klasyfikacja**: dla ostatniej warstwy $L$ (o rozmiarze równym liczbie klas) przyjmuje się $g^{(L)}(x) = \\mathop{\\mathrm{softmax}}(x)$.\n", "* **Regresja**: pojedynczy neuron wyjściowy jak na obrazku. Funkcją aktywacji może wtedy być np. funkcja identycznościowa." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Pozostałe funkcje aktywacji najcześciej mają postać sigmoidy, np. sigmoidalna, tangens hiperboliczny.\n", "* Mogą mieć też inny kształt, np. ReLU, leaky ReLU, maxout." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Uczenie wielowarstwowych sieci neuronowych" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Mając algorytm SGD oraz gradienty wszystkich wag, moglibyśmy trenować każdą sieć." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Niech:\n", "$$\\Theta = (\\Theta^{(1)},\\Theta^{(2)},\\Theta^{(3)},\\beta^{(1)},\\beta^{(2)},\\beta^{(3)})$$\n", "\n", "* Funkcja sieci neuronowej z grafiki:\n", "\n", "$$\\small h_\\Theta(x) = \\tanh(\\tanh(\\tanh(x\\Theta^{(1)}+\\beta^{(1)})\\Theta^{(2)} + \\beta^{(2)})\\Theta^{(3)} + \\beta^{(3)})$$\n", "* Funkcja kosztu dla regresji:\n", "$$J(\\Theta) = \\dfrac{1}{2m} \\sum_{i=1}^{m} (h_\\Theta(x^{(i)})- y^{(i)})^2 $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Jak obliczymy gradienty?\n", "\n", "$$\\nabla_{\\Theta^{(l)}} J(\\Theta) = ? \\quad \\nabla_{\\beta^{(l)}} J(\\Theta) = ?$$\n", "\n", "* Postać funkcji kosztu zależna od wybranej architektury sieci oraz funkcji aktywacji." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "$$\\small J(\\Theta) = \\frac{1}{2}(a^{(L)} - y)^2 $$\n", "$$\\small \\dfrac{\\partial}{\\partial a^{(L)}} J(\\Theta) = a^{(L)} - y$$\n", "\n", "$$\\small \\tanh^{\\prime}(x) = 1 - \\tanh^2(x)$$" ] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.6" }, "livereveal": { "start_slideshow_at": "selected", "theme": "white" } }, "nbformat": 4, "nbformat_minor": 4 }