umz21/wyk/09_Sieci_neuronowe.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Uczenie maszynowe – zastosowania\n",
    "# 9. Sieci neuronowe – wprowadzenie"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "# Przydatne importy\n",
    "\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 9.1. Perceptron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAUDBAQEAwUEBAQFBQUGBwwIBwcHBw8LCwkMEQ8SEhEPERETFhwXExQaFRERGCEYGh0dHx8fExciJCIeJBweHx7/2wBDAQUFBQcGBw4ICA4eFBEUHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh7/wAARCAFoAeADASIAAhEBAxEB/8QAHAAAAQUBAQEAAAAAAAAAAAAAAgABAwQFBgcI/8QAQBAAAQMCBAQEBAQEBQQCAwEAAQACEQMEBRIhMQZBUWETInGBMpGhsRRCwdEzcuHwFSM1UmIHJDTxFiVDc4KS/8QAFwEBAQEBAAAAAAAAAAAAAAAAAAECA//EAB8RAQEBAAMBAQEAAwAAAAAAAAABEQIhMUESA1Fhcf/aAAwDAQACEQMRAD8A+V+BwPxrpaCYOp9CureXaguJ3hclwU7LfuA3g/YrqLioZiYMla/nfUjncWotddkuEGdx6KKlUc3Y6LYr0W1CTlBPU6rKrsNN+XMPklrVS062sk6hTi4AEaKgXBoA0JPNO1rid1jEaBbSqfm0Ttsxqaen6qhlqU/hkI2XFdpHmIHRbgvi0uPyyVHUo1cvmaY5hFQxKq0QQDHdWqOJWzmEVKYLvX+i1RnPzZA12gGwUfPdBdVKlS4c5phpOgB5KE+INZJUE2cdULniRBUeQ/7k4bEyZ9lGUgcnB01MqMuA5FIeVNxYkZueSeVHm5z8k4Pcpi9pmu1UrHESoWMdzBU9KmNcxWrRIC7eVPRzuqNJEkmFEzIByWxw1a/jL6lSI8oPSeSzYOy4dwOnWwhr6wIJAI26Kapw9Qnyz8x+y37em23tKdu0A5QBI05IhHRTExzJwF7fNTkDpIRMw2uwEEH6LpQlDSdQCrjeOcNlWH5SVGKdRv5SupDW8mj5IatrTcPhHyTErmDUdTHmHoibWJZK234dSdyB9tlE/DaewMeyYjOY2TAUotyYJVwWZbrP0Sqt8JmZxEd9ExcV2UgBCMU2tOiYVKbiMtRuvdEGkiQUxQuAMRyTZQnynZI7wSgAU+cckgORGyIN0nMJ6SkQOsR7qgZhDB6Qj9pShSmonjbWU2VSmdNJQz/x+qJgA3RRVGjaSPRWS0BAaYkH9EwxULREAT6qM0SQW5dlcNITM/RJzRuNExZ0z6tmyo0B7ZjZUq+EUHknIdfRbRbqT1Q6piuYueG6ZHlza9x+yz6/DDmz4ebvJBXakb7+ijMtB826tjH5rga3D1y0GKZPuFn1cJumHzUyB7L0p7QRBbKrPotcZcwEdws4n5sedULGr+Lpscx0Eie2qbHfE/ECi1phgH2/qu/qWlGQ5tNoInUAKjc4XQqPLnMaSecJhY8883OU+c8l11bA6ZGjRPZqpV8BjVojf8qYrnvGe34SgEyPMtmpgdYCZ0/lVR2F3DdcrjH/ABKliVRqbnXkVj1Pjd6roKljXEkscBB5FYNZpbVc08j0Qje4K/1B3odfYrpq85yBrBOq53gj/wA2p6H7FdHXacwiVvh/xZFd5dOhhZeJ0XGqCFseGUxYHCDspyi2Obe1zSMx2Th4bzVjFAadYADQkqoHDnopDEza1TdyNlam4+bSFExw2KIU2vBPRVF+lTt6gGUwVI7D9Za4ALObTeD5dFM2tXpgDUkK7oa5piiYLpOqr5gTupbqo+4LS9sFsx/fsogyPi2VvQHxD1SLi7mUfl6pBzW891kwABPWFKKbolyA1nflCFniO3lTVzEzSwSHJFzOQSp27jJJUoo0xuVfTs4qE7KalTqVD5dhukH0W6t1KMV3uBFPbmkX8p6dvp5jBXacAWjTd+IANCfsuKp06lQg7L0jgXy03NHKfsru+n5dPUHnJTKQaDuUxTpcJgkqQMTtGqlpt0MrSYANEJy1TZQmLSh+UIbvCjdTM8lO5pQEIflAWnos7HQRYudzAMfJazws3Gmh1m6eh+yzhrh6l1cU3EtcYlGMZvKZnPI9B+yC7pNDzvuqjqYTMGpR4gqT5xr7K7Sx6lPmaZ9AucyhRua6dFMHYMxO0cRlcrdKvSqguYZ6rgTm3zJCpUafK6FTHoQjklmXB0r6uwaO+itUsbuGkFxB9ioWOxOuybKVzdLiI/maD6Aq7Rx+3Il5IPoUwjZAkpOZsqlPErWoYFQD1BVlteg5py1GmeS1pp8qiIRgtI0LT7ojAGYbpqq5aEBbqpy4uMnRCWyUnYgLVG9sx2U5TZU1rVYwEBAhWyFEW6dFEs1Ve0aQoiw9Fbe0yEDWnVGbNUTSHJA+l5DIk8lo+H6oalPbQoflk+DmHmbISNu127WrT8P1QlpUp+dZbrGm5j8zBo0kfJeVYuzJidw0DQPK9oLf8qp/KfsV4zj3+sXX8/6LOM2Y2OBR/wB5UG+h+xXUVW6jTquZ4C81/U9D9iuvq0xA06rf8pe1ioGpvDEaKx4Y6IQpb2tZl5ZNrOB2ieUrDvKbaNUsIiOey657TIA5rFxbD31qudsDX9E0Ype2fKPqjzugZZ7wUFWj4Jh5gpmF0ExoUZT0q7huPmVcpXlOTnYJPNZ7HCdVNT8N2k6pBbuKlAsDmRImQOaz31C53wkBXGWpcCQ7RQPa1ji0nUK3wRNoucdCfkpG2+X4nD3QsuCBA3QuNSoR5oUaTFtJuhcJTmu3XK0KJlAjVxlSBlNpkqUMK1R5Ia0iOikFOo8SSUIrU26NGpRis/8ALsrBPSotafO4DsrDKtKmCGtBncyqrbeo4y5xA9VZt7emXBuaT6rWKtUHvewva0gDkvQeAHZsxI6/ZcC53hsFICF3n/T7Z3v9lB1h3idkQahRt3ToSN3CmpDdQt3Cno7FaBZURGiSMhBXqjZAWqZ42TZSrggLTCzsZbFo70K0jmWfjU/hHT0P2URw99/ELQPdVCQTGX6q7fD/ADlTjSVKI35Z+FRE9BClq7D3UJUUHh/8pSyjl9VIUCtCEc2j5KF0O3aplHlPZZiVF4eUmJEpZT1KkLTolCoHxntPlkehhXMOfVc8/wCY7lzKpOiVpYS0F59kE1e6uaJ8tVxA7lOzHbhplzSR0k/slfsGu/yWe9rXDQqUjVZxBqfEpkdPN/RWqOO2rvidlPrK5p9IAGXBRZN4Wortad/bOOlRvzCnFalUHle0xvqNFwYzt+Go75ohc3VH4arhKhruiW/7m/NAYXGNxS6b/wDkJ+Sm/wAfuG/Fr7BB1T3NBEqMPaJ5z1XOU+ImkxUaR7BWaeOWrh5pB9v3QbXit5uhMajTsQVlf4nbO2cUTLyk4nI4nqg1RlPID3URaOqqsuWxMpn3WiVdWnNHhVNd2n7LxXiNsY1dD/n+gXrgvPK+dRBXkXEL82NXLur/ANApjFra/wCnn+pv/l/Qrtq5jn1XIf8ATG2qXOKvbTBJykbdivQjhFVzocYg8wtcPGYxTBOqY06g0a1bzcLpU58R7QR1A/dO6pYW4zP8M+pCxybYLbe4cQG0ySVctcJuKzoecg9QrdbHMNps8vhyOjgsDEeKnUnk25BE8qn9EhLilxlw+63q+NTJcABzHUrlB5SQ4QVv4jxBcXlMsdIB0nNKwqjSXZiTJM7KwO1zVIxrZl26hACdpdzkrfxKt0szT5dVFc03OfnKemX6kEgDklVqVHDRsrFZwNJzWndTuusoGXT2VdrQFICyPMAVWyNSrUOYbJNtzGYoxWgRTaD6JEVCeYHRA9NtFurjqpRVgRT+yha1gMuM9lMxzCIY0BA/nc8wMoVy08v+YdS3UKmxrnOlzp9lMamVnhgSDodUFku8Wqap1nT+/kvQP+n+hLeUn7Lzui7QQ2IXofAZEggyTOvsg7H8xHdSsURIzEbnqpGHsgVV7msJjmjtKjnEZuUJn+dpbtKKizLqD05LQuZ9ks3VQwXHeFI/zdk6+kIbk9U6ZrdDrKRTYqFyzsZ/8R3ofstBxWdjP/hu9D9kRxVzs71VR+war1yzQ+bc9Fnl2pMaKURvHlKjA0Uhe10gaxumDRG+nWFAJaUvDcpWAkZuSL2VsEGUhQ5R0Culp6BRGl/y+ikmCvlTFh5KYsg7omtHVBWewzotDCJFQ6dFWeBqeiuYZ5czomY0QSYi4kR3VIq5dnMfhA91TPyUoApkTghWofEZpjugNP191OfUoXtJ/MVFRZFSqUzCvlp/3FQ1GnbKrLjNZdam6Rz3UDmFajqe8jbbRV6zQOSajNzPBJBOqB1atTIOaFYfTEaE6KJ1MEHNr0lZaC3ELhpkO+isUsWujWbTcQQT0/qqj6bY2j2RWdPNesaBMlIldvhtNrrQ1Hblk/ReV49/q91/+z9F7BZUQywaDzZ07LyLiVobjd00D8/6BSo6T/pTeCzxlzjsQeXYru8XxxzWOdTJ3J0AXl3BNQU8Vk9D9iusu7tjnOERuun851SRXveIr5z3QXR6BZFW+vLgQ6o6FfLml5JVd+WZlYs2txSNOpBzPJnugdQkdT6qxWgjyndCHNyQSkiYg8ODoEi0xqFIbhoCgq1gTIWvyBdI/KgGaUznEpK3pKlaXAHoVI13lEbqsjZv7LNNIvnZFSaXSTsN0zhomDdTKmmrLXU2jQSUBqvcPLsoY9VIWtnoVTTjM4jqp2U6kSBCibLNQN0RqOI00Q1Yp+U6pjJeSlnzJDdXFWaWogL0LgJzZEmNT9l5zTXoP/T5zvEbJ6/ZLMV3JiTCJnJA/fedUgVExYbujZsoQ4qRjt1vBOzf2UihDk+YqWEShMVEXFKVMNA5Z2Nf+I4dQfstByzMb/8AEd6FUclcnzws8tlhI5BaF0qZdAJ017KUkc1eXlSjcuY10AmPqrvj1KeHeM52v/tFd4f41YVDOhJUmIWwdhzaY5Hl7qfcXA4deCtSLqhIy8kbL+3qPc1riC0wVRsqdRtCq97dSIWd4OjHZTuZWrxqdOppFtQAhxhGQs6g97cEcToAdPqqvD1y59y6mTDYOvsVMxNbDmqMrJr4m9l5VotggOIGh6laVKXUmvcNwosguRV7DgPDPLZUntMFXbDy0XdwFcA3UZ91VKnrkZioUwC/kEKMpkkwCnCcKSm3eVDURaoXtMq7EKCq0phmqbm6mVUrtElaLxpqJVOvE7JlTFB7dFC4AA9eSuPGhVeoNQpDVZ22qPBKbn4jTnkf0KCsNStHhOkXXocBsf0VzC12tPS3y9G/ovHOJv8AXLr+f9AvZT5WkdQfsvG+J9Mfu/5h9gs2M6bh6saN+HDWQeccitqrduc4+vVc/hM/iQR0P2K0XkzPddf5fWosm4dyP1Qmq4iJj3VUkyn16lc71VqZz3aeZCXGd1ESeqbN3U1NO85j0QkJ0xXRTJwmRFSpSKNm/sowpGb+yiJXscGh2UweyHLC6TEaFF9hTdSaJWE61reISSADsrgrBziYyrXwDCa2JOc5oIa2dYmdJUNHD6jgDou84Go06dtVaGyYO3ooPO7x7aN3VtyADTcW69iR+iiDxvE+6DiEPHEN6J08Z33KChMQUFkPPIKWk4ndQBSU9wtNLTHawvR+BaYa1r43nSOy81YdV6fwdDbZjp6/ZSkdS5/mOnPqia7RVmPzE6zBUzCoqw1w6Iw7soWO1Rgrp4idp0Tyo2lFKlBylKCUpUCc7dZeO1MtoRlmQea0C4rMx7W2Haf0QcvX8xMiFULZMzA6K3W10CrvUvqW4jd5RtKB+VzAC3ZO/ZN+VSen6AaQDC2RB7KF1nTcC0QO8K0/khG5A3Wtq9K1S0z2LrUVMsmZA9eU91Uw+xFrVc7NIgjaOS1RzPMJoEajU8lLadMB9m78XUqEQHOkEjuVu02tFCm3KCQN0NWm1zgCIKZm5GbQKJonwQVathFE9wqYJIMq9RH/AGxPQLSqdd2p02KFnmEzCVT+KR1Sp6aIBO8dEinf8XqmO3sgJm6mp6mAomblS0806bc1kQXN9b278lRzZ7kJFwqNDm6g8xqua4jc0X5lxOo0nsugwZrzhzXOGpPP0C1JqWk/bQKnW1JjVXrhrmkjlrsqVXnlVw1UqbFV3xPRWanwlVnbrOYitXhbvBFPM9zzAgnTfksKu0yur4NpBts50Rr77BJUraqv3AbyK8c4n1x67/mH2C9hqfGZM6FePcT649d/zD7BS2H1WsDFcf3yV6qSI7kyqFi6K48s6H7K6955t5nmnDytTwjmbqELUwf2n3SzLNUfIJSgLp5Jx6qyBzun5oTunG5VZOnTJ5CBIgh9EQQdTaO8TDqUa5T+y0KVFrmF7tyBCy+H3GpaOblkA7/JbdvAZBEyPkgoiGg5d5Wpwlcto3j6JIgz9lmEf5zhrE7QgsKjrfF2zrM6zHIqUczxbTLeJrs8nVCR8yqtPcLS4xAdjZeOck/NZ1IeWZ2QTBG1A2Z6I42VkxpKw6LosIxx1vTFNwAE9CubZHVTAjdalNd7aY9RcTLw3bkVu4RiTKj5FVrhHNeUU6jmknX5q3bXtWiczHvHo4hXTXswrZenzUlO4Y6Qdx2XlttxBe0yC6q946F5WtZcVO1FWnMRBD/6LA9Ca5pE5oRBzeq5O14msXNBqVAw9C5adDFLWpHh3DHejgrBtSE0tVFt03KCCHT0KNtcOGwnpKotl2ix+ITNJnv+i0GVmubM69Fm4yHVKbQ0TEz9FjBz1XmoniQrNak8Ey069lCWu5tK1BXIQEKy6mev0URac2XbuqAhII8p6JFvst2pgUoT5e6I+i50qJ6FG+DHJNHXRRYBrfOB0VyqD4AO0hV2avbpzVy4pjwG+aNE6/wMxwMnzIUb25TvMoAgSSRSQFPdGx2h1kwokTHBs6T7oMilauusZc6oCGjn8lrvzNGWUm5WvLmtAJQVXbn9VYBfrLi5Ubjc+ZTVKmkeqqVSCTqqIn7qFw0d2Uh3kqOq4fNWsqVw05hHVd1w600rDN1P6BcWIqVxTA9139lTFGx667RHRZqw5AkkcwSvHeJP9cu/5/0C9gGznTOh06Lx/iX/AFy7/n/QLFZqrZz44jofsVbfKp2n8Yeiubkla4fWoFJJzRsj8qlUzAnKWnJJJQk4TpBVMJPr0SUrBmEoYjHcQiATHc9ktER0XDziLepl2B/ZdBaQ6g10wY1XLcPO8jm+v2XRYc4OpBp5FXFwNVsVjqNVWv2+Fd0qjTy/dXq7WtrAhQcR0xmpumRA+5UsMc3xG7xbum4jUg6/JUmUyWmFoYzTaDScdAQffZR0A1zcuwUxEOXnCcB0+WFq4JhNW+rZmhwpzqdOi6mlgdtbsBcC50c40+i1i64ZlFzvyytQYPceCajgAB3K6hlu1pgNAA5rncbvn1KwpUXgsAg/VMw9Zb6ZpuLXdUbA2E7aYc2Xak6pxRIMhQxIHN/2pxUicoiUi3KIzICNdJPVDRFxd+aFLSuq1MSyq5p7FQBo5gpneXdDWtbY5f0IJrOcD1A/ZatrxXUYJqNc70AXJZp2SlXTXolrxTaVHDM11OeoH7rSoYlbXXw1A4fuvKZISbVc05g4tjpzTVet5aLz+UqJ1pTd+ULzO2xa7o/BUJC3LDiy78RrXNa49gf3TR1dXD27tj5qtVsHNEwEFpxJTc3/ALimW7awfdaNtiuH1oNOs0HoZ/ZWDKdbkBROokQuncKNQEh4d6FQ1LOmQCNPRbprnfDURp6LfqWDZ0kBVq+HuiRJWOXFKxntDYndNH+7ZaD7So06tIULrd06tKmEqtQE1h6q3efwgOxQ0qWWo0kHdSYiNG9lbKrMe2SFCVbe1RFrZU9EISUgAGydMESRSITDSVApUNVx2UxKqVChekTnDUFV6hapKvL3Vept7q6mmfsoHuEHNy2TlRVfgTTBWEOvWxzK9CqQ2jlBlcTwtTLr8OHIH7hdtV23lPTEInK4ERAK8g4l/wBcvP5h9gvXx+bTkV49xJ/rl3/P+gWbEV7ODW35K27Q/F3VOy0q+ytkjMZWv5/VhjE/Enzf8QkcqYmVm58UQjkITpmg6ynQEkEklYpSjzdNECdGacu1GiduUmDok0E6AaqQUXgZnBEaWBODa5aRE99tF0eHNPi5AQZIXKYcctw2TAXS4NWY69a0OJ1E/NaaXcRYab2k6TzRYoxtag3K2SAO/NWOJmMbQpuaYP8AQqrd3VOnSBc6IGylpHO8RUXGhQqNzFrCQ6B6fsVnisxwAadI3la1fEKdembVzZDidYWLc0207l7G6CP2TUro8Hx4WNE02U2mTuHR+i0P/kTqoALB3Jd/RcVTJBEAj1VuhULXgEzPJVHR3eMVHeVgyzzBXPCWVoJLpOsq4yo1wkiYSo0RnzObJOylWJqVIkBwbuOimFvUcJa0geicPdIgQBoVq27m/hMwbqoqthljb1gS5zZB20K06eC0agim1oJ/47rmMDt7+6vH/h3FoG8nt6LpqQxSztn121GnwxJEdAe3ZGVLEcJqW7S80CQOgXOVXNqiWEH0MrQxPia6vGmhnLSDBgBV7DD3fgzUG3c+iCtSpudAbMnlCerRqUhNTynkDpKsXA8Gk1zXQ8yR7KnXuH3T2tqEkjpogHM7uUz3CNQhc7KFE9xdsiakDhEAhFQuHUnhzZkd1WEypBl35KVqL7sSuHjK5xIPKVXFxUa4ua9zSehIUQjkkrKrVtsexCgAG16o15uK1bPjK6pgtqtNTaDmI/QrnKNnVqGAJ9Vcp4LdPgtZI5kkrpsHXWfGlu4Dx6Zae7p/RbNDiHDK0NFdknqQvPxgdSAKj2t9z+yN+H0aBn8TqOhKzaPShWtKwDm1qZnkCCnFKk7UNa4dQF5XVvq1qYp3Lh7zsrNpxXfURl8QuA7D9kMeiOs2uIcAJHZVr2yqPgtbG/JYNtxs1zgKtMgegWpa8UYdcCHOc0iNDGs+6S6AqWNUflM+irOtncwR7LcoXlrca03ypKlCmRq0JZg5p9E8j9FGWxufouidZ03g5W7d1Wq4frpAQYT2noozPTZatezcAdlRrW72nSFkU3vLT8M+6gfsrNRhBhyqPPJEqvWOUA7zKrOfP5Y91Pcan5qqd0Qx9VFX8rI3lHKiuT5R7o22+DKc13VJ1IOkdwuoqVBmygfVYHCDAykahGkH7rZLvOrEEHauHYryDiRv/wB7dCfz/oF6013ncOgK8m4lP/3t1/OPsFmsqln/ABfZWTGkiVWswfGjsfsrb5gQtfz+rA+WfhRyh15uATpVOCnTDmnUBSkhThGTqRjRmCjUjfi3+isFymRTpudzhBTdWvKTqTdY10/vuiqtDLMkuEvgNB/vutnhu1taNo6vUe0vg+UgDmqOca12YtG43U1KpUpvDmOIIgiOagDntrublJJ6c1bph9OHPomNwCP6LInrX95cMFN7nQOgTi3u7hwdUaQ3qdFYpXNDIHOotGm0D9kGL4nUuMtNjfDaOTTv8kWCtrWnTrBxqSQdZ2VLE8jbxzomR7bBD4znRJII77oasuOaZ7FFCApaO5a3UmICEAwTGo5LouFcIddP/EVWkMBBEtmd1pln2uG3lYhwpuiN1q0cFvtMzfqP3XZMpUKNIMp026c4CLKcoIEqVY5ZmD12kZxHTUK5a4TVymCde4W2/KYzN1GyVKplPw6eqiuapYfcYTdGo0Sw8/ogxLEjUoOph8FwIIXW3LaNVha5rSDyXPXuA06lx4rXEAEmAP6rQ463saj7lzgxzgdZXU3NjcUrPLcVWU2DcAj9yocSqVLCmG29uCRpMf0XN3t5fXBy3FSoQeUmE0Wb+tY0obTf4jtZMc/ks6rcFzxlEBCKQ7D2SdGmkQroKgab67Wl3lB17q9fut5aLcQI1PVX7fA6TsMdch4DhrAbPTnKwDULqtSmNAwx9/2WGUgb4hKkqNNPSdUdkxrhJdHstDCLP/EMSbTiWA6iJ5FWLFPD8LuLyoctNxEjX1XV4Twu2mA64JgjaQuktLO2sqDadOk0OAAJgCYClLjI5jomKouw+3osd4IId1K57HsZbYVW0KRa6qZB7HRbmP3zMPwyq9zofGhmJ1C8t8WrdXlS5qvcSTIBMxqVRfxK9uKz85dDjqqLA9zg7MVZGVw84k90JLWiBAPVShy3KAc0yoXog6AZMj7JEAtzAz7LMgAb6J3OdIMTGylrU6TaGZtQFx5Af1UIcAB5ZI1VnQno1KzRmDg0d1etuJL63cC2pmHcLHqPc4akgdFA2W9D6IldracbV2aVmMgxsD+61rTjCxqCanlPcFebaOEluo21SBJMHZEeuU8YsK5ytuQSeoI/RSGrbuHle0k7wd15G2rUb8L3D0JClp4jfUyPDr1QB/yJW9a16hUt6dQn4RKzbiwpkeV0lchQ4kvqQh5Lo6krStuJs5AqU4HUu/os0XbmwdEN7rOr4fVDjDSfktBuNWrgXEtJ9VVucbozDQ3/AP0oKj7N7ASWqlcsLSGluhV12OUX+U0wJ/5f0Va7vKVVmZjQCAdJRlq4bilOzsMmkqtS4kDjLmyfQrm61Sq4kZiBKibLTo7TorB21vjdOoyo6QCBzB5grzzFqnjYlXqiIc6fotFtVzWuhxMjaVi1TNRx7rNE1nm8X5/ZWX5jGsKrZfxx6H7FXVrh5V1HlHNyJM5JS1RBOmanU0JOECJk6wYVTBKRkyDG5UevVExpzNnaVYYuXrM1GhlJJEmPkhpXNRpNMEgRsnu6zQ5rW8hqq3xO1VMT0qrqdVry2SN9Apby+fcBuZuUNkCAB0VQebUpoLiB0WUS5nEQMyMbSRqgY2ArNINDQ0O0RYiDQ4SRCNlNztlbt6FSu8MptLpIBPRdjgGB06DRWrNJcRtI6oqhw9w14zG1bqIB2BOui66lQpWtIUqbA1sQQOyf8S1jCGlU6115hJHNaTFrKSDl2UghtMSFDY1m1H5Q4KUKerIjqCSI5oSNMo3RFuYlLKmGADiTone4kAdE3h+qRZHVawV6tuyofNTB9Vh4xg9J1Img0AgdSujyhNUAkuClg8uuaTqNUsduCQhAldxjOGMvWbEuAMHTSVyOIYbcWry1zCROh0UwPb4lcUaJpeISw8jCoHWo+o0RmMlO7MT5hCfKIg6T0UZTWlSJlb/CuIUcPvpqtIBP6LmDMQDstaztHOo53GCrKPQxjFjXcXCpE9YSOJ2TQSawMei8+NuQR5tkz2csyurq1xji344+BRccum8dVzzaZpAS6ZGy1MrUL2tcNRMKaqi2oUzabnayrjfDB+FDKaK4YcwaNSVJk8Pdsk8kectIKI1MwSXBUAc0lzhI6JVKgBAyxoptOsoHUw4HlCoqvcXEgBKnSdKtNa2m3MN0RcpOKVXNItieaRpZUb3DRAXKIRaED27JnHRREq6mpMreYB90/iN5EAKGUiosqdjyQYMjsme7XUFViYhOHIuidlB0KIOdGh0G6GUzidERI9wLB1UUpZioDsromJ8jvRZT/jd6rRp7O9Fnv5+qzaJbP+N7H7K4522m6p2f8ce/2Vt/Ja4fQzikmKdZqwhPISizDnoh83JPpzRTxoE7QATzS5nsnGyrGn06IgSB1PIoU6sagiS45junLhIIGyEJ1dUYGY6GB1hTUKLnfC0meYCvYRhjriu1rvK0nUFd1YYPZU2Motpg1CBJkrLLibPB7q4g5XNB5lpWv/8AHvBPmeD/APz/AFXT3lChZ0i3QuHILA/xBzhmc4kIsW8Ms7ezGYgEmPyxsrr75tMaO9iVhVsSdENMbzMLMucSqOeWySiusN9Rk+YR1lZWK4pTHwOBInQO3WVZW99dAZXQD/fRaA4fqeGH1HSTqdSmiTh3GC67yuJAM6l3cLs25Y0cCuItsNNOsPCaBB0K6q28Ro8xVhFobnREIURdJ7omu0VUUIS3b9kScLaI8qCoCdmkBTwk7LGylIpGkZ3j2UNxZUqwio0O9Qr5hCVlK5mvw5QcMzdO2X+qjbw9TgDbuWrp80mITOYXb6Qso5M4Ax1fK3Qdcigq1qdF76TWjymN4ldlQojMXGNF55ePc/Ea5B0BCC0ajcnwwT3VdztZlJ5MADkgIJ5oAzEbmUxqaEZfqhcUA5o1DnedQlm/uUkKFFOYaoQC07ynCElAwEbOSe52kOgdOqDMhc5aBPqGNNPdCanZRvcgzqxKkfUiNPqgNTt9UD3bIZWag3O5RCAlOUKjJJFySZFhn6wmCcpkURdrt9ULjIGsIRqJSkT3QPrz0QHYp5P5kBcgJroBETIVB51PqrgVN/xH1WaJbP8AjD0Vt0wIKrWrf80c9FeYBElXj5ViEtd1TKxA6qEtclUwTpNbvqiyoHThLy9U2nIykZFCdDKeVqh9OaWvJIR0S15GFBr4diT6Lg0x7retMaLAXtIkRpB1XGh3nzfRTsrObDgDA5ArWj0bDrpl1TNSuQ0kKClh9Co57S/zEaFc1hdzXuA1rn+G0Ded9VpXeMWlmw+AfFqjnmj901YKpZttHVHVW5m6wTGiyPxFjRunAN8Qn6KrieLXd4C1z3MaZkAnmqDG5TmJJd1RXp2FZHsa5o/uFove0jfbRcpwtijXUhTqODXTpJ7LbdUnM4OkdldZWmu10KRKpUrjzluWe8q2YyyFLVhTqIU1NwgKueWsyiaTt9VNVaDtVIwyqrXidvqpqTxqqJlGiDmoZExK1xCISyoXu6apNMqg6bRCdzW5TKAODQNZlNUqCInQrAdkhj8vReWfFd1XN2n9l6fRqZm1Q3kP2Xl1MkPquHN37KVKmYRmd1nVNVKFuhLuuqZ5lQ02sKJ5Mo8xhRPMndFPJ7IMyRKj90SpQ/dA4yh90xdHKUQ5KB0Hfkk5yje4yFAZd0QIQ7tKfMkCKUIXHUAJsxgnoqCTFNm7fVLN2Ro6FIO1Qh2p02RORzyTQkHTGiWbQ+XZENHokNNkteqF0jmpGTk6oTumLinO61fFh3fCVSO5V3TKR2VJ259Vzonstaw9P3VpxMCOpVS1/jj0Vt4JAjuun8/GobM7onDtfMo2kNOqlU5Xat6Ly8kkklE0oCWnJEmKRDIkKk0Ws0CE6XoiCYEFIJ01gc073CFHOYnss4LAuHMAbTcR6KLUklwcT1QKRJywEJ57ckxQwCR2TuaIWpy1dT0Kz6Lg5jiCOYWvaY9VawscSYAEmFitDQPiQPidDKuM66W3xoF5LtCugs8Sp1GauP0XnKnpVnMEghTlFlek07hlQnKZhSh+mi8+tMXqW7gAAQYmR0W1bY6xwGdwB9CsrrpM5RtrRMrB/wASocnA/NQ18UY0DKQJla010n4kdVF+NZm+IwuSrYrqYI+SqHEDnJlP1hrum3jZMEomXRJ3XEUMSygkka9lYbiw6j5FP1/o12X4hska6KOtcAtBB1C5P/GD1HyKYYoKjhLmiOxTTXW2tceFXnTT9l5s1zoqd3fsulpYg1rSSQVzmk6dVN1L2LMdkxKF06qMomHLkBKZ26jKuNaMlBKSjS9FuiqOiO6EFIplnUIuQFOmKJpJpTqNIS6LmEhsUJTKqdPOiFC5F0TTqUh8TkCSIIfCCnO4HVAkgL2TO9IS8vf5pjHKVJ0mAKUop9UC1+jBs2Poqjvid6qw7Y+iqu3WL2YsWn8cDt+isvJiBpqVWtP/ACPn9laG5WuH1qBG2rZRZu31Two0vq8hhx5D6p5PT6oWkaynzDsoyOUt0ycJAoRadEyKFsMPSEQTAJ0DGYTsShOAOZhYoSLMmhvVMsgi4yOSRcfRCktQEHDmJ90i4ch9UxTLoyLMjkRESokYELPJYcRpyjuiDiDIKAuCafVZVYFVwMhx+aTqrnRmcfmoZHdMXTtK1jOrAbm5k+yA89PqnoOy7ynMLNWB5BoH1SzGD2TAyfRIbFFKe6Rc5uoOvZDKYkQlMS0rqo3QkkeqmkbgKm3cqcO0TiYIv3kIC5CXaoSVQi5CXDmnTGeQlTQ2YIJ1Ry7/AGqKdJSgikgzaJT3QOmPJNKcbFaxMNKHTqk8wlp1UzCGKUJJIoSYQlwO+iYItRylA3l/sJeX+wnk/wC1MgYxy1SlI8tITIJNOn1QujSAmlIqBT2QwnSUoaPKfRVXb+6uDY+ipu+I+qyJrT+KPRXGTB9VTtCfHECZB+xWgwN1zECSt8fqxE6ZT6qY0SRLSCoiMu6lzVpjKbVOfRMp0CSSSW3MkYKBIGEFikM0t3mFMyzqVHkNMCOoVRrnTLTEKYXNVs5XEH1QL/LpiGmUxOYztCh9dUTT00Qg8o7IUXvCCVK0dJNKUoHThMkEDokJKSlXTpkTWzOsJ8vf6KM4cNRDmmLkhPRaUiUuSYg+g6pckiUjskkdgkqhJinQuKiZCUjToopTtfpt9UiyCedCoipDME9VG7Qqh3boEi5NKn5dDpkpSSQzTGeSb1Sd2Kb2VTDJJpSlEOh0/NunlDm6iUBawR12TCACCPbqhJ8wPIck5drPMICSQ5ki5IySZLN2TSrasI8kx1TlMNFFLypJadExjogdCnlIqUIbFV3fEVOHaERuFXduVMZWMOY59yA3kD9irjvKYMSCUfCWV2Msa7QEH7FdTjGFMqMz0ZkAkjRa4z1uOaoVNhojdb5xIUFenUt6zmvEEc0dK4I0KzZ216huKZpkDrKi1V59TNACgqUzMhMTEaScNSK3jGGSSRayoYTeadMJkynQwyQSSQwYKECEyNSqGUk5TICSRNbI7qanRMa80wVwpfD7qfKKeyHVTl0Iw3Lz3SUhDRqd+SVOm6oYptc49gpO1iJExrnAhq6fBeFq908OuM1NoOokLrbXDsPwygXQNAJLiOQ9FrEeUOZUByzqnXfYrY4Zi+Z1CoDUB2Hy5hcdiGE3Vi8udTcWDnoVRSSTDLpGhG4SOUjaUTDoXoUpIB6KNYZJu4T0qdSqT4YJPZdBhOCubFW6Di0H4dEiWMI5o0UL2ukrrsTrWtrRDdAIIAHZYVb8NctLqbsruiqYzUlLXo1GulwlQHeFrpsSCU6Dzc1KbgimTNyzonG6hpkkKfkrYydAnTadFnQkkkk0JMU6YmIV1MOhT7plNISSSSapJinTGeSaGTlPBSZsfRNSgG6ru3crPNVn/GU0jT4czNxOm5rtROg9Cu+FUOphp0IGsndcRwg1rsYYHajX7FdZibhRqjIDGsq8ONaipilrSuCSWtB6wsC5tXUTuT7QuhbUFQhoBJQVaLajSHgSpZ205xjso1E+qmpvBPwz7q1dWJ1c2ABKoFrqbiJQO5qjLTHoiD5UjdiFthW1nUIpMyiq00ClBA7pSmCdQLTqlolr0S16QgSNAia0qUPvsjZRc7/0jotaJza7Kb4fhQMGNaIBCfMYQ5kjyVCO8lyJzmjbZBKt4dh13eyGNBA5ys8hdwHBLjFH5oLaYIklpMg/+l22GYDY4fTBLKbnAalwAXOcP4lWwet+HuIDSQNANI0/VdY+tSxC0Pg1PMdd0go4vxFY2FNzWOa6pGgBAXC4zj99iQLWufTpyZAcdQfkrePYBdUXOuHOL2xzJ6rNw6xq3dUMpN1kAg+q0LfDFviDrgG1FV7oMxJ5r0V+EuqWTvxbWg7QR+6PhLDaeE2TXPa01CDJ3590eM4kyjSIqvho1MAdUHF4vwo6Kla2cAACSA3fmOa495FGu6k8AOaYIXV8Q8VOcw21m45SCCYC5Slb17u5Lg3NUfrJ5oBEES0yr2EYdVv6oDZDQROk7rTZwtdCy8ZxbmHKT+yp2F5Wwmu5rmwCQDHaVGnR22G2tiwFzGucNyQAs7Gcdp06TmUHAOI0IOyvW2KUL1gaZk6rExHB2kGowyB1JSJXP161e4qOfVqucTqJnn7pUWuaZEn00Ur6DmVS0tGmis0LckDkqgaTqjh59VHWptkubp+qsVctMdFTrVt+SmtInODTB1UQceaTiXGU5EKpTkhpBASDtZhAE6IfMnzDnohS9Ul0Pmam06JvKnSwJJJJQDJ6JT2+qkyoXeWJ5oTsGvolKKUCLYeUpTJIghqOiWxhO3ZD+ZA+bQmNkIdCQ2KZEpxzMbBVXfESrQOh9FWJ1OnNCNfhZxGJtPPX7FdpcNzEF20arieGNMVZzifsV3deMoJ0ESpx8rXFkMmjfFo2I0+imc5BSaal250aDY/JGWGN/on1aAxBBWddW7KjpEytE0yd3R7ITRbyMH0WpBzZovp+XLp1SBMkERC3q9sxzCIjvCzLq1dT8wJI15JqVXBSfTHLVM3npCOkddTKbrMQmm5pmEjK0G0hUaYMR2UFS3c09R6IYrwkG6gQpvBH+76ImNDZMg+ylULKOYy4ypXR+VDMHRL6LKUzp5pwkW6TKQiVr9NQZc3MQNUVKjUrvDabXOMwAAtPAsBusRqhzGOFMmC4tJ5LvsG4fscPYHVGsqVAAZLQII+a2RzPDvCVa4AqXrXMYRoJHX3XaMp2eGWjg0hrW6yVSxrH7LD2QKjZGkBwHJeeYvxDd4kS0VH02nSA4mfspWnaV7jDcYNSm2oM40A2gn1HZY1bDb7C65r2oLm8tQf1XK2NWvQqh9Nzw6RME6r0DhqveXVFv4mm4sjdwJG/cKIWG41SvB+Hu2+G46RB/qtW3t7O0Dq1N4AdBJnp/wC1UxPBbeqDUptaxwEhwELisfxK6sqotBcOc0yDBPKB1WpMHTY3xTRtSadFwc4GJg9FxF7e3eIVDUru30hVgRUcXO8xOpkyV2XC3DP4louLgwwHUFs8vVLRhYJw/cX7xlY4NkSdNiu/wvAbTC6AcZNSIJJH7LRt22tjSNOixgAABIAGyzr28dWdlDpAPIomFiF5TaCxplxC57EadtdFrKjocZj3haNRoc/M7eN1zfELnMrN8IuDpMEH0U1VTEMJuKJLqAzCdNQoLTE7i3llUSO4K28Cbd1WA3BcWx+YHr3QYrhtN1NznZWEDkAn6xKoeNa3Rzkw7oqd/cNojK0ghZdw4UrlzKdQw08tJUVao6oYLifUqfqoavWNY6oQNExbrI0TyQBKuh4Qp839ym91OV1YYpk5ShRTaZdN04nKZTaRATgmCCqycfAkmnSEggR2SjSE+jSDulm1nL7KYB16pjPqlpvH1S0gaQqGSSSlFmfSSSlKUNhJJSlKtqkmTyhlRORxzVd3xlWBsVXPxFZqNXhgxibSeh+xXYXtR9TKxhjSD9FxvDX+pN7g/Yrsg0D3V4/V4wNNopMLY8x5pi4KQjyFATCS91q9AKYqRup9VKfDpgGdVZyRVc10whfQbUb5hIUznyU4GYjqtJYxL+zFHzgaSsxjjK6u9pt/BvLtey4+k8ugdFLMSdNKlULY7qYu8QaBUqZ3KKVF0ZaUJBRhwhC5wkaSEsZ0xdBCQ3RNl2jWyOgXR4BwtdX1Vr6zHU6U6yR0UnS5rDtLSveVW06DZJIB7TsrmIYHfWIa+qxpB6H+i9Rw/DLbDKAbTbJAEuMcgq13f2NaqbWrUbm6f2O6ZqyYzeDMXs/wYt8uSpO8DoO62cUbVrWzjQd5iDB9lzWM8Nii417CZGsSP6KphmO3mHVRQu6fkBAJIOw35ra4wMdw+/ZcVKlw7O3NoJ7eioWlq+tXFKm3UmAV6rRuLDFqIa14JOpG0fMIMMwe3s6viNaJGs+yZpWLwzwq2mBcXbQQQDBJ9V0darQs6MACnTGmgVLHcct7CmXOe0kAw0AnZeeY3jt1idc5SfDnRoGvRMGxxPxW+pNGxe5jCBrA6+653DsPusYuhLs7iR5idpPotfhrhu4vqratxTLKQdqDHReh4fa2uHUQymAIA1PYJqa5xnBtCjYtBcPHI1OY91JYYlXws/hbkE0ydxH9OiPiDiilY1Cym4OfMaA9FWw7HLLFGeHdZQ4mNQUNbfj0LyiPBJEjn3VZ9F1IkblUrjC6lAi4sXQBqBI15hSWmLmfAvqfhkfmgwfuiygqNfnAJ3QHDadar4lQA5dRqef/AKWzRbRc+ab/AK6LPx7FKOH0HZqjS+DA1OyxejENxUt7RgLhlaNNFw2P4866Lqdu5zWkbwOqrYzi1XEK5DvgnSFRDQFUqNgc85i6SdyjAh2qd3JNCID3SPLWU6YpiaGUUpklGvBApZkBTJhoktUtOiWnSE1CSSTkrUoEGU6FxTJ0Q/JJJIqBJkk7t0PTJJzshKJh0kkxRrToUkSmYlphzVd25VnkfRVzupU1r8KtacVaDrofsV2VUBhBGq4zhSRirT2P2K6q/rlrwBK1w+t8UxeMkRv3UQIiS4BVhUc5u6cuMarM9q8huq5SQ1R+I5x823coczXfCNRugfUY349EiLTXCVMK1OmM1QgADTXdY9fEG0wW0wT3VGrc1avxOMa6LcK1MQxBpa6myDPdYLGAOkKZoBMk6pg1W/GSZ5REohmPVM9uyNjHxosgjEeVwKmsbK4vqoZRY4yYJAJiUJt6lF8VRHovSOA/8Pbh8NYPFIEkjnB7q1cQcN8H06DBWui1xI2LI59yumubi2s6Pmc2m0a8gmv6z6VAuZ5gNgOS8w4mxbELq5NN7yG6bQOfoskb3EvFjpNCzdrBBIdv02XGi6ujdOuHVHl56klOyn8L3mZWhZYVWxC4a2gAARMlaiuo4b4grXLhRrUy+TvM8vRbmJ4Pa39AuNJrXEGCGgmShwLA7fDGA1GBz53k9FdxK+o2dE1KjsrQCYVVweJWGIYG816TnCnMAiR3/RHT4zfUtnMLfNtOaP0VPibiCvilY2trJZO5A6f1WFWwu6tSDWZAOuhTtKK/qXGI3ZOd0vOgkncrsOFuEWtY26vSRI0BZ37lcbY1m290ysTOUgx1gr0jB+Ire8o06Tpa5o2gfuptGtc3FtZ0i1uSkwCYEBcTxPxVLXW9q4ayC5tTblyXR45YuvrZ1Njy0kbg9151i+BXFlVh5BAJJMkqsqD6lS4eX1Huc4mZJlS2rajaw8Nzi4nQAmQntqDqzxTpiSF3vDnDtOzivdta542EnoiwfDLb/wDDD8VUcWkCA4HT5q9iNjb3DP8AMIaP90BWLm5o0WFzobTaDt2XCcT8UVKj3W9q8tYDvA6f1RQYnjVXDLo07WuarSOTiBz9Vg399Wv6hfWc6SSYJJGqqHNUcKjnFx6FOfKdoB2Wb6mna1rTO6Y1G9ISJAQZndEzSiLtRAlDmMxH1TOLomITSYlVCzJTKFOEqQ6aU6FZjVPuCeiUaSkN4T84VQ2YpA9UyRUDlyRcPRIIMw7pIlF7ptOZhMJOyQHVXGoKUxSSRCTlMk5A4MgoSnbsUxQOmKQSKBk8pkkSnnQjsoDupuqhO5WaNHh3/U2eh+xXT1yGiYmVzGAuazEWudoIP2K2Lu+Y0EEjnGq1w8ani14gyKCrdNbJLgSsutfPdLWE+xVZ1VxBOp91nMq60a9+RAaPTRVH16lY+bZQjzQS6OkhPqPzCFuGjyjujYIlCHCNdD0RskgnQR1KbXM6YAl4aRv0VqwsLu+qZaFBzu4BI+y7nAeDKdEipfOa5wOxp9u5Ta1HLYNgN3iDw1tNwZI1MDQ+q73A+G7PD6YFXMakanT9lqsZa2NENphjWgCYgbLFx3ii0syadIhzwf8Ad29E7aTcS4NbYnQ8oioBoQR1XC5b3A7zLrlBgE66A9lp4HxO915Fw4uYTGr9tF097ZWmLW4c1rXZhMgAkSFrsRcP8RW91SFK4cGuiNj1VjHMDt8Qti5pl0aEEdVw2NYTeYXWNSkXFk6QCI0lafDfE9SkRRudWk7udtp3WbqU9hwvX/GtbcAikCOY1Eru7GjQsLYU6IEARJVajfWtaiKzajCIkwRouX4l4mFF5o2rszgeR7dlEbGM49b2dJ7W1GuqAaaHquHc/E+IL3K1hLM0SBAAJ7nspsAwO/xm4FW48QUZ1zNJ5dSvQcKw+zwy2FKlTYXACXAAHRGlDh/hu2w2k19Uk1SNRI6+iuYzTs7hpoVA0EjQKtxBjVvYUSZD3zEB8EaLgbnHbq5v21hUcI5SY5oLGN8N3FGq6pbAvaSSRI0HJYY8S3qZX5mOGhHNd3hWO0a1PwrnK10AS5wk/NR4rgdrfUnVqYbmJkFoBJ+SGs3BuJ6lOoGXGUtJ1MHp6rqKdazxKgWsc1xcNjpErzi9wy9s3nxGuyjmAYSw3Eq9rVJp1HCIkAnktSYz09BsMLt7Co6qPiKLFcTo2lsaldzRA0Gplc3R4q/y8tSnJ6l39FzGKYlc3zyalQhv+0kkJyurOvFjH8fuL5/hsIbTBIAAOoKyGt1JcJKTMrQQQD3IT5tDBAlZNGHaKOoZKHNl0TF2o7paadRos2h0TadVGadvNEgBjknzIgUtesJJGOiNH16pktOn1SlAkkpSlAteafQCRukfVMNigMbZkGwTh3l2QEz2QIbk9UUoeRPRITqXaDrKBk4TJ9FQ6FPKaR1V7+BJJackk2hZkp7poTj1We2Taf2EyPXko02rBDf2KgUzdzryUTt95TVW7djm1BUaYifspXS+ZMkdUkljjemip0iToPktOjgV9VAFNgM9/wCiSS1Eq5bcK4jUIFSmKYJ3Jn9Fefwpa0mZri+cDzAA/ZJJbiK9XC8Ipgg37p7gaqi5mH0agLXOrNBnKdJj0KSSyy38J4qt7Olkt8La0xEkyfqStqnxJiVzLqVi0jbdJJVqK9y3GsQIFSn+HYTuDvPsqF3w3atBfdX9QE77fskkprSi/CcPpsLrfEGl3IOMD6BbPDn+I24Ph3VnUaIEGo7v/wAUklrRuVvxF2w0rqjakby17j+gXNYtgmH0H523FZo6NAP3SSSpVJ1S3t6Ja28uoiDIEfdZ1tdUqN2a1Sm24B2zEg/RJJRHX2XFjqgFva2VGkP5j/VWbnFcSqUiynSt6cgiS8kn6JJI0wL/AAp9Qm4vrqqZOzACPrCyzRwyk4guux3DWn9UkkSqFe4pUq4dQL3AGTngHfsVt2nGYoW4Z+CnLoTKSSIq4jxVVuS7w7RmRwiCdQuaqVHur1HggZjMDlukkt1cN4rtJJMJi73PIpJLCUwdunlJJGSbqJQnc9kklmhuY7p9QkkkCJlMkkqEkUkkaMkkkgSSSSAQ6dwU5iQIiUkkCnWEySSCzYUres9zbit4TREHqnq0bUPLW3GZuwPVJJBBlp8nyhIHIykkqG06pa9EklaFrzEJJJLOhpTOdEJJKaybM7khzO6JJJqw2YzsmGqSSiv/2Q==\n",
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"800\"\n",
       "            height=\"600\"\n",
       "            src=\"https://www.youtube.com/embed/cNxadbrN_aI\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.YouTubeVideo at 0x7fea0a582510>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import YouTubeVideo\n",
    "YouTubeVideo('cNxadbrN_aI', width=800, height=600)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img style=\"margin: auto\" width=\"80%\" src=\"http://m.natemat.pl/b94a41cd7322e1b8793e4644e5f82683,641,0,0,0.png\" alt=\"Frank Rosenblatt\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img style=\"margin: auto\" src=\"http://m.natemat.pl/02943a7dc0f638d786b78cd5c9e75742,641,0,0,0.png\" width=\"70%\" alt=\"Frank Rosenblatt\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img style=\"margin: auto\" width=\"50%\" src=\"https://upload.wikimedia.org/wikipedia/en/5/52/Mark_I_perceptron.jpeg\" alt=\"perceptron\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Pierwszy perceptron liniowy\n",
    "\n",
    "* Frank Rosenblatt, 1957\n",
    "* aparat fotograficzny podłączony do 400 fotokomórek (rozdzielczość obrazu: 20 x 20)\n",
    "* wagi – potencjometry aktualizowane za pomocą silniczków"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Uczenie perceptronu\n",
    "\n",
    "Cykl uczenia perceptronu Rosenblatta:\n",
    "\n",
    "1. Sfotografuj planszę z kolejnym obiektem.\n",
    "1. Zaobserwuj, która lampka zapaliła się na wyjściu.\n",
    "1. Sprawdź, czy to jest właściwa lampka.\n",
    "1. Wyślij sygnał „nagrody” lub „kary”."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Funkcja aktywacji\n",
    "\n",
    "Funkcja bipolarna:\n",
    "\n",
    "$$ g(z) = \\left\\{ \n",
    "\\begin{array}{rl}\n",
    "1  & \\textrm{gdy $z > \\theta_0$} \\\\\n",
    "-1 & \\textrm{wpp.}\n",
    "\\end{array}\n",
    "\\right. $$\n",
    "\n",
    "gdzie $z = \\theta_0x_0 + \\ldots + \\theta_nx_n$,<br/>\n",
    "$\\theta_0$ to próg aktywacji,<br/>\n",
    "$x_0 = 1$. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "def bipolar_plot():\n",
    "    matplotlib.rcParams.update({'font.size': 16})\n",
    "\n",
    "    plt.figure(figsize=(8,5))\n",
    "    x = [-1,-.23,1] \n",
    "    y = [-1, -1, 1]\n",
    "    plt.ylim(-1.2,1.2)\n",
    "    plt.xlim(-1.2,1.2)\n",
    "    plt.plot([-2,2],[1,1], color='black', ls=\"dashed\")\n",
    "    plt.plot([-2,2],[-1,-1], color='black', ls=\"dashed\")\n",
    "    plt.step(x, y, lw=3)\n",
    "    ax = plt.gca()\n",
    "    ax.spines['right'].set_color('none')\n",
    "    ax.spines['top'].set_color('none')\n",
    "    ax.xaxis.set_ticks_position('bottom')\n",
    "    ax.spines['bottom'].set_position(('data',0))\n",
    "    ax.yaxis.set_ticks_position('left')\n",
    "    ax.spines['left'].set_position(('data',0))\n",
    "\n",
    "    plt.annotate(r'$\\theta_0$',\n",
    "                 xy=(-.23,0), xycoords='data',\n",
    "                 xytext=(-50, +50), textcoords='offset points', fontsize=26,\n",
    "                 arrowprops=dict(arrowstyle=\"->\"))\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcwAAAEeCAYAAAAHLSWiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1yUZf7/8fcoZzEVT7mmwapYQsUqth5yKaxdUpHWFHTDxNrU1HbxlNpiWlp00KLdVkOtNA+PBH0Y5LElxbJsk3bZDAo1wbTNNHU7iKDA/fvDL/NrHA43h2EYeD0fj3nIXNd9DZ+5He4398x1X1gMwxAAAKhaC2cXAACAKyAwAQAwgcAEAMAEAhMAABMITAAATCAwAQAwwa2afq45AZwgIiJCu3btcnYZQHNlqaiRM0ygEfruu++cXQKAqxCYAACYQGACAGACgQkAgAkEJgAAJhCYAACYQGACAGACgQkAgAkEJgAAJhCYAACYQGACAGACgQkAgAkEJgAAJhCYAACYQGACAGACgQkAgAkEJlCFkydP6pFHHtHAgQPl4+Mji8WigoICU2PLysqUmJgof39/eXl56ZZbbtGWLVscWzAAhyEwgSocPXpUKSkpateunYYMGVKjsQsWLNCiRYs0ffp07dy5UwMGDNCYMWO0Y8cOB1ULwJEshmFU1V9lJ9DUlZWVqUWLK79Xrl69Wg899JDy8/Pl7+9f5bjTp0+rW7dumjdvnp544glr+9ChQ3XmzBl9+umnVY4PDQ1VVlZWnesHUCuWiho5wwSqUB6WNbV7925dunRJsbGxNu2xsbE6dOiQ8vPz66M8AA2IwAQcICcnR56enurZs6dNe1BQkCQpNzfXGWUBqIMq35K9/fbb7Trj4uIUFxen7777TqNHj7Yb8/DDDysmJkYnTpzQ+PHj7fpnzZqlyMhI5eXlafLkyXb9CQkJuvPOO5Wdna34+Hi7/qefflqDBg3Shx9+qMcee8yuPykpSSEhIcrIyNCSJUvs+pOTk9W7d2+9/fbbWrZsmV3/unXr1K1bN23atEkrVqyw69+8ebM6dOigNWvWaM2aNXb9O3bskI+Pj5YvX66UlBS7/szMTEnS0qVLtW3bNps+b29v7dy5U5K0ePFivfvuuzb97du3t04amT9/vg4cOGDTf91112n9+vWSpPj4eGVnZ9v0BwYGauXKlZKkSZMm6fDhwzb9ISEhSkpKknTlTOjkyZM2/QMHDlRiYqIk6d5779XZs2dt+ocOHaoFCxZIku6++25dvHjRpn/EiBGaPXu2JOn222/X1aKjozV16lQVFhZq2LBhdv11fe2FjJ2lf/zXXRculdr1NTbfrI1XlwlJzi4DaBAFzwyX1HiOe5mZmRW+JetWq2cHuKCdX1lUXNb4wxJA48SkHzQb/vO2O7sE0zjDRHNSfobZiHCGCZSrzQ9oTWbJvvHGG5owYYKOHDli8znmmjVrNHHiRB07dkwBAQGVjg/NWKisxncQAZo1Jv0ADhARESEPDw9t2LDBpn39+vUKDg6uMiwBNE6cYQLV2Lx5syTpk08+kSTt3LlTHTt2VMeOHRUWFiZJcnNz04QJE/Tqq69Kkjp16qQZM2YoMTFRrVu3Vt++fbVp0ybt2bNHaWlpznkiAOqEwASqMWbMGJv7U6dOlSSFhYVZZz2XlpaqtNR2QtFTTz0lX19fvfTSSzp16pR69+6tlJQURUZGNkjdAOoXgQlUo5qJcZVu07JlSyUkJCghIcERZQFoYHyGCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBiTr73//+p8WLFyskJEStW7eWn5+fwsPDtWPHDmeXBgD1xs3ZBcC1ZWZm6g9/+IO++eYbm/a9e/cqMzNTK1as0OTJk51UHQDUH84wUWuZmZkaNmyYvvnmG8XGxurgwYM6f/68Pv74Yw0cOFCGYWjmzJk6efKks0sFgDojMFErp0+f1tixY3Xx4kU999xzWrdunUJDQ9W2bVv1799faWlp8vX1VWFhoTZu3OjscgGgzghM1Mrs2bP17bffasSIEZozZ45df8eOHTV48GBJ0r59+xq6PACodwQmauyLL77Qhg0bZLFY9Nxzz1W6XceOHSVJx48fb6jSAMBhCEzUWHJyssrKynTnnXfqxhtvrHS7y5cv2/wLAK6MwESNlJWV6c0335Qk3XfffVVue+7cOUmSt7e3w+tylBMnTmj06NFq06aNrrnmGo0aNUpfffWVqbEWi6XCW3Z2toOrBuAIXFaCGsnOztapU6ckSXFxcYqLi6t2TLdu3RxclWMUFhYqPDxcnp6eWrt2rSwWixISEnTHHXfo008/VatWrap9jLi4OLvLagIDAx1VMgAHIjBRI7WZwNOrVy8HVOJ4q1at0rFjx5SXl6eePXtKkm6++Wb16tVLycnJmjlzZrWP0bVrVw0YMMDRpQJoALwlixr517/+JUn63e9+p4sXL1Z627p1q3VM3759nVVunaSnp2vAgAHWsJSkgIAADR48WGlpaU6sDIAzEJiokcOHD0uSunfvLi8vr0pvH330kXXMb37zG5vH2LRpk/r37y9vb2+1b99e0dHROnbsWIM+DzNycnIUHBxs1x4UFKTc3FxTj7FixQp5enrKx8dH4eHhev/99+u7TAANhMBEjZSv2uPn51fldtu3b5ck3Xjjjerevbu1/ZVXXtHYsWPl7u6uF198UTNnztSePXs0cOBA05NpGsq5c+fUrl07u3Y/Pz+dP3++2vGxsbFavny5MjIytHLlSp09e1bh4eHKzMx0QLUAHI3PMFEjFy9elCR5eXlVus0XX3yhzz77TJI0fvx4a/u5c+f06KOPKiQkRPv27ZO7u7sk6e6771b//v312GOPaf369Q6svuYsFotdm2EYpsauW7fO+vWQIUMUFRWl4OBgJSQkaP/+/Xbbr1y5UitXrpQknTlzppYVA3AUzjBRIx4eHpKkCxcuVLrN8uXLJV25nOTns2jfeust/fjjj/rzn/9sDUvpymec4eHh2rJliwoLCx1TeC20a9fOemnMz50/f77CM8/qtG7dWsOHD9fBgwcr7J80aZKysrKUlZVlXfQBQONBYKJGrr/+eklSXl5ehf35+fnWs6Tp06erS5cu1r6PP/5YkqxL5v3cbbfdpqKiIuuZaWMQFBSknJwcu/bc3Fz16dOnVo9pGEaFZ60AGj8CEzUSFhYmSXrnnXf03//+16bvwoULiomJUXFxsQIDA7Vw4UKb/q+//lqSdN1119k9bnlbY/rLJiNHjtRHH31kMyGpoKBAH3zwgUaOHFnjx/vhhx+0fft2/frXv67PMgE0EAITNTJx4kS5ubmpuLhYI0eOtP5Jr3feeUeDBw/WwYMH5efnp9TUVLsL+8vfbvX09LR73PLVgBrTW7IPPfSQ/P39FRUVpbS0NKWnpysqKkrdunWzWYzg+PHjcnNz05NPPmltW7p0qR566CFt3LhRmZmZWrt2rQYPHqxTp05pyZIlzng6AOqIST+okRtvvFELFy7UggUL9Mknn+jWW2+16f/lL3+pLVu26Oabb7Yb6+PjI0kqLi62Wy6vqKjIZpvGoFWrVtqzZ49mzJih8ePHyzAMDR06VElJSfL19bVuZxiGSktLVVZWZm3r3bu3tm7dqq1bt+r777/XNddco8GDB+vVV1+122cAXAOBiRpLSEhQYGCgkpKSlJOTI8MwFBgYqJiYGE2bNq3S0OvataukK2+7Xr36z4kTJyRV/HatM3Xv3l1btmypcht/f3+7mbORkZGKjIx0ZGkAGhiBiVqJjo5WdHR0jcbceuutSk5O1oEDB+wC84MPPpCXl1eFCwUAQGPAZ5hoMFFRUfL19dVLL72kkpISa/u///1vvfvuuxo1alSjeksWAH6OM0w0mPbt2+uZZ57R9OnTFRYWpvvvv19nz57Viy++qPbt2+upp55ydokAUCkCEw1q2rRp8vPz09KlSxUfHy9vb28NHTpUiYmJ8vf3d3Z5AFApAhMNbty4cRo3bpyzywCAGuEzTAAATCAwAQAwgcAEAMAEAhMAABMITDQKeXl5OnXqlLPLAIBKEZhoFN577z3dd999pv84MwA0NAITjcLEiRN1/vx5rV+/3tmlAECFCEw0Cm5ubkpOTtajjz6qc+fOObscALBDYKLR6N+/v0aPHq25c+c6uxQAsENgolFZsmSJdu7cqf379zu7FACwQWCiUWnTpo1efPFFTZkyRZcuXXJ2OQBgRWCi0Rk9erS6d++uF154wdmlAIAVgYlGx2Kx6O9//7uWLl2q/Px8Z5cDAJIITDRSAQEBmj17tqZNm8a1mQAaBQITjdasWbP01VdfafPmzc4uBQAITDRe7u7ueuWVVzRjxgx9//33zi4HQDNHYKJRu+222xQREaGEhARnlwKgmSMw0eg9++yzSk1N1cGDB51dCoBmjMBEo9e+fXs999xzmjx5skpKSpxdDoBmisCESxg/frzatm2rl19+2dmlAGimCEy4BIvFohUrVmjJkiU6efKks8sB0AwRmHAZvXv31rRp0/SnP/3J2aUAaIYITLiU+fPn67PPPtPbb7/t7FIANDMEJlyKl5eXli9frunTp+unn35ydjkAmhECEy7nzjvv1JAhQ/TEE084uxQAzQiBCZe0bNkyrV27Vv/5z3+cXQqAZoLAhEvq3LmzlixZoilTpqisrMzZ5QBoBghMuKw//vGPatGihVauXOnsUgA0AwQmXFaLFi30yiuvaMGCBTp16pSzywHQxBGYcGk33XSTHnjgAc2cOdPZpQBo4ghMuLzHH39cBw4c0DvvvGPXt3r1amVkZDihKgBNDYEJl9eqVSu9/PLLmjp1qi5evGjTl5+fr48++shJlQFoSghMNAnDhw9XSEiInn76aZv2zp078/kmgHpBYKLJeOmll7RixQp9/vnn1rZrr71W3377rROrAtBUEJhoMrp27arHH39cDz/8sAzDkHQlMDnDBFAfCEy4rLNnz+rw4cM2bdOmTdNPP/2kN954QxJvyQKoPwQmXFZ+fr5uu+02zZo1Sz/++KMkqWXLlkpOTtbcuXN19uxZ3pIFUG8ITLis0NBQffbZZzp79qz69OmjLVu2yDAM9evXTzExMXr00Ud1zTXX6PLlyyosLHR2uQBcHIEJl9apUyetWbNG69ev14IFCzRixAjl5+dr8eLFeuedd7R//3517tyZs0wAdUZgokkICwtTdna2brvtNoWGhurll1/W888/r8mTJ9fpc8wTJ05o9OjRatOmja655hqNGjVKX331lamxRUVFmjNnjrp06SJvb28NHDhQ7733Xq3qAOB8BCaaDA8PD82fP19ZWVn64IMP9OSTT6pNmzb64YcfanWGWVhYqPDwcH3xxRdau3at1q1bpyNHjuiOO+7QhQsXqh3/4IMPatWqVXryySe1bds2denSRb/73e+UnZ1dm6cHwMncnF0AUN8CAgK0bds2bd26VY888oi++eYbHTp0SFLfGj3OqlWrdOzYMeXl5alnz56SpJtvvlm9evVScnJylevX/uc//9HGjRv12muvaeLEiZKunAUHBQXp8ccfV3p6eq2fHwDn4AwTTZLFYtGoUaOUl5en4cOHy8vLq8aPkZ6ergEDBljDUroSxoMHD1ZaWlq1Y93d3RUTE2Ntc3Nz09ixY7V7924VFxfXuB4AzkVgoknz9fXV22+/rTlz5tR4bE5OjoKDg+3ag4KClJubW+3YgIAA+fj42I29dOmSjh49WuN6ADgXgQlU4ty5c2rXrp1du5+fn86fP1/rseX9AFyLpXwJsYpEREQY3333XQOWU70zZ86oY8eOzi6jWWkq+/zQ199bv76pa5tqt//Xv/6lzp07q2vXrjbtX3/9tU6dOqV+/fpVOvbw4cMqKyvTDTfcYNP+ww8/6MiRI+rdu7d8fX1t+s6cOaPyn7fi4mKFhIRUWyPqT1N5nbuSxrrPP/nkk92GYURc3V5lYEqqstMZQkNDlZWV5ewympWmss/95223fl3wzPBqt+/cubPuueceJScn27RPnTpVqampOnPmTKVjY2JilJ2drby8PJv2lJQUxcTE6LPPPlNQUFCl41u1amVqJi7qT1N5nbuSRrzPLRU18pYsUImgoCDl5OTYtefm5qpPnz7Vjs3Pz7dbYSg3N1ceHh42E4kAuAYCE6jEyJEj9dFHH+nYsWPWtoKCAn3wwQcaOXJktWMvX76s1NRUa1tJSYk2bdqk3/72t/L09HRY3QAcw+UCc9KkSc4uodlprvv8oYcekr+/v6KiopSWlqb09HRFRUWpW7dumjx5snW748ePy83NTU8++aS1LSQkRDExMYqPj9fq1av17rvvauzYscrPz9cTTzxR7ffu0KGDQ54TKtdcX+fO5Gr7nMBEtZrrPm/VqpX27NmjwMBAjR8/Xvfdd58CAgK0Z88emwk7hmGotLRUZWVlNuNff/11TZw4UQkJCRo+fLhOnDihXbt2qW/f6hdQaIwTIZq65vo6dyZX2+cuN+kHqK2aTvpxpkY8GQJoDlxz0s8LL7ygyMhIdenSRRaLRYsWLarR+P3792vQoEHy9vbWtddeq5kzZ+rixYuOKbaJKCsrU2Jiovz9/eXl5aVbbrlFW7ZsMTU2Li5OFovF7hYfH+/gql0Di7k3vLrs84peyxaLhfWAq3Dy5Ek98sgjGjhwoHx8fGSxWFRQUGBqbF2OPQ2h0QfmqlWrdPr0ad1zzz01Hvvpp5/qrrvuUqdOnbRt2zYtWbJEr7/+uuLi4uq/0CZkwYIFWrRokaZPn66dO3dqwIABGjNmjHbs2GFqfMeOHXXgwAGb24wZMxxcdePHYu4Nr677XLryS+DVr+fAwEAHV+66jh49qpSUFLVr105Dhgyp0di6HnsczjCMqm5OV1paahiGYVy+fNmQZCxcuND02Hvuucfo2bOncenSJWvb2rVrDUnGJ598Ut+lNgnffvut4eHhYTz++OM27eHh4cZNN91U7fgJEyYYXbt2dVR5dXL93G3WmzMkJSUZLVq0MI4cOWJtO3bsmNGyZUtj2bJlNtv269fP5n52drYhyXjttdesbZcvXzYCAwONyMhIxxbuwmqyzysiyfjLX/7iyBKbnPJjtmEYxqpVqwxJRn5+frXj6nrsqWcVZmKjP8Ns0aJ2JV6+fFm7du1SdHS03N3dre3R0dHy8PCodvHs5mr37t26dOmSYmNjbdpjY2N16NAh5efnO6ky18di7g2vLvsctVPbY7YrHHsafWDW1pdffqmioiK7xbO9vLzUo0ePahfPbq5ycnLk6elpd2F9+ao0Zvbb6dOn1aFDB7m5uSkwMFDPPvusSktLHVKvK2Ex94ZXl31ebsWKFfL09JSPj4/Cw8P1/vvv13eZUP0cexytyf49zPLFrStbAJvFryt27tw5tW3bVhaL7SQxs4uGh4SEqF+/fgoKClJRUZG2bt2q+fPn68iRI1q9erXD6nYFLObe8Oqyz6UrZzcjRozQL37xCx0/flzPP/+8wsPD9Y9//EO33367Aypuvup67GkIDRqYGRkZuuuuu6rdLiwsTJmZmXX6Xsb/XS5z9c7/eV9zUNN9bhhGnfbZ1bNhhw0bJl9fXyUlJWnu3Lnq1auXucKbqNru27r+vzRnddlv69ats349ZMgQRUVFKTg4WAkJCdq/f3+91QjXeI03aGAOGjRIn3/+ebXbXf22U21U9VvJ+fPnq1z4uimp6T4v/8376hdv+W/j5fu1JsaNG6ekpCRlZWU168Bs165dpa/His6Cfs7Pz6/CSyHq8v/SHNRln1ekdevWGj58uF599dX6KA8/44hjT31r0MD08fGx+3NHjtKjRw95enraLZ5dVFSkY8eOacyYMQ1Sh7PVdJ8HBQWpuLhYX375pc1nCeWfH1S36HhFqjrbb07qupj71q1bVVhYaPMLJYu5V60u+7wylZ0JoW4cceypb0120o+Hh4ciIiKUkpKikpISa/vmzZtVXFxc7eLZzVVERIQ8PDy0YcMGm/b169crODhYAQEBNX7MjRs3ymKxqH///vVVpktiMfeGV5d9XpEffvhB27dv169//ev6LBNyzLGn3lV2vYnRSK7DPHjwoJGammps2rTJkGSMGTPGSE1NNVJTU40LFy5Yt3vggQeMli1b2oz997//bXh5eRm///3vjYyMDGP16tVGu3btjNGjRzf003Apc+fONTw9PY1ly5YZe/fuNaZMmWJYLBYjPT3dZrvw8HCjR48e1vsFBQXGkCFDjL///e/G7t27jfT0dGPixImGxWIxpkyZ0tBPw46zr8P86aefjB49ehjBwcHGW2+9ZaSlpRk333yzERAQYPz444/W7QoKCgxJxhNPPGEzPiYmxmjbtq2xatUqIyMjw7j33nsNT09PrimuQk32ecuWLW32+fPPP2/88Y9/NDZs2GDs3bvXWLNmjREcHGy4u7sb7733njOejssoP0ZPmTLFkGQsX77cSE1NNTIzM63btGzZ0njggQdsxpk99jSACjOx0QfmhAkTDF1Z09bu9vOLYcu3u9q+ffuMAQMGGJ6enkanTp2MP//5zzZBC3slJSXG4sWLje7duxseHh7GTTfdZKSmptptFxYWZlx//fXW+2fPnjWioqKM7t27G56enoaXl5fxq1/9yvjb3/5mczGzszg7MA3DMI4fP26MGjXKaN26teHr62tERUXZXdSdn59f4SIdhYWFxowZM4zOnTsbnp6exq233mrs3bu3wWp3VbXd5+np6cagQYOM9u3bG25uboafn58RGRlp/POf/2zYJ+CCKjtmh4WF2WwzYcIEm3Fmjz0NoMJMZPF1NBssvg7AJNdcfB0AgMaAwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMoBJlZWVKTEyUv7+/vLy8dMstt2jLli2mxsbFxclisdjd4uPjHVw1AEdxc3YBQGO1YMECLV26VE899ZT69eunN998U2PGjNG2bds0bNiwasd37NhR6enpNm1dunRxVLkAHIzABCpw+vRpLV26VPPmzdPs2bMlSXfccYeOHj2qefPmmQpMDw8PDRgwwNGlAmggvCULVGD37t26dOmSYmNjbdpjY2N16NAh5efnO6kyAM5CYAIVyMnJkaenp3r27GnTHhQUJEnKzc2t9jFOnz6tDh06yM3NTYGBgXr22WdVWlrqkHoBOB5vyQIVOHfunNq2bSuLxWLT7ufnZ+2vSkhIiPr166egoCAVFRVp69atmj9/vo4cOaLVq1c7rG4AjkNgolnIyMiwuX91EJYLCwtTZmamDMOocBvDMEx9v6tnww4bNky+vr5KSkrS3Llz1atXL7sxK1eu1MqVKyVJZ86cMfV9ADQcAhPNwqBBg6SMvdb7n3/+eYXb+fj4SLpyJnn+/Hm74Dx//ry1v6bGjRunpKQkZWVlVRiYkyZN0qRJkyRJoaGhNX58AI5FYKJZKA/CcjfccEOV2wcFBam4uFhffvmlzeeY5Z9d9unTp8Y1lJ+dVnZ2C6BxY9IPUIGIiAh5eHhow4YNNu3r169XcHCwAgICavyYGzdulMViUf/+/eurTAANiDNMoAKdOnXSjBkzlJiYqNatW6tv377atGmT9uzZo7S0NJtthw4dquPHj+vo0aOSpOPHj2v8+PEaO3asevbsqeLiYm3dulVr1qzR5MmT1aNHD2c8JQB1RGAClXjqqafk6+url156SadOnVLv3r2VkpKiyMhIm+1KS0tVUlJivd+6dWv5+fnp2Wef1bfffiuLxaIbb7xRf/3rXzV16tSGfhoA6omlmll/5qYEAi7Af95269cFzwx3YiXVCw0NVVZWlrPLAJqrCica8BkmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCQCACQQmAAAmEJgAAJhAYAIAYAKBCVTihRdeUGRkpLp06SKLxaJFixbVaPz+/fs1aNAgeXt769prr9XMmTN18eJFxxQLwOEITKASq1at0unTp3XPPffUeOynn36qu+66S506ddK2bdu0ZMkSvf7664qLi6v/QgE0CDdnFwA0Vjk5OWrRooVKSkr0yiuv1GjswoULdd111yk1NVXu7u6SJA8PD02YMEFz585V3759HVEyAAfiDBOoRIsWtfvxuHz5snbt2qXo6GhrWEpSdHS0PDw8lJaWVl8lAmhABCZQz7788ksVFRUpODjYpt3Ly0s9evRQbm6ukyoDUBdVviV7++2327XFxcUpLi5O3333nUaPHm3X//DDDysmJkYnTpzQ+PHj7fpnzZqlyMhI5eXlafLkyXb9CQkJuvPOO5Wdna34+HibvoIBc6p7PoApFb22K3vtGYYhSTpx4oQk6cMPP9Rjjz1mNz4pKUkhISHavXu3JCkxMVHJycnW/uTkZPn5+enw4cMVfv9169apW7du2rRpk/Ly8uy22bx5szp06KA1a9ZozZo1duN37NghHx8fLV++XPMRzZkAAALqSURBVCkpKXb9mZmZkqSlS5dq27ZtNn3e3t7auXOnJGnx4sV69913bfrbt2+vLVu2SJLmz5+vAwcO2PRfd911Wr9+vSQpPj5e2dnZNv2BgYFauXKlJGnSpEk6fPiwTX9ISIiSkpIkSbGxsTp58qRN/8CBA5WYmChJuvfee3X27Fmb/qFDh2rBggWSpLvvvttuctWIESM0e/ZsSRX/30dHR2vq1KkqLCzUsGHD7PqdedyTpKefflqDBg2q9rWXkZGhJUuW2PUnJyerd+/eevvtt7Vs2TK7/p+/9lasWGHX39xee+X1Xo3PMNHslBUXat++fXbt+/btU1hYmPWHp7bKA7amfRs2bNDmzZt17tw5Xb58uU41AKh/lqp+gCVV2dnQ/Odtd3YJcHHebhbFhvjp3uC2Ffb7+Pioe/fuNm0lJSVyd3fXwoULTV1a8vnnn6tPnz7auHGjxo0bZ9PXp08fBQUFKTU1tcrHCA0NVVZWVrXfC4BDWCpqdKkzzIJnhju7BKBaPXr0kKenp3Jycmzai4qKdOzYMY0ZM8ZJlQGoCyb9APXMw8NDERERSklJUUlJibV98+bNKi4u1siRI51YHYDacqkzTKAhZWVlqaCgQGVlZZKk3Nxcbd68WZI0bNgw+fj4SJIefPBBrV271iYcFy1apIEDByo6OlrTpk1TQUGB5syZo9GjR6tfv34N/2QA1BmBCVTi5Zdf1tq1a633U1NTrZ895ufny9/fX5JUWlqq0tJSm7Hls2Xnzp2r4cOHq02bNrr//vv19NNPN1j9AOqXS036AZoLJv0ATlXhpB8+wwQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADCBwAQAwAQCEwAAEwhMAABMIDABADChuj8gDcAJLBbLLsMwIpxdB4D/j8AEAMAE3pIFAMAEAhMAABMITAAATCAwAQAwgcAEAMCE/weovYTBIgUU4wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bipolar_plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Perceptron – schemat\n",
    "\n",
    "<img src=\"perceptron.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Perceptron – zasada działania\n",
    "\n",
    "1. Ustal wartości początkowe $\\theta$ (wektor 0 lub liczby losowe blisko 0).\n",
    "1. Dla każdego przykładu $(x^{(i)}, y^{(i)})$, dla $i=1,\\ldots,m$\n",
    "    * Oblicz wartość wyjścia $o^{(i)}$:\n",
    "    $$o^{(i)} = g(\\theta^{T}x^{(i)}) = g(\\sum_{j=0}^{n} \\theta_jx_j^{(i)})$$\n",
    "    * Wykonaj aktualizację wag (tzw. _perceptron rule_):\n",
    "    $$ \\theta := \\theta + \\Delta \\theta $$\n",
    "    $$ \\Delta \\theta = \\alpha(y^{(i)}-o^{(i)})x^{(i)} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$$\\theta_j := \\theta_j + \\Delta \\theta_j $$\n",
    "\n",
    "Jeżeli przykład został sklasyfikowany **poprawnie**:\n",
    "\n",
    "* $y^{(i)}=1$ oraz $o^{(i)}=1$ : $$\\Delta\\theta_j = \\alpha(1 - 1)x_j^{(i)} = 0$$\n",
    "* $y^{(i)}=-1$ oraz $o^{(i)}=-1$ : $$\\Delta\\theta_j = \\alpha(-1 - -1)x_j^{(i)} = 0$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Czyli: jeżeli trafiłeś, to nic nie zmieniaj."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$$\\theta_j := \\theta_j + \\Delta \\theta_j $$\n",
    "\n",
    "Jeżeli przykład został sklasyfikowany **niepoprawnie**:\n",
    "\n",
    "* $y^{(i)}=1$ oraz $o^{(i)}=-1$ : $$\\Delta\\theta_j = \\alpha(1 - -1)x_j^{(i)} = 2 \\alpha x_j^{(i)}$$\n",
    "* $y^{(i)}=-1$ oraz $o^{(i)}=1$ : $$\\Delta\\theta_j = \\alpha(-1 - 1)x_j^{(i)} = -2 \\alpha x_j^{(i)}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Czyli: przesuń wagi w odpowiednią stronę."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Perceptron – zalety i wady\n",
    "\n",
    "Zalety:\n",
    "* intuicyjny i prosty\n",
    "* łatwy w implementacji\n",
    "* jeżeli dane można liniowo oddzielić, algorytm jest zbieżny w skończonym czasie"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Wady:\n",
    "* jeżeli danych nie można oddzielić liniowo, algorytm nie jest zbieżny"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "def plot_perceptron():\n",
    "    plt.figure(figsize=(12,3))\n",
    "\n",
    "    plt.subplot(131)\n",
    "    plt.ylim(-0.2,1.2)\n",
    "    plt.xlim(-0.2,1.2)\n",
    "\n",
    "    plt.title('AND')\n",
    "    plt.plot([1,0,0], [0,1,0], 'ro', markersize=10)\n",
    "    plt.plot([1], [1], 'go', markersize=10)\n",
    "\n",
    "    ax = plt.gca()\n",
    "    ax.spines['right'].set_color('none')\n",
    "    ax.spines['top'].set_color('none')\n",
    "    ax.xaxis.set_ticks_position('none')\n",
    "    ax.spines['bottom'].set_position(('data',0))\n",
    "    ax.yaxis.set_ticks_position('none')\n",
    "    ax.spines['left'].set_position(('data',0))\n",
    "\n",
    "    plt.xticks(np.arange(0, 2, 1.0))\n",
    "    plt.yticks(np.arange(0, 2, 1.0))\n",
    "\n",
    "\n",
    "    plt.subplot(132)\n",
    "    plt.ylim(-0.2,1.2)\n",
    "    plt.xlim(-0.2,1.2)\n",
    "\n",
    "    plt.plot([1,0,1], [0,1,1], 'go', markersize=10)\n",
    "    plt.plot([0], [0], 'ro', markersize=10)\n",
    "\n",
    "    ax = plt.gca()\n",
    "    ax.spines['right'].set_color('none')\n",
    "    ax.spines['top'].set_color('none')\n",
    "    ax.xaxis.set_ticks_position('none')\n",
    "    ax.spines['bottom'].set_position(('data',0))\n",
    "    ax.yaxis.set_ticks_position('none')\n",
    "    ax.spines['left'].set_position(('data',0))\n",
    "\n",
    "    plt.title('OR')\n",
    "    plt.xticks(np.arange(0, 2, 1.0))\n",
    "    plt.yticks(np.arange(0, 2, 1.0))\n",
    "\n",
    "\n",
    "    plt.subplot(133)\n",
    "    plt.ylim(-0.2,1.2)\n",
    "    plt.xlim(-0.2,1.2)\n",
    "\n",
    "    plt.title('XOR')\n",
    "    plt.plot([1,0], [0,1], 'go', markersize=10)\n",
    "    plt.plot([0,1], [0,1], 'ro', markersize=10)\n",
    "\n",
    "    ax = plt.gca()\n",
    "    ax.spines['right'].set_color('none')\n",
    "    ax.spines['top'].set_color('none')\n",
    "    ax.xaxis.set_ticks_position('none')\n",
    "    ax.spines['bottom'].set_position(('data',0))\n",
    "    ax.yaxis.set_ticks_position('none')\n",
    "    ax.spines['left'].set_position(('data',0))\n",
    "\n",
    "    plt.xticks(np.arange(0, 2, 1.0))\n",
    "    plt.yticks(np.arange(0, 2, 1.0))\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqwAAADGCAYAAAAXMtIlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAWVUlEQVR4nO3df5DUd33H8dd7cRO5XJYcaUQn1YEh0Qop2IYqmBACiS1YQ1J6RB1Bo2PUOUwnV5z6MzVJUepUClYnJiZq6FG1sM4kqYBOmhFsMFQ5M57lgt4dXEhik4YcJnCXHyu8+8f3e7C3t7e3y+339nN3z8fMdzb3/X73u5/95vPm+9rvfr+fNXcXAAAAEKpUrRsAAAAAlEJgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWEeJmW03Mzez/ymxjsfTz4ZYfmW8fFPB/HvznutmljOzHjP7VbzsXWY2qdrvCZhozOwdZpY1s6fM7GUzO2JmPzazj5jZq4qs311QmyfM7Fkz22lmy2rxHoDxwMw+GdfU14ZY/iYz6zOzg2Z2TsGyt5lZi5k9bmYvmdlRM/upma01s8lDbO/hIrXcY2a7zOy9SbxHDGT8cEDyzOx1kp5Q9AHBJL3V3X9eZL38/xmN7v79guVXSvqxpK+4+8158++V9AFJd0l6On6djKQ3S1ooabKkn0l6t7t3V+t9ARNFHEbvkvQhSccl/UDSIUnnS1om6fWS9kl6l7s/k/e8bkmvlfSP8ayzFdXlNYrq9APu/q+j8y6A8SM+CbNH0lslLXH3XQXLHpb0tiLLvijp05JelrRT0q8l1Uv6c0kXS+qQ9E537yx4vYclXSZpvaRXJKUlXSTpOklnSfqsu38xgbeKfu7OlPAk6VOSXNKX48evD7GeS/qtpBclPSZpUsHyK+N1NhXMvzee/5Yi2zxfUku8/ICk+lrvDyamsTZJ2hDX0COSXluw7GxJX89b/qq8Zd2Sfldke9fH6x+u9XtjYhqrk6IPfy9JOijpnLz5n4zr66sF6zfnHQtnFixLSbo1Xt4h6dyC5Q/Hy+oL5l8u6YSkXkmTa71PxvPEJQGj4wZJz0v6nKTfSHqPmb16iHX/T9LXJP1R/LwRcffnJL1f0oOS3iTp4yPdJjCRmNkbJd0s6TlJy9396fzl7v6ypCZJP5E0X9G3HcPZpuhM7evN7ILqthiYGNz9MUl/L2mGpC9JkpnNknSbpC5FJ4sUzz9f0j8oCrjXuHtXwbZOuvutkr6r6Mzp35bZhocldUqqU3TcRkIIrAkzs8sUBcWsu78kaYuk8yStKPG09YoC7udLBNuyefQxsP+riutHuj1ggvmAon8rv+HuzxZboaDGPljmdi1+zI2secCEtkHSf0tqMrOrJW1W9BX9h9y9N2+96yWdI2mbu3eU2N4X4scPnUFbqOUEEViT13/waokftyj6WmHIg5q790j6J0XXxa2pUjt+qqiY5ha7OQTAkN4ePz40zHo/kfR7SX9Wxk2O71F08HzM3X83wvYBE5a7n1D0beTLiq4tnyfpX9z9JwWrllXH7r5f0jOS3hDff1KSmS1SdO3rEUWXEiAhBJcExXcmXi/psKKDmdz9kJn9VNISM3uDux8e4umbFH19/2kzu9vdXxhJW9z9FTPrkTRN0lRFlx4AGN5r48cnS63k7i+a2XOKaux8na6xV5vZrfF/ny1plqKbrl5U9T6QAhOWux8wsy2SPqwobH6myGpl1XHeOtMkvU7S/xYs+4yZvaIoP10s6a8knZT08fjyICSEwJqsRknnSvpa/JVhvxZFdxveIOn2Yk90914zW6foetZPKLpOZ6Rs+FUAjEB/jeXX+9mSPl+w3ouSlrn77lFpFTCOmdkMSf1DS01TdDb1P0eyyfix2DBKny74+4Sk97r7thG8HsrAJQHJ6v/af0vB/K2KhsW4wcxKhchvKLr7sdnMXjOShpjZ2YrOrJ6Q1DOSbQETTP9NVn9YaqX4evOpimo7v8aed3dzd5M0RdGB9aSkrWZWcpsASouPod9UdInNzYpuqrrbzOoLVi2rjmMXFjwn37lxLddLeqeimzG/bWZzKm07KkNgTYiZzZR0Rfzn/vwBhxUdzM5SdGfjlUNtw91zis7M1CsaYWAk3q7ojPov3f33I9wWMJE8Ej9eNcx6VyiqsZ/H19UN4u4vuPv3JH1E0msUDYcF4Mw1SVos6R53/4qiY+Z0nR77uF9ZdRyPMjBN0ZBzhZcDnOLuve6+U9JKRWH528OcgMIIEViTc4OirxV+rOjTX+F0f7zecHcUf0dSm6SPKirCisVF1P81xr+fyTaACWyzoq8Gb4yHxhmkoMa+PdwG3f07kvZKepeZLaxWQ4GJJL4U4EuKfphnbTx7g6IfymmKb4jqt1XRpTgr4xNKQ+m//vVb5bQhvrnr+5L+VNHNlEgIgTUBZpZSNBTOCUnvc/cPF06KbsZ6TtJfm1lmqG25+0lJn1V0Rrbis6xmNlXRAfcdin7R446K3xAwgbn7ryV9VdIfSLrfzKblLzezs+LlVyoaXqfcX666LX68tSoNBSaQ+EPitxSd3byx/8bk+NuNDyq6NOebZlYXzz+i6OzrqyX9Rxx287eXMrPPSXqfonFV/7mC5tym6EPtLfHxHwngpqtkXK1oSKrtQ32lEN+1/2+S/kbSuyXdPdTG3P0H8c/CXT7M637MzJ5WdGY3o2gQ40Ua+NOsxyt9MwD0CUXjJ79fUoeZFf406xsktUq6Nr6UZ1ju/kMz+5miEUMujwcgB1CeNYo+JN7j7j/KX+Du7WZ2u6IxVdfp9I8AfFnRB8+/k9RuZjsU/ZhP/0+zvlFRWF3m7sfKbYi7/8rM7lM0YsD1kr43gveFIdjAm9dRDWb2XUVfDax092yJ9d4i6VFJe919QXx96y/d/S1F1r1c0n/Ff37F3W/OW3avBv66zglJxxQNzdEqKStpR3y2FsAZMrO/UHR5znxFB77jii7Z+a6kbxWGVTPrlnSeu583xPb+UtHYkQ+5+9UJNh0YN+Kzo79SdD/IJcWGfYzHG98r6U8kXe7uj+QtW6Bo2MiFiq5XfVHRz7VmJd3h7n1FtvewotF9zi124ifveN4u6Y853lYfgRUAAABB41oLAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0IYLrD6a09KlS0f19ZiYzmAK3ajuD2qWKfApdKO6P6hXpjEwDSmoM6xHjhypdRMAVICaBcYO6hVjWVCBFQAAAChEYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKBVPbA++eSTuummm7RgwQLV1dXJzNTd3V3tlwFQBdQrMHZQr5jIqh5YOzs7tXXrVjU0NGjhwoXDP6GrS2pqkjIZqbU1emxqiuYDRXT1dKlpe5My6zNK3ZZSZn1GTdub1NVDn6lUxfWqgfu/9bet7H+URL1WD/WKUZGfy1KpYHKZuXup5SUXFnPy5EmlUlEOvueee3TjjTfq0KFDmj59+uCVd+6UGhulXE7K5TRP0j5JSqejKZuVli2rtAkYx3Z27FTjtkblTuSUO5k7NT+dSis9Ka3syqyWXZxon7EkN14FFdVsRfWqIvv/LkkfHdX9jzGEeh0W9YqwFOSyU0Yvlw1Zs1U/w9pfTMPq6op2Sl/fwJ0iRX/39UXLOdOKWFdPlxq3Naov1zfg4CdJuZM59eX61LitkTMHFSi7XsX+R2XoL9VHvSJRgeey2t10tWHD4B1SKJeTNm4cnfYgeBse2aDcidJ9Jncip4176TNJYP+jEvSX2mL/o2KB57LaBdYtW8rbMS0to9MeBG9L25ZBZwoK5U7m1NJGn0kC+x+VoL/UFvsfFQs8l9UusB4/Xt31MO4df6W8vlDueqgM+x+VoL/UFvsfFQs8l9UusNbXV3c9jHv1Z5XXF8pdD5Vh/6MS9JfaYv+jYoHnstoF1lWrojvOSkmnpdWrR6c9CN6qOauUTpXuM+lUWqvn0GeSwP5HJegvtcX+R8UCz2W1C6xr15a3Y5qbR6c9CN7aBWuVnjTMP8CT0mqeT59JAvsflaC/1Bb7HxULPJclEliz2ayy2axaW1slSTt37lQ2m9Xu3btPrzRzZjSeV13d4B2UTkfzs9loPUDSzKkzlV2ZVV26btCZg3Qqrbp0nbIrs5o5lT5TibLqVex/VIb+kgzqFYkJPJdV/YcDJMms+LivixYt0q5duwbO7OqKhkhoadG8F17QvkwmOt3c3ExYRVFdPV3auHejWtpadPyV46o/q16r56xW8/zm0fjHd1wNRC5VWK8auP9f+OoLytyUGc39jzGGei2JekV48nKZjh+PrlkdvVw2ZM0mEljP1Lx587Rv377RfEmgUuPuADgS1CwCR73moV4xBozeL10BAAAA1URgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0AisAAAACBqBFQAAAEEjsAIAACBoBFYAAAAEjcAKAACAoBFYAQAAEDQCKwAAAIJGYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0AisAAAACBqBFQAAAEEjsAIAACBoBFYAAAAEjcAKAACAoBFYAQAAEDQCKwAAAIJGYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0AisAAAACBqBFQAAAEEjsAIAACBoBFYAAAAEjcAKAACAoBFYAQAAEDQCKwAAAIJGYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABA0AisAAACCRmAFAABA0AisAAAACBqBFQAAAEEjsAIAACBoBFYAAAAEjcAKAACAoBFYAQAAEDQCKwAAAIJGYAUAAEDQCKwAAAAIGoEVAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQNAIrAAAAgkZgBQAAQNAIrAAAAAgagRUAAABBI7ACAAAgaARWAAAABI3ACgAAgKARWAEAABC0qgfWJ554Qo2NjZoyZYoymYxWrFihw4cPV/tlAFQJNQuMHdQrJqqqBta+vj4tWbJEBw4c0ObNm9XS0qKOjg4tXrxYvb29xZ/U1SU1NUmZjNTaGj02NUXzgWLy+0wqRZ8ZAWoWSevq6VLT9iZl1meUui2lzPqMmrY3qauH/lIp6hWjIdiadfdSU0U2bdrkqVTKOzo6Ts07ePCgT5o0yTds2DD4CTt2uNfVuafT7pJfKrlL0d91ddFyIF9Bn/HR7zPD1Uytp4pQs0jSjt/s8Lov1Hn69rTrVp2a0renve4Ldb7jN9RrJahXJC3kmjV3L5lnKwm/V111lV566SXt2bNnwPxFixZJknbv3n16ZleXNGeO1Nd3atY8Sfvyn1hXJ7W1STNnVtIMjFdF+swgyfcZS2rDVULNIghdPV2ac+cc9eWGrte6dJ3aPtammVOp13JQr0hS6DVb1UsC9u/fr0suuWTQ/NmzZ6u9vX3gzA0bpFyu9AZzOWnjxiq2EGMafabqqFkkZcMjG5Q7Ubq/5E7ktHEv/aVc1CuSFHrNVjWw9vT0qKGhYdD8qVOn6ujRowNnbtlSXjG1tFSxhRjT6DNVR80iKVvatih3cpiD38mcWtroL+WiXpGk0Gu25CUBS5cu9SNHjpS9sV/84heaNm2aLrzwwgHzn3rqKT399NO69NJLT89sbR30/MckvbnYhvOfh4mrSJ8ZUkJ9prW19UfuvjSRjVcBNYtQtP62zHo16dLXUa/loF6RpNBrtqrXsE6bNk3XXXed7rrrrgHzm5qatG3bNj377LOnZ2Yy0rFjA9YbdH1N/3rPP19JMzBeFekzQ66XXJ8ZV9fEUbNISmZ9RsdeGb5eM2dn9PynqNdyUK9IUug1W9VLAmbPnq39+/cPmt/e3q5Zs2YNnLlqlZROl95gOi2tXl3FFmJMo89UHTWLpKyas0rpVOn+kk6ltXoO/aVc1CuSFHrNVjWwLl++XHv37tXBgwdPzevu7taePXu0fPnygSuvXVteMTU3V7OJGMvoM1VHzSIpaxesVXrSMAe/SWk1z6e/lIt6RZJCr9mqXhLQ29uruXPnavLkyVq3bp3MTLfccouOHTumtrY21dfXD3zCzp1SY2N04Xcud/rrinQ6mrJZadmyCt8SxrWCPnPK6PWZcfUVIzWLJO3s2KnGbY3KncgNuJkjnUorPSmt7Mqsll1MvZaLekXSgq7ZUoO0nsmIr48//rivWLHCzz33XK+vr/drr73WDx06NPQTOjvd16xxz2SiQY0zmejvzs4zeXlMBHl9xlOp0e4ztR5onJrFmNL5XKev2b7GM+sznrot5Zn1GV+zfY13Pke9nskbol6RtFBrtqpnWEdq3rx52rdv0CXhQEjG1RmbkaJmETjqNQ/1ijFgdG66OhNPPPGEGhsbNWXKFD366KNasWKFDh8+XOtmIWBPPvmkbrrpJi1YsEB1dXUyM3V3d9e6WRMGNYtKUK+1Rb2iEiHXa00Da19fn5YsWaIDBw5o8+bNmjFjhjo6OrR48WL19vbWsmkIWGdnp7Zu3aqGhgYtXLiw1s2ZUKhZVIp6rR3qFZUKuV5rGljvvvtuHTx4UPfdd5+uu+46nXfeeXrggQf0+OOPDxpnDuh3xRVX6JlnntGOHTu0cuXKWjdnQqFmUSnqtXaoV1Qq5HqtaWB94IEHNH/+fF100UWn5s2YMUOXXXaZ7r///hq2DCFLpWp+JcuERc2iUtRr7VCvqFTI9VrTlu3fv1+XXHLJoPmzZ89We3t7DVoEoBRqFhg7qFeMJzUNrD09PWpoaBg0f+rUqTp69GgNWgSgFGoWGDuoV4wnNT/3azZ4BINhhtoCUEPULDB2UK8YL2oaWBsaGtTT0zNo/tGjR4t+KgRQW9QsMHZQrxhPahpYZ8+erf379w+a397erlmzZtWgRQBKoWaBsYN6xXhS08C6fPly7d27VwcPHjw1r7u7W3v27NHy5ctr2DIAxVCzwNhBvWI8qelPs/b29mru3LmaPHmy1q1bp7Vr1+qcc87RsWPH1NbWpvr6+iRfHmNYNpuVJD300EO68847dccdd+iCCy7QBRdcoEWLFiX50hP6px6pWZwJ6nVI1CuCU8N6lUrUbE0DqyQdPnxYzc3NevDBB9Xb26trrrlGmzZt0vTp05N+aYxhxW4kkKRFixZp165dib50khuvAmoWwaFeh0S9Ijg1rFcp5MCab968edq3b99oviRQqQl/AMxHzSJw1Gse6hVjwJA1W/NhrQAAAIBSCKwAAAAIGoEVAAAAQRvuGtZRZWY/dPeltW4HgPJQs8DYQb1iLAsqsAIAAACFuCQAAAAAQSOwAgAAIGgEVgAAAASNwAoAAICgEVgBAAAQtP8HLYi+hk3C0xEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x216 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_perceptron()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Funkcje aktywacji\n",
    "\n",
    "Zamiast funkcji bipolarnej możemy zastosować funkcję sigmoidalną jako funkcję aktywacji."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "outputs": [],
   "source": [
    "def plot_activation_functions():\n",
    "    plt.figure(figsize=(16,7))\n",
    "    plt.subplot(121)\n",
    "    x = [-2,-.23,2] \n",
    "    y = [-1, -1, 1]\n",
    "    plt.ylim(-1.2,1.2)\n",
    "    plt.xlim(-2.2,2.2)\n",
    "    plt.plot([-2,2],[1,1], color='black', ls=\"dashed\")\n",
    "    plt.plot([-2,2],[-1,-1], color='black', ls=\"dashed\")\n",
    "    plt.step(x, y, lw=3)\n",
    "    ax = plt.gca()\n",
    "    ax.spines['right'].set_color('none')\n",
    "    ax.spines['top'].set_color('none')\n",
    "    ax.xaxis.set_ticks_position('bottom')\n",
    "    ax.spines['bottom'].set_position(('data',0))\n",
    "    ax.yaxis.set_ticks_position('left')\n",
    "    ax.spines['left'].set_position(('data',0))\n",
    "\n",
    "    plt.annotate(r'$\\theta_0$',\n",
    "                 xy=(-.23,0), xycoords='data',\n",
    "                 xytext=(-50, +50), textcoords='offset points', fontsize=26,\n",
    "                 arrowprops=dict(arrowstyle=\"->\"))\n",
    "\n",
    "    plt.subplot(122)\n",
    "    x2 = np.linspace(-2,2,100)\n",
    "    y2 = np.tanh(x2+ 0.23)\n",
    "    plt.ylim(-1.2,1.2)\n",
    "    plt.xlim(-2.2,2.2)\n",
    "    plt.plot([-2,2],[1,1], color='black', ls=\"dashed\")\n",
    "    plt.plot([-2,2],[-1,-1], color='black', ls=\"dashed\")\n",
    "    plt.plot(x2, y2, lw=3)\n",
    "    ax = plt.gca()\n",
    "    ax.spines['right'].set_color('none')\n",
    "    ax.spines['top'].set_color('none')\n",
    "    ax.xaxis.set_ticks_position('bottom')\n",
    "    ax.spines['bottom'].set_position(('data',0))\n",
    "    ax.yaxis.set_ticks_position('left')\n",
    "    ax.spines['left'].set_position(('data',0))\n",
    "\n",
    "    plt.annotate(r'$\\theta_0$',\n",
    "                 xy=(-.23,0), xycoords='data',\n",
    "                 xytext=(-50, +50), textcoords='offset points', fontsize=26,\n",
    "                 arrowprops=dict(arrowstyle=\"->\"))\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4sAAAGKCAYAAACl0NTvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3hUZfrG8XvSOxAIEHrokEgNCCKigIIi4KJiWVxAWXApKygqKEVWFHVRwZ+A4NqWoiAugqAioOCKsvSW0CGhk0ACCenJnN8fWQazQwlkMicz8/1cVy7zvmfOzJM45Mmdc857LIZhCAAAAACA3/MyuwAAAAAAQNlDWAQAAAAA2CEsAgAAAADsEBYBAAAAAHYIiwAAAAAAOz7X2c5SqUAxdO/eXd9//73ZZQCuwGJ2AW6A3gwUA70ZKLar9maOLAIOcPbsWbNLAAAAv0NvBkqOsAgAAAAAsENYBAAAAADYISwCAAAAAOwQFgEAAAAAdgiLAAAAAAA7hEUAAAAAgB3CIgAAAADADmERAAAAAGCHsAgAAAAAsENYBAAAAADYISwCAAAAAOwQFgEAAAAAdgiLAAAAAAA7hEUAAAAAgB3CIgAAAADADmERAAAAAGCHsAgAAAAAsENYBAAAAADYISwCAAAAAOwQFgEAAAAAdgiLAAAAAAA7hEUAAAAAgB3CIgAAAADADmERAAAAAGCHsAgAAAAAsENYhNs4fvy4RowYofbt2ysoKEgWi0UJCQnF2tdqtWrKlCmqU6eOAgIC1Lx5c3311VelWzAAAG6O3gy4NsIi3MbBgwe1aNEiVahQQR07dryhfcePH69XXnlFw4cP13fffad27drp4Ycf1rfffltK1QIA4P7ozYBrsxiGca3t19wIlCVWq1VeXoV///jHP/6hP//5zzpy5Ijq1Klzzf2SkpJUs2ZNjRkzRpMmTbLNd+nSRcnJydq5c+d1Xzs2NlabN28uUf2Ah7CYXYAboDfDZdCbAZdw1d7MkUW4jUvN6EatXLlSubm56tevX5H5fv36adeuXTpy5IgjygMAwOPQmwHX5rQji3feeafdXN++fTV06FBlZmbqvvvus9s+YMAADRgwQGfPntVDDz1kt/0vf/mLHnnkER07dkxPPPGE3fbnnntOPXv21L59+zRkyBC77ePGjVPXrl21fft2jRw50m7766+/rttuu02//vqrXnrpJbvt06ZNU4sWLbR69WpNnjzZbvvs2bPVqFEjffPNN3r77bftts+dO1c1a9bUwoULNWvWLLvtixcvVqVKlfTpp5/q008/tdv+7bffKigoSDNnztSiRYvstq9du1aSNHXqVC1fvrzItsDAQH333XeSpFdffVVr1qwpsr1ixYq26wLGjh2r3377rcj2GjVqaN68eZKkkSNHavv27UW2N2zYUHPmzJEkDR48WPv37y+yvUWLFpo2bZqkwh/8x48fL7K9ffv2mjJliiTpwQcf1Llz54ps79Kli8aPHy9Juvfee5WVlVVke0REhBYvXqwjR45owIABtvkLkbE6X6ODDG8/OdKpz0Yqsv80hz4nUBYlvNGjpE/BkcWSozfTm92qN1/y+/de48aNdfz4cd1xxx227QMGDFDTpk116623KiYmRhUrViyy//++97Zs2aLWrVvbtvPeWyvJM997999/v0aPHi3JvX7uXfp/6gAcWQQklUpQBAAAjpWfny8fHx+7+fDwcNv2K5kzZ4569OihLVu2KC8vr1RrBDwB1yzCLV3tuog6Y1aUyutxZBGegiOLZQK9GS7pRq5Z/POf/6zly5fr1KlTReYPHDighg0b6p///OcVj+D8HtcsAsV21d5s/ycbwENc+qX3xRdf1PTp05WVlSWL5fK/lY0bN+rWW2/V8uXL1aPHtX9Bjl09UZtL/ks0AABQ4RHE1NRUGYZRpDenpqbatgPuKCe/QGcv5io5PUdn03N09mKOzmXk6uzFHKVk5ColI1fnLuYqNTNXF3PytXPiPUX+jTgaYREeLzo6Wjk5OTp06JDq169vm4+Pj5ckNW3a1KzSAADwSPRmuBvDMHQ+M08nL2Tp9IVsnbyQrdMXsnQmLUdn0rJ1Ji1bSek5Op95Y6dPZ+YWKNi/9CIdYREer3v37vLz89P8+fM1ceJE2/y8efMUExOjqKgoE6sDAMDz0Jvhii5k5ikxJUPHUrJ0LDVTx1IydTw1SyfOZ+nk+Sxl5hY4/DVTMnIJi0BxLV68WJK0ZcsWSdJ3332niIgIRUREqFOnTlfcp3Llyho1apSmTJmi0NBQtWrVSgsXLtSPP/6opUuXOq12AADcUXF6s4+Pj/r376+PPvpIEr0ZZVdmbr4OJ2fo8NkMHU6+qCNnM5RwLlOJ5zJu+KjglXh7WVQx2E+Vw/xVKaTwo2KInyoF+ys82E/hIX6qGOxX+Hmwn4L8SjfOERbhVh5++OEi46FDh0qSOnXqdM3lhV977TWFhIRo+vTpOn36tBo1aqRFixapZ8+epVkuAABurzi9uaCgQAUFRY+60JthpqzcAu0/k659p9O1/0y6DiRd1MGkizpxPuv6O19FsJ+3IssHKrJcgCLLBahqucLPq4T5q3JogKqEBSg82E/eXmVnLThWQ4VH+f1qqA5Y1dGGFdeAYis7HdB10ZuBYqA3o7iS03O0++QFxZ9MK/w4laaEcxm6dkyy5+/jpdoVg1QrPEg1KgSpZniQalQIVI0KgapePlDlAn1LdTGaEmA1VAAAAACeLT07TzuOXdCO4+e18/h57Tx+QacuZBd7f28vi2pXDFLdSiGqFxGsqEqFH7UrBqtyqL+8ytBRQUcgLAIAAABwO4Zh6GhKpjYeSdHWo6namnhe+5PSi3XE0Msi1akYrEZVQ9WoaqgaVglV/cohqlMxWH4+XqVffBlBWAQAAADg8gzD0JGzGfrt8Dn9duicNiWk6ExaznX3C/D1UtPIMEVXK6foamFqWi1MDauEKsDX2wlVl22ERQAAAAAu6ezFHP1y4Kx+PpCsXw+e0+m0a59S6mWRGlcNU8ta5dW8Rnk1q1lO9SNC5OPtOUcLbwRhEQAAAIBLKLAa2nH8vH7ck6S1+5O0+0TaNR8f6u+j1nUqqE2dcLWqVUHNapQr1fsSuhu+UwAAAADKrKzcAv18IFk/xJ3R2n1JOpeRe9XHhvr76Na6FdW+XkW1qxuuxlXDytStKFwNYREAAABAmXIxJ19r9pzR97tPa+2+ZGXlFVzxcd5eFrWqVV53NIhQx4YRiqkWximlDkRYBAAAAGC67LwCrd2XpGU7TmrNniTl5Fuv+LhKIf7q3DhCnRtXVof6lRQa4OvkSj0HYREAAACAKQzD0KaEVP1r63Gt2HlK6Tn5V3xc/coh6hZdRfc0rapbqpdzu/sZllWERQAAAABOdfpCthZtPqbFW47raErmFR/TuGqoetwSqXtvqar6lUOdXCEkwiIAAAAAJyiwGlq7L0mfbzyqH/cmyWrYP6ZOxSD1al5NPZtXU4MqBESzERYBAAAAlJrzmblauOmY/vlbok6cz7LbHhbgo/ubV9ODrWqoVa3yslg4xbSsICwCAAAAcLiDSen66JcjWrLthLLz7Berua1eRT3atpbuaVpFAb7eJlSI6yEsAgAAAHCISwvWzPn5kFbvSbLbXiHIV31ja+rRtrUUVSnYhApxIwiLAAAAAErEMAz9tC9J7/94UFuPnrfb3jQyTAM61FGv5tU4iuhCCIsAAAAAborVamjVnjP6vx8PaPeJNLvtXZtU0eA76qpNnQpci+iCCIsAAAAAbohhGFq7L1l/X7lP8aeKhkQ/by/1aVVdgzrWVf3KISZVCEcgLAIAAAAoto1HUvT3lXu1KSG1yLy/j5f+eGttDelUV1XCAkyqDo5EWAQAAABwXQeTLmrKt3u0Zm/RhWsCfL30p/Z1NKhjlCqHEhLdCWERAAAAwFWdu5ij6WsOaP5/jqrAatjmfb0terxtLQ3rXJ+Q6KYIiwAAAADs5BdYNXdDot5ZtV/p2fm2eYtF+kOL6hp1d0PVDA8ysUKUNsIiAAAAgCI2JaRo/Ne7tfd0epH5dnXDNa5HU8VUL2dSZXAmwiIAAAAASVJKRq5eW7FHX209XmS+TsUgvdyjqbo2qcwtMDwIYREAAADwcIZhaOn2k/rb8nilZOTa5gN9vTW8c30N6hglfx9vEyuEGQiLAAAAgAc7npqpl5fs1rr9yUXm742pqnH3N1X18oEmVQazERYBAAAAD2QYhhZuOqZXl8crI7fANl+tXIAm/yFGnRtXMbE6lAWERQAAAMDDnEnL1pivduqnfZePJlosUv/2dTS6WyOF+BMTQFgEAAAAPMqKnaf00pJdupCVZ5urWylYf3+4uVrXrmBiZShrCIsAAACAB8jMzdffvonXF5uOFZl/skOUnu/WSIF+LGCDogiLAAAAgJuLP5mmEZ9v1aHkDNtc9fKBmvpwc7WvV9HEylCWERYBAAAAN2UYhhZsPKpJy+KVW2C1zd/fLFKv97lFYQG+JlaHso6wCAAAALihzNx8vbxkt5ZsO2GbC/Lz1iu9ovVw6xqyWCwmVgdXQFgEAAAA3Myh5IsaOm+r9p1Jt801iQzTjMdbqm5EiImVwZUQFgEAAAA38uPeM3rm8+1Kz8m3zT3cuoZefSBGAb4sYoPiIywCAAAAbsAwDH2w7rDeWrlXhlE45+fjpVd7R+uRNrXMLQ4uibAIAAAAuLjsvAK9+NVOLd1+0jZXvXygZj/RWjHVy5lYGVwZYREAAABwYcnpORr0z83acey8ba5tnXDN7NdKlUL8TawMro6wCAAAALiog0npGvDJJh1PzbLNPda2lib1ipafj5eJlcEdEBYBAAAAF/TrobN6eu4WpWUXLmTjZZEm9oxW/9vqmFsY3AZhEQAAAHAxS7ef0OgvdyivoHAlmyA/b/3fYy3VpUkVkyuDOyEsAgAAAC7kk/VHNOmbeNu4cqi/Ph7QhoVs4HCERQAAAMAFGIaht3/Yr/d/Omiba1glRJ8ObKtq5QNNrAzuirAIAAAAlHEFVkPjl+7Wgv8ctc21rl1BH/dvo3JBviZWBndGWAQAAADKsPwCq577ckeReyh2blxZMx5vpUA/bxMrg7sjLAIAAABlVG6+Vc98sU3f7T5tm+vTsrrefKiZfL25NQZKF2ERAAAAKIOy8wo0bP5WrdmbZJvr166W/tYrRl5eFhMrg6cgLAIAAABlTHZegQbP3aKf9yfb5p66PUrjejSRxUJQhHMQFgEAAIAyJCe/QEP+JygOu6ueRt/TiKAIpyIsAgAAAGVEbr5VQ+dt1brfBcWRXRtoZNeGJlYFT8VVsQAAAEAZkFdg1fAFRa9R/Gvn+gRFmIawCAAAAJiswGpo1MLt+iH+jG1u6J31NOpugiLMQ1gEAAAATGQYhsZ9vUvLd56yzQ2+o66e78Y1ijAXYREAAAAwiWEYeuO7vfp84zHbXP/2tTX23sYERZiOsAgAAACYZObaQ5r982HbuE/L6prYM5qgiDKBsAgAAACY4PONR/X3lfts465NqujNh5rJy4ugiLKBsAgAAAA42Zo9Z/Tykl22cfu6FfX+4y3l682v5yg7eDcCAAAATrTtaKqGLdgqq1E4jqkepg/7xyrA19vcwoD/QVgEAAAAnOTI2Qw99dlmZedZJUk1KgTq4wFtFOLvY3JlgD3CIgAAAOAE5y7mqP/HG5WSkStJqhDkq8+ebKvKoQEmVwZcGWERAAAAKGXZeQUaPHeLjqZkSpICfL300YA2qhcRYnJlwNURFgEAAIBSZBiGXvxqp7YkpkqSLBbpvUdbqlWtCiZXBlwbYREAAAAoRe+tOail20/axi/f10T3RFc1sSKgeAiLAAAAQClZtuOk3l293zZ+rG0tPXV7lIkVAcVHWAQAAABKwc7j5/X8lzts49vrV9LfekfLYrGYWBVQfIRFAAAAwMGS0rM1ZO4W5eQX3iKjbkSwZvyxlXy9+fUbroN3KwAAAOBAuflWDZ23VacuZEuSwgJ89FH/NioX6GtyZcCNISzCrRw7dkwPPfSQypUrp7CwMPXp00dHjx4t1r4Wi+WKH9u3by/lqgEAcF+e1psNw9DEZbu1+b8rn3pZpP97vJWiKgWbXBlw43zMLgBwlMzMTHXu3Fn+/v767LPPZLFYNG7cON11113auXOngoOv/0N6wIABGjJkSJG5hg0bllbJAAC4NU/szfP/c1SfbzxmG4+5t7E6NYwwsSLg5hEW4TY+/PBDHT58WPv27VP9+vUlSc2aNVODBg00e/ZsPfvss9d9jurVq6tdu3alXSoAAB7B03rz1qOpmvRNnG38QItq+nPHuiZWBJQMp6HCbSxbtkzt2rWzNSNJioqKUocOHbR06VITKwMAwDN5Um8+ezFHw+ZvVV6BIUmKrhamNx5sxsqncGmERbiNuLg4xcTE2M1HR0crPj6+WM8xa9Ys+fv7KygoSJ07d9a///1vR5cJAIDH8JTenF9g1V8/32Zb0KZcoK8+6NdaAb7eJlcGlAxhEW4jJSVFFSpUsJsPDw9Xamrqdffv16+fZs6cqdWrV2vOnDk6d+6cOnfurLVr117x8XPmzFFsbKxiY2OVnJxc0vIBAHA7ntKb3161X78eOidJslikaY+2UM3wIKe9PlBauGYRbuVKp3oYhlGsfefOnWv7vGPHjurdu7diYmI0btw4/fLLL3aPHzx4sAYPHixJio2NvcmKAQBwb+7em1fFn9GstYds42e6NNBdjSo75bWB0saRRbiNChUqKCUlxW4+NTX1in/VvJ7Q0FD16NFDmzZtckR5AAB4HHfvzcdTM/Xcosu38birUYT+2rmBiRUBjkVYhNuIjo5WXFyc3Xx8fLyaNm16U89pGAYXpgMAcJPcuTfn5ls1fME2pWXnS5Kqlw/Uu4+0kJeX+bUBjkJYhNvo1auXNmzYoMOHD9vmEhIStH79evXq1euGny8tLU0rVqzQrbfe6sgyAQDwGO7cm/++cq+2HzsvSfLxsui9x1qqfJCfyVUBjmW5zjnjxTuhHCgDMjIy1Lx5cwUGBmry5MmyWCwaP3680tPTtXPnToWEhKjOmBW2xz8ZtEUTJkyQJE2dOlX79u3TXXfdpWrVqikxMdE2t2bNGnXs2PGarx0bG6vNmzeX6tcHuAn+5F5y9Ga4jOL0ZklKTExUvXr1NGHCBJfozavjz2jQPy8/90v3NdbgO+qVymsBTnDV3swCN3AbwcHB+vHHHzVq1Cg98cQTMgxDXbp00bRp02zN6PesVqvt80aNGmnJkiVasmSJLly4oLCwMHXo0EEfffSR2rZt68wvAwAAt1Hc3mwYhgoKClyiN588n6XRi3fYxp0bV9ag2+uaVg9QmjiyCI/y+yOLCW/0cNjzcmQRKDaOLJYcvRkohtLozQVWQ49/uEH/OVK4aE9kuQB9+9eOqhDM6adwaVftzVyzCAAAABTDrLUHbUHRyyJNf7QlQRFujbAIAAAAXMeWxFS9u/qAbTyicwO1jQo3sSKg9BEW4RDnz5/Xq6++qhYtWig0NFTh4eHq3Lmzvv32W7NLAwDAI9GbHSctO0/PfLFNBdbCs8Bja1fQiM71Ta4KKH0scIMSW7t2rR5//HGdOnWqyPxPP/2ktWvXatasWRoyZIhJ1QEA4HnozY414evdOp6aJUkKDfDRtEdbyMebYy5wf7zLUSJr167Vfffdp1OnTqlfv37atGmTUlNTtXHjRrVv316GYejZZ5/V8ePHzS4VAACPQG92rGU7Turr7Sdt4zf6NFONCkEmVgQ4D2ERNy0pKUmPPvqosrKy9NZbb2nu3LmKjY1V+fLl1aZNGy1dulQhISHKzMzUggULzC4XAAC3R292rFMXsjRuyS7b+OHWNdSjWaSJFQHORVjETRs9erTOnDmj+++/X88//7zd9oiICHXo0EGStG7dOmeXBwCAx6E3O47Vamj0lzuUlp0vSaoZHqiJvaJNrgpwLsIibsrevXs1f/58WSwWvfXWW1d9XEREhCQpMTHRWaUBAOCR6M2O9emvCVp/8JwkyWKR3unbQiH+LPcBz0JYxE2ZPXu2rFarunbtqiZNmlz1cXl5eUX+CwAASge92XEOnEnXG9/vtY2f7lRPbepwmwx4HsIibpjVatUXX3whSfrjH/94zcempBTeuDYwMLDU6wIAwFPRmx0nr8CqUYu2KzffKklqGhmmUV0bmlwVYA7CIm7Y9u3bdfr0aUnSgAEDZLFYrvqxatUqSVLNmjXNLBkAALdGb3acmT8d0u4TaZIkPx8vTXu0hfx8+JUZnol3Pm7YzVwQ36BBg1KoBAAASPRmR9l94oL+78cDtvFzdzdUwyqhJlYEmIuwiBu2detWSVK3bt2UlZV11Y8lS5bY9mnVqpVZ5QIA4PbozSWXk1+g0V/uUL7VkCS1rl1BgzrWNbkqwFyERdyw/fv3S5Jq1aqlgICAq35s2LDBts8dd9xR5DkWLlyoNm3aKDAwUBUrVlTfvn11+PBhp34dAAC4C3pzyb235oD2nk6XJAX4emnqw83l7WUxuSrAXIRF3LDjx49LksLDr70q2IoVKyRJTZo0Ua1atWzzH3zwgR599FH5+vrq3Xff1bPPPqsff/xR7du319GjR0uvcAAA3BS9uWR2HDuvWWsP2cYvdm+sqErBJlYElA3cLAY3LCsrS5IUEBBw1cfs3btXu3fvliQ98cQTtvmUlBS98MILatGihdatWydfX19J0r333qs2bdropZde0rx580qxegAA3A+9+ebl5Bfo+cU79N+zT9Wubrj6t69jak1AWcGRRdwwPz8/SVJGRsZVHzNz5kxJhctyDxgwwDb/9ddfKz09Xc8884ytGUmF10107txZX331lTIzM0uncAAA3BS9+ebN+OmQ9p+5KEkK9PXWWw82lxennwKSCIu4CbVr15Yk7du374rbjxw5ojlz5kiShg8frsjISNu2jRs3SpI6dOhgt9/tt9+u7Oxs2189AQBA8dCbb078yTTN/OmgbfxC90aqVTHIxIqAsoWwiBvWqVMnSdIPP/ygkydPFtmWkZGhRx55RDk5OWrYsKEmTpxYZPuJEyckSTVq1LB73ktzl667AAAAxUNvvnF5BVY9v/jy6qextStw+inwPwiLuGEDBw6Uj4+PcnJy1KtXL23atEmpqan64Ycf1KFDB23atEnh4eH68ssvFRxc9OLwS6ex+Pv72z1vYGBgkccAAIDioTffuDk/H1bcyTRJkr+Pl956qBmnnwL/g7CIG9akSRPbXyW3bNmitm3bKjw8XN26ddOOHTtUt25drVmzRs2aNbPbNyio8NSOnJwcu23Z2dlFHgMAAIqH3nxjDiZd1PTVB2zjZ+9uqLoRISZWBJRNhEXclHHjxmnhwoVq3769wsLCFBoaqtatW+utt97Srl271KJFiyvuV716dUlXPp3l2LFjkq58GgwAALg2enPxWK2GXvrXLuUWWCVJzWqU01O3R5lcFVA2cesM3LS+ffuqb9++N7RP27ZtNXv2bP32229q0KBBkW3r169XQECAYmJiHFkmAAAeg958fZ9vOqqNCSmSJB8vi958sJl8vDl+AlwJ/zLgVL1791ZISIimT5+u/Px82/y2bdu0Zs0a9enTx+1OdQEAoCzzpN58+kK23vh2r208pFNdNYkMM7EioGzjyCKcqmLFinrjjTc0fPhwderUSX/605907tw5vfvuu6pYsaJee+01s0sEAMCjeFJvnrB0t9JzCgNx3UrBGtG5wXX2ADwbYRFON2zYMIWHh2vq1KkaOXKkAgMD1aVLF02ZMkV16tQxuzwAADyOJ/Tm73ef0g/xZ2zj1/vcogBfbxMrAso+wiJM8dhjj+mxxx4zuwwAAPBf7tyb07LzNGFpnG38WNuaale3ookVAa6BaxYBAADg1qau3Kek9MJbg0SE+mvMvU1MrghwDYRFAAAAuK1tR1M1d0OibTyxZ1OVC/Q1sSLAdRAWAQAA4JbyCqwa+69dMozC8Z2NItTjlkhziwJcCGERAAAAbumT9Ue093S6JCnA10uv9o6RxWIxuSrAdRAWUWbs379fr776qtllAACA/9q7d69ef/11s8u4Kbn5Vr276oBtPLJrQ9UMd4/7RQLOQlhEmREZGak5c+Zo7dq1ZpcCAABU2JtnzJihX375xexSbohhGDp5IUtZeQWSpMZVQ/XU7VEmVwW4HsIiyozQ0FBNnz5dTz/9tHJycswuBwAAj1euXDm9++67evrpp5Wbm2t2OcX2Q/wZpWfnS5IsFum1P9wiX29+7QVuFP9qUKb84Q9/UIMGDfTWW2+ZXQoAAJD08MMPq2bNmnrnnXfMLqVYMnLyNWnZ5XsqPtqmllrXrmBiRYDrIiyiTLFYLHr//fc1bdo0HTx40OxyAADweBaLRTNmzNDUqVN15MgRs8u5rvfWHNDJC9mSpIrBfnqxeyOTKwJcF2ERZU7t2rU1ZswYDR06VMalta4BAIBp6tatq+eee07Dhg0r07153+l0ffTL5UA79r4mKh/kZ2JFgGsjLKJMGjlypE6fPq0vvvjC7FIAAICk5557TomJiVq8eLHZpVyR1Wpo3Ne7lG8tDLPBfj56sFV1k6sCXBthEWWSr6+vZs+ereeee07nz583uxwAADyen5+fZs+erVGjRiktLc3scuz8a9sJbUpIlST5eFlUrXwg91QESoiwiDKrffv26tWrl8aOHWt2KQAAQNLtt9+u7t27a9y4cWaXUsSFzDxN+XaPbTyoY10F+PJrLlBS/CtCmTZlyhQtXbpUGzZsMLsUAAAg6c0339SiRYu0efNms0uxeXvVPp3LKLy1R7VyAfprl/omVwS4B8IiyrQKFSpo6tSpGjJkiPLy8swuBwAAj1exYkW99dZbGjJkiPLz880uR7tPXNC8DYm28fj7myrIz8fEigD3QVhEmffYY4+pcuXKmj59utmlAAAASU888YTKlSunGTNmmFpH4aI2u/XfNW10R8MIdY+pampNgDshLKLMs1gsmjVrlt544w0lJiZefwcAAFCqLvXmV199VcePHzetji+3HNP2Y4UL4fl5e2lSr2gWtQEciLAIl1C/fn0988AjNG8AACAASURBVMwzGjFiRJm+vxMAAJ6iUaNGGjZsmJ555hlTXv98Zq7e+G6vbTz4jrqKqhRsSi2AuyIswmW88MIL2r9/v77++muzSwEAAJLGjh2rXbt2afny5U5/7bd/2K/UzML1DKqXD9Swu1jUBnA0wiJchr+/vz744AP99a9/VXp6utnlAADg8QICAjRz5kwNHz5cGRkZTnvd3ScuaP5/ii5qE+jn7bTXBzwFYREu5c4771SXLl00YcIEs0sBAACSunbtqttvv12TJk1yyusZhqGJy+Jsi9p0bFBJ3aKrOOW1AU9DWITLmTp1qhYsWKCtW7eaXQoAAJD09ttv69NPP9XOnTtL/bX+tfWEtiSmSpJ8vS16hUVtgFJDWITLqVSpkqZMmaIhQ4aooKDA7HIAAPB4VapU0eTJkzVkyBBZrdZSe5207DxN+d2iNk/dXlf1IkJK7fUAT0dYhEsaOHCgAgMDNWvWLLNLAQAAkgYNGiSLxaIPP/yw1F5j+uoDOnsxR5JUNSxAIzqzqA1QmgiLcEkWi0UffPCBJk2apJMnT5pdDgAAHs/Ly0sffPCBxo0bpzNnzjj8+Q+cSddnvybYxi/1aKJgfx+Hvw6AywiLcFlNmzbV4MGDNWrUKLNLAQAAkpo1a6aBAwfq2WefdejzGoahV76JU/5/V7W5NSpcPZtFOvQ1ANgjLMKljRs3Tps3b9b3339vdikAAEDSxIkTtX79eq1atcphz7ky7rTWHzwnSfKyiEVtACchLMKlBQYGaubMmRo6dKgyMzPNLgcAAI8XHBys999/X0OHDlV2dnaJny8rt0CvLt9jGz/RrraaRIaV+HkBXB9hES6vW7duatu2rSZPnmx2KQAAQNL999+vZs2a6fXXXy/xc32w7pBOnM+SJIUH++nZuxuV+DkBFA9hEW7h3Xff1Ycffqi4uDi7bfn5+bpw4YIJVQEA4LmmT5+umTNnau/evXbb8vPzlZaWdt3nOJaSqQ/WHbKNn+/WSOWCfB1aJ4CrIyzCLURGRmrSpEl6+umn7e7v9NNPP2ngwIEmVQYAgGeqUaOGxo8fr6efflqGYRTZtmrVKg0aNOi6z/H6t3uUk1/Y12Oqh6lvbM1SqRXAlREW4TaGDBmi3NxcffLJJ0Xmw8LCdOzYMZOqAgDAcw0fPlzp6en65z//WWS+OL3514Nn9d3u07bxpF7R8vZiURvAmQiLcBve3t6aPXu2XnrpJSUnJ9vmq1atWir3ewIAANd2qTe/+OKLOnfunG3+er05v8CqV765fGlJn5bV1bp2eKnWCsAeYRFupUWLFurXr59Gjx5tm6tSpYrOnDljdwoMAAAofbGxserbt69eeOEF21yVKlV0+vTpq/bmeRsStf/MRUlSkJ+3Xry3sVNqBVAUYRFuZ9KkSVq7dq1++uknSVJAQIACAwOVmppqcmUAAHimyZMna+XKlfr3v/8tSQoJCZGXl5cuXrxo99hzF3P0zqr9tvGIzg1UJSzAabUCuIywCJeWkZGh4cOHKzEx0TYXEhKi9957T08//bRycnIkcSoqAADOkp6erhEjRujo0aO2ubCwME2bNs22voBU2JtPnz5tt//bq/YrLTtfklSnYpCevL2OU+oGYI+wCJcWFBSkyMhItW7dWm+++aby8vIkSb1791aTJk305ptvSrp8ugsAAChdISEhioiIUKtWrTR16lRbb37wwQcVFRWlqVOnSrpyb9594oI+33g5ZI6/v6n8fbydVzyAIgiLcGkWi0Uvv/yyNm7cqHXr1qlly5a2U1z+7//+T++9954OHDhw1b9eAgAAx7JYLJowYYI2bNigVatWqVWrVlq/fr0sFotmzJihd955R4cOHbI768cwDE36Jk6XLmPs1DBCnRtXNumrACARFuEm6tatqxUrVmjSpEl6/PHH9eSTTyowMFAvvfSS/vKXv9gWuQEAAM5Rv359ff/99xo3bpz69u2rQYMGKTQ0VM8//7yGDRtmd2Txm52ntCmhcH0BHy+LJvRsKouFW2UAZiIswm1YLBa1bdtWrVq10vz581W5cmXNnTtXJ0+e1OnTp697ZDE7O1vPP/+8IiMjFRgYqPbt2+vnn392UvUAALgfi8Wi2267TbGxsZo7d64iIiL0xRdf6MiRIzp79qztD7mZufl6fcUe234DO9RRvYgQejNgMh+zCwAcJTMzU507d5a/v78WLlyoI0eO6OWXX1Z+fr4SExPVu3dvqdbV93/qqae0YsUK/f3vf1fdunU1Y8YMdevWTb/99ptatGjhvC8EAAA38b+9+fDhwxo3bpwKCgp0/PhxBQYGSpJmrT2k02nZkqRKIX4a0aWBJHozYDbCItzGhx9+qMOHD2vfvn2qX7++JKlXr15q0KCBrFarfvvtN6nWH6+4744dO7RgwQJ9/PHHGjhwoCSpU6dOio6O1oQJE7Rs2TKnfR0AALiLq/Xmhg0bqqCgQL/++quOpWRq9s+Hbfu80K2xwgJ86c1AGcBpqHAby5YtU7t27WzNSJLq1aunjh07qlWrVnrttdeuua+vr68eeeQR25yPj48effRRrVy50nYLDgAAUHxX6s3169dXx44d1bJlS02ePFmvrdij3HyrJKlZjXJ6qHUN2770ZsBchEW4jbi4OMXExNjNR0dH68iRI3r88cevuW9UVJSCgoLs9s3NzdXBgwcdXi8AAO7uWr05ISFBNVt30fdxl9cUmNgzWl5eFtu+9GbAXIRFuI2UlBRVqFDBbj48PFypqak3ve+l7QAA4MZcszefv6BJ38Tb5v7Qsrpa165QrH0vbQdQuizGpZvZXEH37t2Ns2fPOuSFkpOTFRER4ZDnwvV54vd769atqlKliqpXr15k/sSJEzp9+rRat26tXScu2OZvqV7O9vn+/ftltVrVuHHjIvumpaXpwIEDatSokUJCQopsS05O1qV/Hzk5OVxo70Se+P42m6O+51u2bFlpGEZ3B5TksejNrssTv9/X6s1JaVnyCSv8fnhZLGpYJVS+3pdvlUFvdi2e+P42mzN68zXDoqRrbrwRsbGx2rx5s6OeDtfhid/vKlWq6IEHHtDs2bOLzA8dOlRffvmlkpOTVWfMCtt8whs9bJ8/8sgj2r59u/bt21dk30WLFumRRx7R7t27FR0dfdXXDg4OVkZGhoO+ElyPJ76/zebA7zk3TSs5erOL8sTv99V686Bhz2iV322y+BeGvee7NdKwu+oXeQy92bV44vvbbM7ozZyGCrcRHR2tuLg4u/n4+Hg1bdr0uvseOXJEmZmZdvv6+fkVuTAfAAAUz9V6838yImxBsVZ4kJ66PeqK+9KbAXMRFuE2evXqpQ0bNujw4cvLbyckJGj9+vXq1avXdffNy8vTl19+aZvLz8/XwoULdc8998jf37/U6gYAwF1dqTev2bJXaZWb2cbjejRRgK/3FfelNwPm8n7llVeutf2aG29U69atHfl0uA5P+37fcsst+uKLL7R48WJVr15d+/fv1+DBgxUQEKCPPvpIfn5+mrb6gO3xab9+oU6dOkmSqlatqr1792rGjBmqVKmSUlNTNWbMGG3cuFHz5s1TZGTkNV97zpw5GjVqVKl+fSjK097fZYGDvueTHPEkHu4VRz4Z/5acy9O+3//bm/ft26/hC7bKCK4oSerYoJJG39NIR48eVaVKlSSJ3uzCPO39XRaUdm922jWLgDMcPXpUo0aN0qpVq2QYhrp06aJp06apTp06klTkmsUBAZv0+z+WZGVl6eWXX9aCBQt0/vx5NW/eXG+++abuvPPO674u5+kDxcY1iyVHb4ZL+X1v9q3bVqHdCwOct5dF3z/TUQ2qhCohIUFRUVGaOHEivRlwvqv2ZsIiPMrVFrgpKRoSUGyExZKjN8MlZeUWqOs763TifJYk6ckOUZrQ89prCpQEvRkotrKzwM3+/fv1zDPPqFmzZgoJCVFkZKR69eqlHTt2OLsUj/HOO++oZ8+eioyMlMVi0XVOPUYxHTt2TA899JDKlSunbdu2qU+fPjp69KjZZbmt48ePa8SIEWrfvr2CgoJksViUkJBgdllua/HixXrwwQdVu3ZtBQYGqlGjRho7dqzS09PNLg2lgN7sfJ7Ym2f/fMgWFCsG++mZrg0c/hr0ZueiNzuXGb3Z6WHxhx9+0E8//aT+/fvrm2++0cyZM5WcnKxbb71VW7ZscXY5HuHDDz9UUlKSHnjgAbNLcRuZmZnq3Lmz9u7dq88++0xRUVE6cOCA7rrrLpbpLiUHDx7UokWLVKFCBXXs2NHsctze1KlT5e3trddff13ff/+9/vKXv2jWrFm6++67ZbVazS4PDkZvdj5P683HUzM1a+0h23h0t0YqF+jr0NegNzsfvdm5TOnNhmFc68PhkpOTDavVWmTu/PnzRvny5Y0nnniiNF7S4xUUFBiGYRh5eXmGJGPixInmFmSi2i8ut32UxLRp0wwvLy/jwIEDhmEYRuvWrY3Dhw8b3t7exttvv+2IUvE/Lr2PDcMwPvzwQ0OSceTIEfMKcnNJSUl2c5999pkhyVizZk1Jnvp6fYeP6384HL3Z+TytNw+dt8XWf3u897ORX2C9/k43iN7sfPRm5zKjNzv9yGKlSpVksRQ9LbZcuXJq2LChTpw44exyPIKXF3dIcbRly5apXbt2Re7xFBUVpQ4dOmjp0qUmVua+eB87V0REhN1cmzZtJImf1W6I3ux8nvQz7bdD57Ri1ynb+JWe0fL2cvzly/Rm5/Ok93FZYEZvLhP/h1NSUrR79241adLE7FKAYomLi1NMTIzdfHR0tOLj402oCCh969atkyR+VnsIejMcIb/AqleWxdnGvVtUU2yd8FJ5LXozPFFp9+YyERZHjBghwzA0cuRIs0sBiiUlJUUVKlSwmw8PD1dqaqoJFQGl68SJE5owYYK6du2q2NhYs8uBE9Cb4Qjz/3NU+84ULr4R5OetsfeW3h8f6M3wNM7ozSUOi6tXr5bFYrnux9XuhzNlyhQtWLBA77//fpHTBnBlJf1+w3H+95QtqfAaYMDdXLx4Ub1795aPj48++eQTs8tBMdCbnYvefGUpGbl6+4d9tvGwu+qrarmAUn1NejM8hbN6s09Jn+C2227Tnj17rvu4oKAgu7kPPvhAL730kiZPnqwnn3yypKV4hJJ8v+E4FSpUUEpKit18amrqFf+qCbiq7Oxs9erVS4cPH9a6detUo0YNs0tCMdCbnYvefGVTf9intOx8SVLtikEa1DGqVF+P3gxP4czeXOKwGBQUpMaNG9/wfnPnztXQoUP13HPP6eWXXy5pGR7jZr/fcKzo6GjFxcXZzcfHx6tp09K7wTDgTHl5eXrwwQe1ceNGrV69WrfccovZJaGY6M3ORW+2t/vEBX2+8fL9DSfc31T+Pt6l+pr0ZngCZ/dmU65ZXLJkiQYOHKhBgwZp6tSpZpQAlEivXr20YcMGHT582DaXkJCg9evXq1evXiZWBjiG1WrVH//4R61Zs0ZLly5Vu3btzC4JpYzeDEcxDEOvLIvTpbM/72wUoc6NK5f669Kb4e7M6M0lPrJ4o37++Wc99thjatasmQYMGKANGzbYtvn7+6tly5bOLsntbd68WQkJCbabdcbHx2vx4sWSpPvuu8/jTotxhD//+c96//331bt3b02ePFnnz59X7969VbNmTQ0ZMsTs8tzWpfftpZuEf/fdd4qIiFBERIQ6depkZmluZ9iwYfryyy/18ssvKzg4uMjP6ho1anA6qpuhNzufO/fmr7ef0ObEwgVlfL0tmnB/0yteS+ho9GZz0Judx5TefK2bMJbkzo5XM3HiREPSFT9q165dGi/p8fr373/V77mn3Tj10g2Ba7+4vMTPlZiYaPTp08cIDQ01vLy8jN69e3vc99PZrvY+7tSpk9mluZ3atWtf9ftdwpuHm31De3f4cDh6s/O5a29Oy8o1YievsvXa17+Nd+rr05udj97sPGb0Zotx7RWiWD4KbqXOmBW2zxPe6OGw542NjdXmzZsd9nyAGyv9wwvuj96MMuv1b/dozs+Fp4FWCfPXmufuVIi/009kk0RvBm7AVXtzmbjPIgAAAFzbwaR0ffzLEdv4pfuamBYUATgGYREAAAAlYhiGXlkWr3xr4YHvtnXC1at5NZOrAlBShEUAAACUyMq40/rl4FlJkpdFmtQ72imL2gAoXYRFAAAA3LSs3AK9unyPbfxEu9pqEhlmYkUAHIWwCAAAgJs246eDOnE+S5IUHuynZ+9uZHJFAByFsAgAAICbcuRshm31U0kac29jlQvyNbEiAI5EWAQAAMANMwxDE5fFKbfAKklqWau8HmpVCjcFB2AawiIAAABu2Mq4M/p5f7IkyWKRXu0dIy8vFrUB3AlhEQAAADekcFGbeNu43621FVO9nIkVASgNhEUAAADckPd/OlBkUZvR97CoDeCOCIsAAAAotkPJF4ssavNi90YsagO4KcIiAAAAisUwDE1Yult5BYYkqVWt8nq4dU2TqwJQWgiLAAAAKJZvdp7S+oPnJEleFmnyA7ewqA3gxgiLAAAAuK607Lwii9oMuC1KTauFmVgRgNJGWAQAAMB1vbtqv5LTcyRJlUP9NeruBiZXBKC0ERYBAABwTbtPXNBnvybYxuPvb6rQABa1AdwdYREAAABXVWA19PKSXbIWrmmjDvUr6v5mkeYWBcApCIsAAAC4qvn/SdSO4xckSX7eXnq1d4wsFha1ATwBYREAAABXdCYtW3//fp9tPPSueqobEWJiRQCcibAIAACAK3p1ebzSc/IlSVGVgvV0p3omVwTAmQiLAAAAsLNuf7KW7zxlG7/2QIwCfL1NrAiAsxEWAQAAUERWboHGf73bNv5Dy+q6rX4lEysCYAbCIgAAAIqYtma/jqZkSpLCAnz0co8mJlcEwAyERQAAANjEnbygf/z7iG38co8mqhTib2JFAMxCWAQAAICkwnsqjv3XLhX896aKt0aFq29sTZOrAmAWwiIAAAAkSZ/+mqCdl+6p6OOlKX1u4Z6KgAcjLAIAAEDHUzP19g+X76k44q763FMR8HCERQAAAA9nGIbGfb1bmbkFkqSGVUI0hHsqAh6PsAgAAODhvt5+Qmv3JUuSLBZpSp9b5OfDr4mAp+OnAAAAgAc7ezFHk76Jt437t6+j1rXDTawIQFlBWAQAAPBgE5fF6XxmniSpevlAPd+tkckVASgrCIsAAAAeamXcaa3Yeco2ntLnFgX7+5hYEYCyhLAIAADggS5k5Wn817tt44da19AdDSNMrAhAWUNYBAAA8ECvLo9XUnqOJCki1F/jezQ1uSIAZQ1hEQAAwMP8uPeMFm85bhu/2jta5YJ8TawIQFlEWAQAAPAgFzLzNPZfu2zjns2rqXtMpIkVASirCIsAAAAe5G/L43UmrfD000ohfprUK9rkigCUVYRFAAAAD/Hj3jP6auvl008nP3CLwoP9TKwIQFlGWAQAAPAA5zNzi5x+2qt5NXWPqWpiRQDKOsIiAACAB5iwNO53p5/6c/opgOsiLAIAALi5b3ac1LIdJ23jKX1uUQVOPwVwHYRFAAAAN3YmLVvjvt5tG/eNraG7m1YxsSIAroKwCAAA4KYMw9ALi3fqQlaeJKl6+UCNv7+pyVUBcBWERQAAADc1/z9HtW5/siTJYpHe7ttcoQG+JlcFwFUQFgEAANzQwaSLmrwi3jZ+qkOU2tWtaGJFAFwNYREAAMDN5OZbNXLhNmXnWSVJDSqHaHS3RiZXBcDVEBYBAADczDur9mv3iTRJkp+3l6Y/2lIBvt4mVwXA1RAWAQAA3Mivh85q9s+HbOMXujdS02phJlYEwFURFgEAANzE+cxcPbtwhwyjcNyxQSU92SHK3KIAuCzCIgAAgBswDEMvfrVTp9OyJUkVgnz19sPN5eVlMbkyAK6KsAgAAOAG5m5I1Mq4M7bxmw82U+WwABMrAuDqCIsAAAAuLu7kBU1evsc27t++tu6JrmpiRQDcAWERAADAhWXk5GvEgm3KLSi8TUbTyDCNva+JyVUBcAeERQAAABc2/uvdOnw2Q5IU5Oet9x/nNhkAHIOwCLdhtVo1ZcoU1alTRwEBAWrevLm++uqrYu07YMAAWSwWu4+RI0eWctUAANy8RZuO6V/bTtjGr/0hRnUjQkysqCh6M+DafMwuAHCU8ePHa+rUqXrttdfUunVrffHFF3r44Ye1fPly3XfffdfdPyIiQsuWLSsyFxkZWVrlAgBQInEnL2j80t228UOta+gPLWuYWJE9ejPg2giLcAtJSUmaOnWqxowZo9GjR0uS7rrrLh08eFBjxowpVkPy8/NTu3btSrtUAABKLC07T0Pnb1VOfuF1io2qhOpvvaNNrqooejPg+jgNFW5h5cqVys3NVb9+/YrM9+vXT7t27dKRI0dMqgwAAMcyDEOjF+1Q4rlMSVKwn7dm9mulIL+ydQyA3gy4PsIi3EJcXJz8/f1Vv379IvPR0YV/ZY2Pj7/ucyQlJalSpUry8fFRw4YN9eabb6qgoKBU6gUA4Gb9499H9EP87+6n+FAz1StD1yleQm8GXF/Z+hMUcJNSUlJUvnx5WSyWIvPh4eG27dfSokULtW7dWtHR0crOztaSJUs0duxYHThwQP/4xz+uuM+cOXM0Z84cSVJycrIDvgoAAK7t10Nn9cb3e23jAbfV0f3NqplY0dXRmwHXR1hEmbR69Wrdfffd131cp06dtHbtWhmGYdeMpMJTdYrjf1dWu++++xQSEqJp06bpxRdfVIMGDez2GTx4sAYPHixJio2NLdbrAABws46nZmr4gm0qsBb2tpa1yuslJ95Pkd4MeB7CIsqk2267TXv27Lnu44KCgiQV/pUyNTXVrjGlpqbatt+oxx57TNOmTdPmzZuv2JAAAHCW7LwCPT1vi1IyciVJlUL8NeuPreXn47wriujNgOchLKJMCgoKUuPGjYv9+OjoaOXk5OjQoUNFro24dD1E06ZNb7iGS3/5vNJfRQEAcBbDMPTSv3Zp94k0SZKPl0Wz+rVS1XIBTq2D3gx4Hha4gVvo3r27/Pz8NH/+/CLz8+bNU0xMjKKiom74ORcsWCCLxaI2bdo4qkwAAG7YJ+sT9K9tJ2zjib2i1abOjR+VczZ6M+D6OLIIt1C5cmWNGjVKU6ZMUWhoqFq1aqWFCxfqxx9/1NKlS6+5b2Jiop544gk9+uijql+/vnJycrRkyRJ9+umnGjJkiOrVq+ekrwIAgKLW7kvS5BWXVw3tG1tD/W6tZWJFxXcjvblLly5KTEzUwYMHJdGbgbKCsAi38dprrykkJETTp0/X6dOn1ahRIy1atEg9e/a85n6hoaEKDw/Xm2++qTNnzshisahJkyZ67733NHToUCdVDwBAUQeT0jViwTb9dz0btahZXn/rHeNSp2AWtzcXFBQoPz/fNqY3A2WD5TorUhVvuSrARdQZs8L2ecIbPRz2vLGxsdq8ebPDng9wY67zW27ZRW/2AKkZuXpg5nolnsuUJEWWC9DS4R1UOdS51ym6MnozUGxX7c1cswgAAFCG5OZb9Zf5W2xBMdDXWx/+KZagCMDpCIsAAABlhGEYemnJLm04fPmG9e8+0lwx1cuZWBUAT0VYBAAAKCOmrzmgxVuO28aj72mo7jGRJlYEwJMRFgEAAMqAxVuOa9rqA7bxQ61raNhd9a+xBwCULsIiAACAyX45cFZjvtppG3dsUElT+tziUiufAnA/hEUAAAATxZ28oL/M26L8/94jo3HVUM38Yyv5evNrGgBz8VMIAADAJInnMtT/401Kzym8x2DVsAB9MrCNQgN8Ta4MAAiLAAAApkhKz9YTH23U2Ys5kqTQAB99MrCNIssFmlwZABQiLAIAADhZWnae+n+8SUdTCu+l6O/jpY/6t1GTyDCTKwOAywiLAAAATpSZm69Bn27WnlNpkiRvL4tmPN5KbaPCTa4MAIoiLAIAADhJdl6BBv9zizYmpNjm3uhzi7o2rWJiVQBwZYRFAAAAJ8jNt2ro/K365eBZ29y4Hk30cGxNE6sCgKsjLAIAAJSy/AKrnvlim37cm2Sbe+7uhhrUsa6JVQHAtREWAQAASlF+gVWjFu3Qd7tP2+aG3VVPI7o0MLEqALg+H7MLAAAAcFd5BVaN/GK7Vuw6ZZt7skOURt/TyMSqAKB4CIsAAAClIK/Aqr9+vq3IEcU/ta+t8fc3kcViMbEyACgewiIAAICD5eZbNeLzrVoZd8Y2N+C2OprYsylBEYDLICwCAAA4UGZuvobM3aJ/H7i86ulTt0dpXA+OKAJwLYRFAAAAB7mQlacnP92kLYmptrnBd9TV2HsbExQBuBzCIgAAgAMkp+foTx9v1J5Taba5kV0b6JkuDQiKAFwSYREAAKCEEs9lqP/HG5VwLtM2N+H+pnry9igTqwKAkiEsAgAAlMCOY+f15KebdC4jV5LkZZHeeqi5Hmpdw+TKAKBkCIsAAAA3ac2eMxq+YJuy8gokSf4+Xpr+aEt1j6lqcmUAUHKERQAAgJsw/z+JGv/1blmNwnH5IF/940+xiq0Tbm5hAOAghEUAAIAbkF9g1Wvf7tEn6xNsczUqBOrTgW1Vv3KIeYUBgIMRFgEAAIopLTtPIxZs07r9yba5mOph+nhAG1UODTCxMgBwPMIiAABAMSSczdCgf27WwaSLtrnu0VX1ziPNFeTHr1QA3A8/2QAAAK5jzZ4zGrlwu9Kz821zw++qr2fvbigvL+6hCMA9ERYBAACuwmo1NG3NAb235oBtzs/HS39/qJl6t6huYmUAUPoIiwAAAFeQmpGrUYu2a+2+y9cnVi8fqFn9WqlZjfImVgYAzkFYBAAA+B+bElL+v727D6qqzuM4/rnI40VFEFAUCEUBn0YWzUm0pJRSkHJTaXfGihlnNW2mMdO1Wk1mZK3ZSddm3TTbHXVqZ0fBKZNVbHXXhronKQAACypJREFUWM0pwVDB5weSBx8BEbtCXO7dP1pZr8dSE7nce9+vf8Tf9xzvd4Yz53c/nnN+R6/8/Rudq29sHRvVr7v+9OskhQT6OrEzAGg/hEUAAID/sdnsWvXFKS3/53G13HiBoqSZY/pq/pPx8u7k5cTuAKB9ERYBAAAknau/rnm5B7TnZE3rWDezj5ZNHaqxA3o4sTMAcA7CIgAA8HifHajWwk8O6epNq50+HBOs9371C/XqFuDEzgDAeQiLAADAY9VbmvXWZ6XaXFLdOmYySbNTYvXquDhuOwXg0QiLAADAI31edl4LPy3VxYam1rHI4AAtz0zUiD4hTuwMADoGwiIAAPAoNdealL3lsLYcqHYYnzIsUoszBqqLv4+TOgOAjoWwCAAAPILdbtem/VVauvWIar/7vnU8tLOfciYN1vjBPZ3YHQB0PIRFAADg9k5caNDvPi3V12dqHcYnJ0Vq0cQB6mbm3YkAcCvCIgAAcFvXmqxa+a+T+st/Tst603sTewX56/fPDtHj8eFO7A4AOjbCIgAAcDs2m12b9lfqD9uP6dJNC9h4e5k0fXQfvTK2vwL9+BoEAD+FsyQAAHAr+8prtST/sA5W1juMPxwTrJxJQxTfs4uTOgMA10JYBAAAbuHY+Qb9oeCodh696DDeo6ufXp+QoGeG9paXl8lJ3QGA6yEsAgAAl1ZRa9F7O09o0/5K2f//WKL8vL0047G+emlMLLecAsDPwJkTAAC4pMo6i/7875PKLap0WLzGZJJ+mdhbr6bGKSrE7MQOAcC1ERYBAIBLOVtj0erCU8otqlBzi92h9nh8mH47PkEDIro6qTsAcB+ERQAA4BIOV1/V6i9OKf9gtWyOGVEj+oRobmqcHunb3TnNAYAbIiwCAIAOy263q/DEZf119xkVHr9kqD8cE6xXx8VpZGx3mUwsXgMAbYmwCAAAOhzL91Z98k2V1u4p18mL1wz1R/uHataYWEIiADxAhEUAANBhnLjQoL99dVab9leqodHqUDOZpLTBEXppTKyGRAY5qUMA8ByERQAA4FSW763adui8NhRV6OsztYZ6Zz9vZQ6PUlZyjKK7s7opALQXwiIAAGh3drtdxd/WKa+4UvkHz+lak9WwTZ/QQE175CFlDo9UF38fJ3QJAJ6NsAgAANrN8QsN2lxSpc0l1aqsu26od/Iy6cmBPTTtkYeUzPOIAOBUhEUAAPBAnbjQoK2Hzmtb6TkdPd9w2236hgVq6rAoPZvUWz26+rdzhwCA2yEsAgCANmWz2XWg8op2HLmgz8su6MRtVjOVpK7+3kobEqGpwyOVFB3MVUQA6GAIiwAA4L41NDZrz8nL2nXsknYevahLDU233c7P20tjB4TrmcTeSokPk593p3buFABwtwiLAADgnrXY7CqtqteeU5dVePySisrrZLXZb7utv4+XnkgI14TBEXoiIVyBfnz9AABXwNkaAADckc1m17ELDfrqdI32nq7R3lM1utpoXMH0hpBAXz2REK7UgT30aP9QmX35ygEAroYzNwAAMGhsblFpVb2Kvq1TUXmdir6t1RVL80/uM6hXV6XEhyklPlxJ0cHq5MUziADgygiLAAB4OJvNrvKa73Sg8ooOVNSrpOKKyqrr1dxy+9tKbwjv4qfR/UKV3C9Uj8WFKrwLq5gCgDshLAIA4EFsNrtOX76m0qqrOlRVr9Kqeh2uvqqGph+/pfSG0M6+GtEnRCNiQjSqX6j6hXdmBVMAcGOERQAAPEijtUVP/rFQP7IWjYO+YYEaFh2sYQ8Fa3hMiGLDAgmHAOBBCIsAAHgQs6+3YsM6G959GGz20dCobhoa2U2J0T/8GRLo66QuAQAdAWERAAAP82j/MEWFmDW4V1cN7h2kwb2DFBHkz1VDAIADwiIAAB7mrYyBzm4BAOACvJzdAAAAAACg4yEswm0sX75cGRkZioiIkMlkUnZ29j3tv3v3biUnJysgIEA9e/bU3Llzdf369QfTLAAAHoC5GXBthEW4jQ8//FAXL17UpEmT7nnfgwcPKjU1VeHh4crPz1dOTo7Wrl2rrKystm8UAAAPwdwMuDaeWYTbKCsrk5eXl6xWq1avXn1P+y5evFiRkZHKzc2Vj4+PJMnX11cvvviiFixYoKSkpAfRMgAAbo25GXBtXFmE2/Dy+nmHc3NzswoKCpSZmdk6GUlSZmamfH19tXnz5rZqEQAAj8LcDLg2k93+k2/lvYtX9t5ZzOv/aIt/BmhT5e+ky2KxKCUlRfv27dOAAQMUHh4uScrKylJWVpbi4uJUV1enQYMGOew7a9YsPffcc6qoqNDzzz+v4uJiDRs2rLX+2muvKSMjQ8eOHdPMmTMNn71w4UKNGzdOJSUlmjNnjqG+dOlSJScn68svv9Sbb75pqK9YsUKJiYnasWOHcnJyDPUPPvhA8fHx2rJli5YtW2aof/TRR4qKitKGDRu0atUqQz0vL0+hoaFat26d1q1bZ6hv3bpVZrNZ77//vjZu3Gio79q1S5L07rvvKj8/36EWEBCgbdu2SZKWLFminTt3OtS7d++uTZs2SZLeeOMN7d2716EeGRmpjz/+WJI0Z84clZSUONTj4uK0Zs0aSdKMGTN0/Phxh3piYqJWrFghSZo2bZoqKysd6iNHjtTbb78tSZo8ebJqamoc6mPHjtWiRYskSRMmTDA8OzNx4kTNmzdPkpSSkqJbZWZmavbs2bJYLEpLSzPUbxx7ly9f1pQpUwz1W4+9Wz2oY+/G77QN8G6G+9cmc7PkXsfoDZwfO+b50W63q7CwUIsXL1Z2dvYdjz3mZo69mzE3O29u5soiPJKPydb6s9VqlSR5exvvyg4ODm6t32rNmjVKT09XcXGxmpubH0yjAAB4IOZmoGPgyiI8TqBvJ80ZF6ffPNZXkrRnzx6NHj1aBQUFeuqppxy2HTVqlPz9/Q3/w3ar4cOHq6io6IH1DLgRrizevza7sgjcix07dig1NfWO240ZM8ZwxcNqtcrHx6f1yuKdMDcD7epH5+Z2WeCm/J309vgYuBGLxaKzZ8/ecTuz2azo6GiHsXudkEJCQiRJtbW1htrtbnMBAMATJScn68iRI3fczmw23/dnMTcDHQOroaJDMpvNSkhIaJfPio2NlZ+fn8rKyhzGGxsbdfr0aU2dOrVd+gAAoCNjbgY8D88swuP5+vpq/Pjx2rhxo8MzEHl5eWpqatLTTz/txO4AAPA8zM1Ax8CVRbiNoqIilZeXy2b7YfGaw4cPKy8vT5KUlpbWelvM9OnTtX79eofJJzs7WyNHjlRmZqZefvlllZeXa/78+ZoyZYrDSmoAAODuMTcDro2wCLexcuVKrV+/vvXvubm5ys3NlSSdOXNGMTExkqSWlha1tLQ47JuYmKjt27drwYIFSk9PV1BQkF544QUtXbq03foHAMDdMDcDrq1dVkMF3B0rrgF3jdVQ7x9zM3AXmJuBu8Z7FgEAAAAAd4+wCAAAAAAwICwCAAAAAAwIiwAAAAAAA8IiAAAAAMCAsAgAAAAAMCAsAgAAAAAMCIsAAAAAAAPCIgAAAADAgLAIAAAAADAgLAIAAAAADAiLAAAAAAADwiIAAAAAwICwCAAAAAAwICwCAAAAAAwIiwAAAAAAA8IiAAAAAMCAsAgAAAAAMCAsAgAAAAAMCIsAAAAAAAPCIgAAAADAgLAIAAAAADAgLAIAAAAADAiLAAAAAAADk91ud3YPgMszmUwFdrt9vLP7AAAAP2BuBu4fYREAAAAAYMBtqAAAAAAAA8IiAAAAAMCAsAgAAAAAMCAsAgAAAAAMCIsAAAAAAIP/Ao9YXFZFum1OAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x504 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_activation_functions()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Perceptron a regresja liniowa"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"reglin.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Uczenie regresji liniowej:\n",
    "* Model: $$h_{\\theta}(x) = \\sum_{i=0}^n \\theta_ix_i$$\n",
    "* Funkcja kosztu (błąd średniokwadratowy): $$J(\\theta) = \\frac{1}{m} \\sum_{i=1}^{m} (h_{\\theta}(x^{(i)}) - y^{(i)})^2$$\n",
    "\n",
    "* Po obliczeniu $\\nabla J(\\theta)$, zwykły SGD."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Perceptron a dwuklasowa regresja logistyczna"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"reglog.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Uczenie dwuklasowej regresji logistycznej:\n",
    "* Model: $$h_{\\theta}(x) = \\sigma(\\sum_{i=0}^n \\theta_ix_i) = P(1|x,\\theta)$$\n",
    "* Funkcja kosztu (entropia krzyżowa): $$\\begin{eqnarray} J(\\theta) &=& -\\frac{1}{m} \\sum_{i=1}^{m} [y^{(i)}\\log P(1|x^{(i)},\\theta) \\\\ && + (1-y^{(i)})\\log(1-P(1|x^{(i)},\\theta))]\\end{eqnarray}$$\n",
    "\n",
    "* Po obliczeniu $\\nabla J(\\theta)$, zwykły SGD."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Perceptron a wieloklasowa regresja logistyczna"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"multireglog.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Wieloklasowa regresji logistyczna\n",
    "* Model (dla $c$ klasyfikatorów binarnych): \n",
    "$$\\begin{eqnarray}\n",
    "h_{(\\theta^{(1)},\\dots,\\theta^{(c)})}(x) &=& \\mathrm{softmax}(\\sum_{i=0}^n \\theta_{i}^{(1)}x_i, \\ldots, \\sum_{i=0}^n \\theta_i^{(c)}x_i) \\\\ \n",
    "&=& \\left[ P(k|x,\\theta^{(1)},\\dots,\\theta^{(c)}) \\right]_{k=1,\\dots,c} \n",
    "\\end{eqnarray}$$\n",
    "* Funkcja kosztu (**przymując model regresji binarnej**): $$\\begin{eqnarray} J(\\theta^{(k)}) &=& -\\frac{1}{m} \\sum_{i=1}^{m} [y^{(i)}\\log P(k|x^{(i)},\\theta^{(k)}) \\\\ && + (1-y^{(i)})\\log P(\\neg k|x^{(i)},\\theta^{(k)})]\\end{eqnarray}$$\n",
    "\n",
    "* Po obliczeniu $\\nabla J(\\theta)$, **c-krotne** uruchomienie SGD, zastosowanie $\\mathrm{softmax}(X)$ do niezależnie uzyskanych klasyfikatorów binarnych."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Przyjmijmy: \n",
    "$$ \\Theta = (\\theta^{(1)},\\dots,\\theta^{(c)}) $$\n",
    "\n",
    "$$h_{\\Theta}(x) = \\left[ P(k|x,\\Theta) \\right]_{k=1,\\dots,c}$$\n",
    "\n",
    "$$\\delta(x,y) = \\left\\{\\begin{array}{cl} 1 & \\textrm{gdy } x=y \\\\ 0 & \\textrm{wpp.}\\end{array}\\right.$$\n",
    "\n",
    "* Wieloklasowa funkcja kosztu $J(\\Theta)$ (kategorialna entropia krzyżowa):\n",
    "$$ J(\\Theta) = -\\frac{1}{m}\\sum_{i=1}^{m}\\sum_{k=1}^{c} \\delta({y^{(i)},k}) \\log P(k|x^{(i)},\\Theta) $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Gradient $\\nabla J(\\Theta)$:\n",
    "$$ \\dfrac{\\partial J(\\Theta)}{\\partial \\Theta_{j,k}} = -\\frac{1}{m}\\sum_{i = 1}^{m} (\\delta({y^{(i)},k}) - P(k|x^{(i)}, \\Theta)) x^{(i)}_j \n",
    "$$\n",
    "\n",
    "* Liczymy wszystkie wagi jednym uruchomieniem SGD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Podsumowanie\n",
    "\n",
    "* W przypadku jednowarstowej sieci neuronowej wystarczy znać gradient funkcji kosztu.\n",
    "* Wtedy liczymy tak samo jak w przypadku regresji liniowej, logistycznej, wieloklasowej logistycznej itp.\n",
    "  * Wymienione modele to szczególne przypadki jednowarstwowych sieci neuronowych.\n",
    "* Regresja liniowa i binarna regresja logistyczna to jeden neuron.\n",
    "* Wieloklasowa regresja logistyczna to tyle neuronów ile klas."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Funkcja aktywacji i funkcja kosztu są **dobierane do problemu**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 9.2. Wielowarstwowe sieci neuronowe\n",
    "\n",
    "czyli _Artificial Neural Networks_ (ANN) lub _Multi-Layer Perceptrons_ (MLP)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"nn1.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Architektura sieci\n",
    "\n",
    "* Sieć neuronowa jako graf neuronów. \n",
    "* Organizacja sieci przez warstwy.\n",
    "* Najczęściej stosowane są sieci jednokierunkowe i gęste."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* $n$-warstwowa sieć neuronowa ma $n+1$ warstw (nie liczymy wejścia).\n",
    "* Rozmiary sieci określane poprzez liczbę neuronów lub parametrów."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Sieć neuronowa jednokierunkowa (_feedforward_)\n",
    "\n",
    "* Mając daną $n$-warstwową sieć neuronową oraz jej parametry $\\Theta^{(1)}, \\ldots, \\Theta^{(L)} $ oraz $\\beta^{(1)}, \\ldots, \\beta^{(L)} $ liczymy:<br/><br/> \n",
    "$$a^{(l)} = g^{(l)}\\left( a^{(l-1)} \\Theta^{(l)} + \\beta^{(l)} \\right). $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"nn2.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Funkcje $g^{(l)}$ to tzw. **funkcje aktywacji**.<br/>\n",
    "Dla $i = 0$ przyjmujemy $a^{(0)} = \\mathrm{x}$ (wektor wierszowy cech) oraz $g^{(0)}(x) = x$ (identyczność)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "* Parametry $\\Theta$ to wagi na połączeniach miedzy neuronami dwóch warstw.<br/>\n",
    "Rozmiar macierzy $\\Theta^{(l)}$, czyli macierzy wag na połączeniach warstw $a^{(l-1)}$ i $a^{(l)}$, to $\\dim(a^{(l-1)}) \\times \\dim(a^{(l)})$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "* Parametry $\\beta$ zastępują tutaj dodawanie kolumny z jedynkami do macierzy cech.<br/>Macierz $\\beta^{(l)}$ ma rozmiar równy liczbie neuronów w odpowiedniej warstwie, czyli $1 \\times \\dim(a^{(l)})$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* **Klasyfikacja**: dla ostatniej warstwy $L$ (o rozmiarze równym liczbie klas) przyjmuje się $g^{(L)}(x) = \\mathop{\\mathrm{softmax}}(x)$.\n",
    "* **Regresja**: pojedynczy neuron wyjściowy jak na obrazku. Funkcją aktywacji może wtedy być np. funkcja identycznościowa."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "* Pozostałe funkcje aktywacji najcześciej mają postać sigmoidy, np. sigmoidalna, tangens hiperboliczny.\n",
    "* Mogą mieć też inny kształt, np. ReLU, leaky ReLU, maxout."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Uczenie wielowarstwowych sieci neuronowych"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Mając algorytm SGD oraz gradienty wszystkich wag, moglibyśmy trenować każdą sieć."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Niech:\n",
    "$$\\Theta = (\\Theta^{(1)},\\Theta^{(2)},\\Theta^{(3)},\\beta^{(1)},\\beta^{(2)},\\beta^{(3)})$$\n",
    "\n",
    "* Funkcja sieci neuronowej z grafiki:\n",
    "\n",
    "$$\\small h_\\Theta(x) = \\tanh(\\tanh(\\tanh(x\\Theta^{(1)}+\\beta^{(1)})\\Theta^{(2)} + \\beta^{(2)})\\Theta^{(3)} + \\beta^{(3)})$$\n",
    "* Funkcja kosztu dla regresji:\n",
    "$$J(\\Theta) = \\dfrac{1}{2m} \\sum_{i=1}^{m} (h_\\Theta(x^{(i)})- y^{(i)})^2 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Jak obliczymy gradienty?\n",
    "\n",
    "$$\\nabla_{\\Theta^{(l)}} J(\\Theta) = ? \\quad \\nabla_{\\beta^{(l)}} J(\\Theta) = ?$$\n",
    "\n",
    "* Postać funkcji kosztu zależna od wybranej architektury sieci oraz funkcji aktywacji."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "$$\\small J(\\Theta) = \\frac{1}{2}(a^{(L)} - y)^2 $$\n",
    "$$\\small  \\dfrac{\\partial}{\\partial a^{(L)}} J(\\Theta) = a^{(L)} - y$$\n",
    "\n",
    "$$\\small \\tanh^{\\prime}(x) = 1 - \\tanh^2(x)$$"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "livereveal": {
   "start_slideshow_at": "selected",
   "theme": "white"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}