{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Uczenie maszynowe UMZ 2019/2020\n",
    "### 2 czerwca 2020\n",
    "# 12. Rekurencyjne sieci neuronowe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## RNN – _Recurrent Neural Network_\n",
    "\n",
    "## LSTM – _Long Short Term Memory_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAUDBAcJCAgJCAgKCAgGBwgIBwcJCQYICAgICAgHCAkI\nCQgIChANCAkQCggJDiEODhERHx8TCAsiJCISJBwSExIBBQUFCAcIDwkJDx4NDw0SEhISEhUeEhIS\nEhISEhISEhISEhUVEhISEhISEhISEhISFRUSEhUSEhUSEhISEhUSEv/AABEIAWgB4AMBIgACEQED\nEQH/xAAcAAEAAgMBAQEAAAAAAAAAAAAABQYCBAcDAQj/xABKEAACAgEDAgIHBQUFBgQEBwABAgAD\nBAUREhMhBhQWIjFRU5LTBxUjQZEyYZXU1SQzQlW1VHF1gZShQ0RidDRSsfAIJWRlcpOl/8QAGgEB\nAAMBAQEAAAAAAAAAAAAAAAECAwQFBv/EADQRAAIAAggEBgEFAQADAQAAAAABAhEDBBIhMVFhkRVB\nUvATFHGBocEiMrHR4fFCcqLSBf/aAAwDAQACEQMRAD8A/GUREAREQBERAEREAREQBERAEREAREQB\nERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQ\nBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAEREARE\nQBERAEREAREQBERAEREAREQBERAEREAREQBERAPu0bS1+guX8Sn5rvpx6C5fxKfmu+nO3h1Y6Gcv\nnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY89QdaKptG0tfoLl/Ep+a76ceguX8Sn5rvpxw\n6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY89QdaKptG0tfoLl/Ep+a76ceguX8Sn5\nrvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY89QdaKptG0tfoLl/Ep+a76cegu\nX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY89QdaKptG0tfoLl/Ep+a7\n6ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY89QdaKptG0tfoLl/\nEp+a76ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY89QdaKptG0t\nfoLl/Ep+a76ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY89QdaK\nptG0tfoLl/Ep+a76ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY8\n9QdaKptG0tfoLl/Ep+a76ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOH\nVjoY89QdaKptG0tfoLl/Ep+a76ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy/iU/N\nd9OOHVjoY89QdaKptG0tfoLl/Ep+a76ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXfTj0Fy\n/iU/Nd9OOHVjoY89QdaKptG0tfoLl/Ep+a76ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4lPzXf\nTj0Fy/iU/Nd9OOHVjoY89QdaKptG0tfoLl/Ep+a76ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr9Bcv4\nlPzXfTj0Fy/iU/Nd9OOHVjoY89QdaKptG0tfoLl/Ep+a76ceguX8Sn5rvpxw6sdDHnqDrRVNo2lr\n9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY89QdaKptG0tfoLl/Ep+a76ceguX8Sn5rvpxw6sdDHnqDrRV\nNo2lr9Bcv4lPzXfTj0Fy/iU/Nd9OOHVjoY89QdaOhxET7s+HES3/AGN6bj5WuYePlVLfRbXqBelx\nujGrTM65CR+6ytWH71E1/D/gnIyMOnMuzMDTsbMd6sSzUcp8fzbUkJc1K11WE1o5Cl2CqCdt5jHW\nIYInC9Pmf8M2hoIooU1r8S/krES4YP2baq+TqWKwx8a3Q66rc85OQlNSUWugGQt2xRqhW4tLbj1B\n2DHZT4a14Ezsc4HRfG1GvWrWowb9PuN9NuSliVvjk2V1mq0NYnZ1A2JO/Y7QqzRtytLtT/a8OgpJ\nTsvty/e4q0S85P2Y5y15z1Z+mZJ0fGvyNQpxsy227GXHUtYjo1C833DJuhYBlIJXtvRpajpoaT9L\nKx0UUH6kIiJqZiIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIJ\nPhMlPE+jPg5Jx3uoyCKca4X41tV9DLk49WQAllZIbbqcd/z47jsQTGSU8U/39f8Aw3SP9J0+Ztu0\nlo/oskrLeq+yLiImhQREQSIiIIEREAREQBERAEREAREQC4/Yrn0Y2u4V2TclFNdeoB7bXrrrU2aX\nn1IGdyACzuij3lgPzklk41GraVoi06hgYWTo2NdgZmLn5deCoDZD3VZdDWja9GRjyCbsGUDZvbOe\nT4Zy0lWtx205O74tf/R00dYlBYamr/mX8HYfEXibTrKfEFNGWtiUeHtE0rDvdum+ovgZNAutqrfZ\nn39c7DfdU39hmr9n3irEwMLw67st1mB4lzbsnEQh8lMa/Eoo64oXdyAGZh27msgTlAP/AHmzpmbf\njX05GPYar8W1LqbV48q7K2DKwDAg7EDsQQfYQQZk6jDYcPf6bJoq5FbT9v8A2mdW8FeGMXDr8SWV\n6xg54fwrrC4tWJbbdfZjscb8fJXpgYjD8IGtzuXt7bhSTyGWzW/tAzsijIoXHwcJdQI+8LcHDpxb\ns0B+pxvtUklTZ6xVOO5333BIlTmlVo44ZxR4uXx7IpWKSByUGCn87iIidZyiIiAIiIAiIgCIiAIi\nbOLihgtllgrpNnBnBqstHqGz8PG6ivZ2AAbsu7oCV33kNyJSNaemNRZYQtVbWMzKoFaO5LNuFUBQ\nSWOx2H57Gevma1TilK8npCWPaVuPPluz0jgq07jYdw5Gx2O5nzMz77eXVtewWPzZWZuBcKEDdP8A\nZBCKqjYdlVQNgAJE2Tceq6RmkKwxMgrZb0UIoyCHuDFDSpC7NZyVhxHfdSNu0JpGaxULiZDGxrVQ\nCjIJdqN+sqgJ6zJsdwPZsd9pocR7h+kcR7h+kX9/6TNG592ZXHn5a7h0PM8+jfx8vvt5jlx26O//\nAIns/fM30jNUsGxMhTW1SuDRkAo1+3RVgU9Vn3GwPt3G280OI9w9/sjiPcP0i/v/AETRILo2cSFG\nJklmuagKMfI5G9FLPQBw3NoUElPaACdpgmlZZAYYt5V67LVIpvIaqk8bbVIXuiEgFh2G/faaXEe4\nfpHEe4fpF/f+kTWXexuvpWWAWOLeFSuu1iabwFquPGq1iV7I5BAY9jt23mbaNnAlTiZIZbloKnHy\nOQvdQyUEcNxaVIIT2kEHaR/Ee4fpHEe4fpF/f+iay72N9NIzWKhcTIY2NaqAUZBLtRv1lUBPWZNj\nuB7NjvtMPuzK48/LXcOh5nn0b+Pl99vMcuO3R3/8T2fvmnxHuH6RxHuHv9kX9/6TNG++kZqlg2Jk\nKa2qVwaMgFGv26KsCnqs+42B9u423n1dGziQoxMks1zUBRj5HI3opZ6AOG5tCgkp7QATtI/iPcP0\njiPcP0iT7/0iay72N1NKyyAwxbyr12WqRTeQ1VJ422qQvdEJALDsN++0PpWWAWOLeFSuu1iabwFq\nuPGq1iV7I5BAY9jt23mlxHuH6RxHuH6Sb+/9FxINo2cCVOJkhluWgqcfI5C91DJQRw3FpUghPaQQ\ndp8TSM1ioXEyGNjWqgFGQS7Ub9ZVAT1mTY7gezY77TQ4j3D9I4j3D9JEn3/omsu9jc+7Mrjz8tdw\n6HmefRv4+X328xy47dHf/wAT2fvmb6RmqWDYmQprapXBoyAUa/boqwKeqz7jYH27jbeaHEe4e/2R\nxHuH6Rf3/pM0SC6NnEhRiZJZrmoCjHyORvRSz0AcNzaFBJT2gAnaYaTpmRlXpj41L3ZFzcUqRd3L\nfnuD+yB33J2A2O+00uI9w/SfSAex2290i/v/AEi4nMvwfrNVllT6bmB6LXrsC42TaoetijBbKlZb\nF5KdmQkH2gkEGeXovqv+WZ3/AEeb9OQ/THuH6LHBfcP0WJR5rb+y04Mnv/RMei+q/wCWZ3/R5v04\n8Z1vXkhHVksq0/S1dHVldHTScAMjq3dWBBBB7jaQ/BfcP0WSviofj1j/APbdJH/+Rp8hTtqeT+tS\nbrDlmvsvOX9n2DWc1rDn11aQLjZzXFR88V4PnQ+KzV8akAHAkiztkY7f4umMc37O8Ktd1yb7mx6e\nrdXUlx84DpORqYXEvfDFVTfgBR62QTXYXAHE1yiaxrOXlWtbkXM7uGB9ZlQK/d0RF9VEJ7lQNiST\nNYZl/wCF+LZ/ZTvj/iW/gEsHJq7/AIR5AN6u3cAznVDSyvj777uN3TUU7oTo3iHwlhCvqJRkJXiY\ni3th1pXXn3FdJ8Db12WGtxW/W1m65j02A4XbAct12KvAmFZQiJTlUOKi17WV1WZl1mPqniah6K6Q\neOPmtVplVXTR3HOykdyC1nM0zskOtgutFiMWW0WWixWZVRmVw3JWKKqkg9woHsEwGTcFVBa4RG5K\nnN+CtuG5Ku+ytuAdx+YlVVqWylbJdYo7TdjvYv8Ag+B8FxhM7ZtP31fgY+NQ/lerhvnfeCK+STUP\nM1h8JbAFWotVlIfVI9bnKHcA+8A/rNizMvZzY1tjWM4sZ2sdnNgGwsLk7mwD/F7Z4zoo4IoZzdo5\n6SOGKUlIRETYyEREAREQBERAEREAn/s+8NnVNQpwhaaeql9jOqda0pj0WZDpTQGU3XMtZATcdzv7\nAZYK9G0yjUNKfDyMxLxrmFRkaVqmKmJqFS9amwZY6LsjUMd02PE8h/iHcVjwbl4NWZXZnrecYLYD\nZiWNVl41rIRTl0EMoayuzi3Fjsdj7fZL/rPjvE8vp2LbqWXrzYWtYeoDUcrE8vZiY1DDq0Um26y/\nItfbl67bbBQO/eefWHS+JKGcpe3Od8npddpPA7qBUdicUpz/AI5TWp98ZeE9Jy9U8Rpi5+Q+p4V2\nsao6ti0pp9i032X3YNdnWN3mEDsnUZApaptu2xNdr8L6Ri4uDZrGdlY9+s4ozMajDxachcTEssau\nnMy2ttQ2LZwZhXSCQtZ37kCemmeK8OvWdezm6nQ1enX0xdq9331J7zRzXf1B6439u375nfqWh6ni\n6b955eVp+VounV6bauPiJmV5+HjPY2P0HNyeVygljqTaGUkoR7CswhVNAlC25XYK/DDDORrF4Ubb\nUp343LH1yNvG+zbGot14apntRT4XfTTbZi0rkNmU6j1TSaFZwFsdehtyOw6zEnZe/p4t0zQ08Oae\n+Ccq2+/VtQTHtfExKnySj4qFchq72dUWliVChiWcghN954+JPH2LmVeKCanofX7NDXTqNua10aS5\nTa6zf1XNKoe24LMw32G8jKPEWN9x4mOt9+Lqeh6pbm4PCoPVeMhsU8uuHBxrazUzjdSDxUfmSt7N\nO3DFHPFXcv038sLTc/6IcVCpwwywd/P9V3PJXEh9g+lY7eIMKvUMe3i9Wa+PW+OjV2XU4WTaDYuQ\nVAVFVrAyhiLEp7AEsvhoPgvBsw21G99RbTrsx8TTcfEwqb9SyRUgazIuXrdDGqQsqftsSxIG3tOX\ngz7RcsazpubrOXk5tGnedT1m61lKZuHdiO9asQD3etiN/ZV79hJHQfFmFXgfdC65qOmV6dm3X4Gs\nYdWZUmVjZCobaMzBx8pHVxavJX5vsGYer/iin8dRtrKGcr1c4pyuxw5Tv0JovBcKTzcpyTwUp34Y\n8zyx/swq+8bMa7PajC+4LNdo1BsWxbBipx5LkYZblVan4oKBifw12237a1HhTQHxMvUl1HNXT8HO\nxcYVtiY3nsg345teupOsKxYH/wATMBwSw9zxDbNnjjBGXqDC7UMmm7wpnaPj5WffkZeTk5uTWB5l\n0uucYdD2b+ojEAKDtuSBWsXW8ddBycA8vM5Gs42anq/h9GrFvpbd9+z8nHbb2SYfHavb/wCeS5u9\n4YyKxOhWCXPPlhzwLWfs/wBCXL07Fs1bL5eJaMS/RymFQfL1ZuyYx1LleN2a/qV8KOX91uSoIA5t\nqGK9N11Nm3UxrrKbOJ3XnU7VvxJHdd1PeXXL8WYbal4WyR1Ol4fwNDx83dPW6mnZdl9/SXl+IvBh\nse2590qPiHLS7MzLq9+nk5mTcnIbNwtuexOQ/I7MO06Kr4qf5t4c5YzeSyMafw2vwSx5ZS11NGIi\ndhyCIm9WOgquy/2ixQ9SWJj2VLj21XL1XVmLC4lkdQyjYBW77oZDciUhZjijkL15ZHro2K62qtIe\npGS6x67FPVBsO1R9hT1vYUOrlZFljmy2xrLG2DPYzOxCqFUFm77BQAB+QAH5TyA2n2Eg2IiJJAiI\ngCIkzk6dp9XTW7LyRY+Ni3steDj2Vr5rGoylRbH1BGfZblBJVe4PaVcSRZQt4ENElOhpf+15n8Ox\nP6pHQ0v/AGvM/h2J/VJFta/JPhvuRFxJV9PxHqyLMfJvsbEqS50uxKMdWrbJxsY8LK8y08w+TWdi\noGwbvuADFSYYk8CIoWsRETc0jS8jKc149fNkTm+71VIic66gz23MqoDZbWg3I3a1ANyQDLaV7ISb\nuRpxJ/SPB+o324yGlqVy82jD6lg26LX5gwBbbSD1UoXJJqNhXbmhXfftNXH8NajYmO6YtjDN6fQA\n4Fj1q3uqaxA3KhHqrstD2BQUqdgSASKeNR5ov4UcpyZFRLI/gnUVo69laqp8xwrR67rn6FODcHVK\nS3KmyvNpZbFLAgg9g1Zfy9DNX6hq8owZUZ2JtxRWgS5cewNcbOnXYtzrWUZgQ7qCAdgaqmg6luT4\nMfSyAiZ5NL1u6WK1dlLtXZW6srpYjFXR1PdWDAgg+wgzCbGYiIggREQBERAElPFP9/X/AMN0j/Sd\nPkXJTxT/AH9f/DdI/wBJ0+Uf6l6P6Lr9D9V9kXERLlRERAEREECIiAIiIAiIgCIiAIiIAiW37HtE\nxNQ1vCxM1WfFyRmG5Vd0fanT8zIRlZCCCLKkbb2HjsdwSJY9PXwzfo+XqY0S2p9JzMbHXEXU856s\n1M1HCHKtYcq2Tpu58v0tzxHqg9uWlrSgisybwwl/02li1i0dNHV3HDamljjPkk3yeZy+J17B8Aab\nmarhnHx7KcLI8Kr4iu0xcocjYj2UHBpzssjpVvcKvxLW7BrO69iuj478GVJo+XnHTqtEytNuoCY9\nOrY+qVZ+PkWGpuKeZuspvqc1kncAq7dt9ytF/wDoUbiUPNyy9MJzxymvY0dSjsuLKef8S3OXxO45\n3hnwr9+6non3XfVXhY2TkLqdeoZbZSNRjV5jV149u9PTCM1Y6gcnbcnvstW+4dM1PT8DIwMH7pvy\nvE+LoToMrLzqnrzcc2pkN5k79Stl22TiCCfZ2AhV+G5yaVzm5YPDBvH/AEOpxKaTTd91/LHlyObx\nOr6do3h7P1a/QcbTrcW3nmYuDrDZuXde+Xg15Di3KxX2o6Npx2BWtEI5jYj2iKyKdBwtD0fKu0ls\n/O1vFz3ex8/PxqqGxc26iuwVUH8RmVkUqSoAoH5sWlvOKaVlzcrruabnjLk+c9CnlXjaUr77+TSl\nhPmtDnsTqGb4M02nVdRyDW1mg4mjjW8NXsvTzNGdUqadiddX51k5dpr3JJIxW3333nLh7O82oaeG\nl/Tp88vVc/UypKGKj/Vr8c/R8j7ERNjIRE+GAbuBSFU32KTXSwFW9fOq7IGzrjvuwHDjyY7b+qu3\nYspmpY5ZmY7buxY7KqjdiSdlQBVG59gAAmzqhQP004MmLyqFlZcpewdueQC3t59tiAPUSobbjc6s\nrDmWeQiIlioiIgCIm3pmnvcXIK11UKHvyLCwppQnYFyoLFiewRQxJ7AGQ3LElJu5GpJfxEU81R1A\nxrGBo3UCFQ5T7q07mEJ7Biu+xP57TZ0nWNPxLlevTq89a0tUnUubJY1lFlSt5THsFdSKz8+LNad0\nUhkOxHj4qvTztdnSQJ5PSLPLg29Hh916e3R3dy/T29XuxO3tJPeYuJuPDkzZQpQ4819nTPtE0Lwx\nqtVOR4YvwsKvTaSmq15P3pjMFfoDGuZWx3XgrdVGvZgOT1Ak+rI7wd9g+u5xLM2NjYzUdXGzevRl\nU5e52UVDGdmCezdmC7AjYN7J1H0hyM2+vUKfDFumUabp5p87djYuPqT22gNVi6e+XX5evF4LYrW3\nLt073AKEhLNGvw7ia7qr5lPii3BtOllH0/HzcJ8vDCMqugyMHKtpsw2YCxl5uxZ92O57eHBW6WCG\nzOSXN/m1pd+79D2IqrRxO1KbfJfinrf9HCNF0u/lq2IFDZCYq45VWQqba9a0mttrCQoQFSeZIAA3\nJAmm2ThU+rTSmYy/tZV/mBUzD88bGR6+NW+/e/mWAUlatygm/BOE4s13HxiMmwaLm0Y7VKzrkBc7\nT0ZqlG5YPQLNgNyea7d5teD9DooVrtR0LWM7Kpt5Y2EuK9Om3KFXj5u01m5hz5Eoi7EKoPYsJ67p\nJTnpcrm7keWqO1KWt7vSvZo5mjWriY2XqGnDBw9RYJi6jTyrd2KO6WnAe49bF4qW5V117gKQx/Zf\nS8P6ndpWZcWR2cVvj2CrIejdTZVYHWxVZMilxWvq212KyWbgb8HXZ8TYXiDUMu7LzMDNsyMlt2Iw\ns5URR2SmtOn+HUigKF9wHtO5mPiDUcnHspoaqhbKMHBW9b9P0u21X8pQVR2ycZrOS1GtSGPYqwkw\nziVlyc8VfL7feAilC7Sulg+f0S9f2nZQOO3l0Q4eVj3V0U3ZOPgrTi6kup146YNZ4rs6LUGJYCut\nPV5AWzWT7Q8joUUvSbFqxa8PJrbKyxj34len3aWa0xkYLj2PjXEmw89rEUqEBZWr/wB/5Pw8P+F6\nD/KR9/5Pw8P+F6D/ACkeUg6Vux5iLq/b+Sc0nxv5RQuHg10Gi22zFY222Gvr06XW5sBUC+xn0vHt\nLDgOT2gAAqF09V8Uq+JkYdGGmNjZZvsZerfe635OTp2Ra6u4H4e2nY1YQjcKhJLklpH/AH/k/Dw/\n4XoP8pH3/k/Dw/4XoP8AKSyq8Kc5LdkOmbUrT+Dw1/UDlZmXlMoRs/LyMpkUsVRsi57mRSe5UFyN\nz7ppSU+/8n4eH/C9B/lI+/8AJ+Hh/wAL0H+UmqUUKkktzF2W5tv4IuJKff8Ak/Dw/wCF6D/KR9/5\nPw8P+F6D/KSZxZIiUGbIuJKff+T8PD/heg/ykff+T8PD/heg/wApE4skJQZsi4kp9/5Pw8P+F6D/\nACkff2T8PD/heg/ykTiyEoM3sbfgzwplak96443GJQ1zqvB8i4qllnl8TGZ1OXklKrHFakerTYd9\n9g2/q/h9srCOq4xs8rQmJhccmqjGtyWw8Gmi2zDRciw5S1rjNYwAHEN7W2crL+F9WxNQya7MoJpz\naTa2fi4Wj464d2e610KcbG6R/DyuePUwfcnjZfttsu0t9oniAajT956hVdpWoYtWbgV6TZXkDFza\ns+nLRcrBFoDUW1G8G0sNm41kbEhDwxU1L4iUv601nderlzO6Gio/Dbn3npK+7mckiInonnCIiAIi\nIAiIgCIiAIiIAiIgCIiATngHxEdM1HHzlpGQcUZCikv0g3mMXIxSeYVuPEXlvYd+O3bfeY6br5q0\nrN07pBhqGXhZJyOpsa/JraAgq4nny6vt5Dbj7Dv2hYmToYW5tZfDmtmaKkiSknn8qT3RdMX7Qra7\n9PsXErerB0IaFl4lju9WoYTNkG5XKqpo5i4bBSdmqU7sN1kfr+vabZhPiafo9enrk2rbkZV2S2o5\nZFYbp0U3WUV+VqBYk8Bu3YE7bg1uJRVWjTml8v5U5P3LusRyk3+3xl7F4yPtCd9bztX8oobUsfJo\nOL1yRUMnDXD5C7pbvxA5bcRv7O3tkPpnim+jTq8KhOFuPrmPrNGaG3au/GxrMetBSUKnZmWzkT/g\n22PtlfiFVqNKUsvjDvch08bc55/OJ0E/aLi15GTn4ei14usZldwbUPOZF2PRflI6ZWVi4D1/hWuL\nX25WuBzPYjcGsa54g8xgaViGkVrolGXULupyN4yst8osU4jpceXHbdt9t+3skLEQ1eCFzS+W+TXN\n5N3YXkxU8cSk38LNP6V51H7Qc/KwNA0zRMrprnta92eK7abrasDGvybNPw8i6lnRx1cvItChvVVa\n/ZOXT4Bt7O3+6fZagofCUs22/VlaWltueSSXohERNjITd0ounUyF6inFC9GxOlsmTY21XJn/AGfV\nW1wVBO9I2493XSm0wAxk7LytyLNyLGNgWmuoKGpHZEJvfZzuSVYDYA8qxZFkagn2IliBERBAiIgH\ntg4tl1tdVQ5WXOERd1Ubn82Y9kUe0sewAJOwE29Yy02XHx2LYuMxKvxZDlXEcXy7FPcb91VT+zWF\nHtNjP6Yv4GI9vstz+pi4/vTHUDzdw/Mc+SUBhuCrZgPcSKlFe55F8FIS06ro2pPfjX42DlWqMDR3\npuTEyLqmarTNP7grWVcB0II7jdSDKtMSg9w/SRFC25oQxJKTOofa34h8ReIHwmy9FvoGm1Wqi04e\npHlZkdE3WE2ISFPQr2X8tj3beUY+FtVPt0zNP+/DzfpyI4L7h+ixwX3D9FlKKhVHCoYJJLR/yaUl\nLbitRTb9UWPG0fNx8XUnycS/HSzBqrRr6L6Uew6ppVgrVrVAZ+FbtxHfZGP5GVzgvuH6LAUD2AD/\nAJTKaQpqczOKJOUjHgvuH6LPoG0+xLlRERBAiIgCIiAIiIAiIgCIiCT4w3BHvlh+0POGTn9YUU45\nu0/SmNWMltdKk6XgkcEsscqACFA39iDfc7kxeBpd9qGxQqUo/Bsi6yqikPsGKB7WHUsCkN005Nt3\n2kp4hwDa3VxrasuqjAwUuNLt1K/KYGJj3O2PcqXdJXqY9UIV22JImLlbXo/mRqk7DWq+JlfiImxk\nIiIIEREAREQBERAEREAREQBERAEREAREQBERAEREAREQBERAE3tWpdEww4K8sIOoNNtJKWZOVYjc\nnA8yCHBFq9tiAN+O50ZJeIs3LufHGW3JsXTsDHxwCpFeGuLS+Og2J2/DsDEfkzt7PYKuc0XUpMjY\niJYoIiIAnpi0WWWV11DnZfYlVa7qOVljBEXcnYbsQO/vnnJXQz068nL9hx6xj45//VZi2Vq3buCm\nOmVYGBGz10+/aVickWhU2eXiDIre7hSQ2PiIuNjMBsHqpJ3uA23Xq2tZcQfYchh+Uj4iSlJSIbm5\niIiSQIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCe+n4j33U017dTJurpTf2c7XWtd/3bsJ4TOi163R6y\nVsqdXRh7VdCGRh+8EA/8pDwuJUuZt69nJbaenuuNRvVhVHt08ZWPT3XfbqPubGI9r2Ofzli1Xwn9\n20Lbm6muFq4SrJxdHqqybcyoOUas5WTWyrp13A9QI3JtuO+xOw+6brV2n2X6jp+JiWplMllGTfje\nbs0XI5WOKqw7cMe5XfZbLUYMtVZABDqtboqys6+1t2yL7We/JyHfcDk275GRfYeNabnc2OQO/tnO\nrTd34wrHCb0015vTn0OSxviey11PXX0QtRciLUM/GGQ1KKq112LfkYtvBR2WtrMZ7AgACi0KBsAT\nGzf1y+tnrrpPOnDoXHrtKspt2ey623i3cI191rAEAhDWCNwZoTeDAxjxEREsUEREAREQBERAEREA\nREQBERAEREAREQBERAEREAREQBERAE2c3gVxyvSBOP64r/aDJdfXvaD7LSqI247EMh9u81pu4u9l\nL0jkzUGzJoRUqK7CoHKZ27ONqcet/aQBQ/bc7yrzLI0oiJYgREQQfCZK67+EtGJ7GxFazJ94zMkV\nNah77q1ddePSV7bPj2+8zDQq0DPkWKrVYAWwo4UpdexIxscq3axWdSzJ2JqoyCO4mhbY7szuzO9j\nF3dizM7sSzMzHuzEkkn98pjF6F8F6mMREuUEREAREQBERAEREAREQBERBIiIggREQSImzjadlWBD\nVRdYLnNdRrqtcWWKvNq0KqebhfWKjc7d5lpOm5GVemPjUvfkXNxSpF3ct+e4P7IH5k7AbHfaRaRN\nlmpEm8vwfrNVllT6bmB6LXrsC42TaoetijBbKlZbF5KdmQkH2gkEGeXovqv+WZ3/AEeb9OV8WHNF\nvCjyZHYmVdU3Uotsos4lRZVY9bgN7RyQg8T7pY/tI1ZsjKArOTXitiabemJkZuRnLXZbp2Lazh7V\nXdybW3YrvuznsDxEb6L6r/lmd/0eb9OPGdViZIR0ZLKtP0tGR1ZXR10nAVkdW7qwYEEHuCJn+MUa\nayf0X/KGBp5r7ImfN/8A7/8Av/eP1nSM7TNIDZnl6dLsaprxo1T6taqZuMMrBVMjPZtQRcXJ8s9r\nitraCWe8FBwQSRv+4GWvE81S2FVkAVK1ybsKMn7RbcVHZr6W6RsydOBd7KfUzKSWqDBhjFXUlNJ9\n99yNYao25OJHJ4nRdX07Q1Sw4teDbYEbzi5OoPQmMo0+qxH08Yufb5l3yDdvXXZlkPXWvqhiJK+J\nMPw1a+oXtZjA3ZeqWs2Nk0u1K8VfTjiKdTBdXDI5RMbI9e29D0OIFU+dV1zI8pFfejkm8+y6faKM\nRKManFXCrSrVdZepMLM89yxHr0avFyLmOTc1Vli0MNmKb9FjxU8t6XOiipPEhtGFJBYisiIiaGYi\nIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCfa3KkMvYj2dtx+8EHsQRuCD7QSJ8iCTazq0O9tQ2\nqsbdkRb+GK9j3lMVrLN+R4VMwPIkqNz3DhdWe2HlGtt9ldDt1abOfSuUd+FgRlJX3EEEHYgqQCLJ\n4Z8F2Zr1DzFWnJbjPaL9UbytNj103XMcezY9ejjWrGzYcRZ7H23fKKNQKcWBeGBxuUKvKrPXCxbL\nbErqUvZYdlUFRvsCSSzEKiBQWLMQAFJJABMtmg+A/N2NXXq2mF0d16NeS7ZFxRipGLVbXWuSTtyG\n1gBXvvtPDVtGyaS2LRXXUSoW82ZuljOygeLKrY/XDUVN6rChQSdxubdkKx48Lck79S3gxJTau0IX\nVcivjXj0HlTj7s1vrAZOS2wsyAGAYV7KqKpAIRNyFLOJHzY1DAyccquRRbjs4JUXV20lgDsSosUc\ngD27TXmkMpXGcU53iIiWKiIkzk6dp9XTW7LyRY+Ni3steDj2Vr5rGoylRbH1BGfZblBJVe4PaVcS\nRZQt4ENElOhpf+15n8OxP6pHQ0v/AGvM/h2J/VJFta/JPhvuRFxJV9PxHqyLMfJvsbEqS50uxKMd\nWrbJxsY8LK8y08w+TWdioGwbvuADFSYYk8CIoWsREST0fQsrKpzLqlHR0uhbslyHIUWOK6614KxN\njtvsDsPUYkgCTFEkpsKFtyRGRN27R81GtR8TIRsZBZkK+PkK1NZVnD2qyb1JxVm5NsNlJ/KZnQtQ\n6vR8jldbgbOh5bK6vTUKWs6fDlwAZSW22HIe+V8SHMWIsiPiT2L4P1Wymq8YjrjZAxWTJcbVcMzJ\nsxKLCRuQjW1MPZ7Ap9jJyjBpWXxobyt/HMYLinoX7ZDMFKrSeO1zEMvZd/2h75CpYXg0S4Ilimak\nT1y8a6p2S6t6bE7NXYj12L//ACRwGH/OeunUoxd7AWpx0Flyq61u4LpUiIzA9y9ib7AkKHIHaXnd\nMrK+R8xcdCostbjUXav8PoWXl1r57ChrVYJuUU2HsOXbkQVn3zxUbVVpTutIYqOdpek8hal9vJ6H\nL9yKigOyjbtPHKyLLG52Nycqi77KAFrRa60VVGyoqKqhR2AUAeyeciWZM8jLIsd3Z7Gax7CWd7Cz\nu7E7lnZtyxJ/MzAgHse4n2JYgx6Y9w/RY4L7h+izKJAmY8F9w/RZK+Kztch/IabpP/bSNPkZJPxY\nN7kHv03SR+ukafKv9S9H9F1+h+q+yczfAV9fmmbOwmp0u22nUslG1Bq8K6qymoV2KcUWWs9tyIOk\nrgsGBKgbz7b9nmatXVN+Ka/MY9IsV81qQuU+ItF9uSuN0sOt0zKrQMhq2NfI7bgKfHXPHmo5GTfc\nprpqybsmw4gxtNsoZck18kyUbHCZ7AVVAWXq5BqUgg95qjxpqoZn8yOo7b9fy2n9dVFtN4pS/o9S\nrGD01kUowUBNtttweVQ08leu/Y3boZ4Pv3JenwA9dt9eZctIoqazrldQx1r30vxFm1GzGysAXWVm\nzRxu6D9g+qLSwavLTfAJYJ1LFsGR0ziXU3PXVkLdkeGkrJW7D50J09abdyGIdNuA4nqwdfi/UlK8\nLakWtOmtCYel14y19PU6jWMRMcUhGTVdQBXhsTluSCQpAeL9UDBhkhSro6hKMJEQ1nTTWqVrUErR\nfurBARQABjAAAFw0Kjp7WK79izpKGVyffuTtv2e2P0UxbRdZdTiW298puh18bXcu2vo1YZfJK1aS\n+zVbksNgthcdPVz/ALPc+mrKteyrjhILHVa9UNvTONXlB3r8pywlK2cAcro72V2L7RIpfFupAIvX\nBWoKoDY+AwdVr1CoV28qT5ivpalmVlLOQKZBBBHED43ijPK2L1KQLUsr9XD0tDVXdi14NteMyY4O\nEj4yJWVoKAqO+/eWVHWE8V37FXFQNYOfeuZCxETsOUREQQIiIAiIgCIiAIiIAiIgCIiAIiIAiIgC\nIiAJ8Jn2SugjpLbmMNzhmtMUEbg5tvNqXII2Zakqst2O4LVUgghjIickWhU2eh2wtvUV9Q9rCxVe\nvA37hDWw42ZvvDAivfbYvv0IjItd3ayxmsssbk9jszu7HuWd2JZm/eZ8ZiSSSSWJJJO5JPckk9yS\nfznyVUMr+YcU7uR8MttGrjMxmrzQ1zYil2cDnldAnezLx2YjlkVb8nqY8bK+THg6G81Oe+nZb0W1\n218S9LhgrjdHA/arsUH16mUlSv5qzD85EcFpa8i0Edn0eJvPbm4DtVXeRXaFsHTZnw8yp1/Du6Ng\n4X1sv+GxNx6wIUhlHzzmJb/8RQaHP/mcMIu527s+DYwqf2AAUtjgcmOzdhJLJxEsD4dYZgqNn6M7\ncmsbHtTr24BYDZ34BzsNwMjFuUDd2MrMrDKL1JinD6Eo2hXuC2KVzkHcnF5vagG+5sxGUX1KNu7l\nOO5GxO8iQw94/WZDsQR2KkEEdiCPYQfyMlfSPUT2fLtuX/5MlvN1/v8Aw8oOn/aX/Ja/BX8Xp8/w\nRUl/ERTzVHUDGsYGjdQIVDlPurTuYQnsGK77E/ntPTS/E11Nq2NiafkhQ46F+maW1Lc0dAWFNKP6\npYONmHrIvtG4PzxXkb5tdr1owOHpFrUhVrpYfdmnu1YSvYV1nuuy7bA9tpRuK3euTNEobFz5r7Om\nfaJoXhjVaqcjwxfhYVem0lNVryfvTGYK/QGNcytjuvBW6qNezAcnqBJ9WR3g77B9dziWZsbGxmo6\nuNm9ejKpy9zsoqGM7ME9m7MF2BGwb2TqPpDkZt9eoU+GLdMo03TzT527GxcfUnttAarF098uvy9e\nLwWxWtuXbp3uAUJCWaNfh3E13VXzKfFFuDadLKPp+Pm4T5eGEZVdBkYOVbTZhswFjLzdiz7sdz28\nKCt0sENmckub/NrS7936HsRVWjidqU2+S/FPW/6OEaLpd/LVsQKGyExVxyqshU2161pNbbWEhQgK\nk8yQABuSBNNsnCp9WmlMxl/ayr/MCpmH542Mj18at9+9/MsApK1blBN+CcJxZruPjEZNg0XNox2q\nVnXIC52nozVKNywegWbAbk8127za8H6HRQrXajoWsZ2VTbyxsJcV6dNuUKvHzdprNzDnyJRF2IVQ\nexYT13SSnPS5XN3I8tUc5S1vd6V7NHM0e1cTGy9Q04YOHqDBMXUaeVbuxR3S04D3HrYvFS3Kuuvc\nBSGP7L/PCmuvo+RkLZj9a1MjAcBXVUPks2nL5LbxPquiAraoP7SMAR7cPE2F4g1DLuy8zAzbMjJb\ndiMLOVEUdkprTp/h1IoChfcB7TuZj4g1HJx7KaGqoWyjBwVvW/T9LttV/KUFUdsnGazktRrUhj2K\nsIScasuTnilOX2/29EG7DtK6WD5ueyJbA8b00LgY2Mtq06fqGl5HmMp92urw83VM2yvKXFRm8v1M\n9dkQOfwbDsSwVZrI8ZadhUY+DTZbmVJX1LsoNg5ttdy6jkZtdavmYors5CwOWVd1s6Tbkh65z37/\nAMn4eH/C9B/lI+/8n4eH/C9B/lJDqibw1x55loa04Vc/hXLJFuwvtAxg9NtuNcbq8nEyX4vi8GfE\n1zP1VVAWtAA9efYhIVQGqQgbEhdfw74spLmi7eqrJxtMxTe9zotCYPhnVNCtZHSqw1PY+cLFPHbt\n6xUcmFZ+/wDJ+Hh/wvQf5SPv/J+Hh/wvQf5SHU4XO7HVkKsu6/DREn4xNOXqVVWFb1q6sLT8Nb7L\nPU/seDj1XW2X2IirTWarCbSFXjUx9mzGx43hCwHL0I1Bc9Lkz7dSuw8RMbHw8bGv6lh1N8gvVpbl\nq3FwRf2NuJLFBXtG8T8l8tkUafSmR5mt9RTTsOrKpXLw3xFBsxK1Pl0ZxYQiFtuoAW7JOk5WfVbh\nHw6y5lWjY9CJj+KLFyBVdkY7XXrdk7HpHQ2azZKwzcQtTbk/sYU8VJAlBDgvfDnrLpxZtRQwRtxc\n37Y8tJ54I4lmY1lVj12qUsqYq6kqdiPcykqykbEMpIIIIJBBnlJ3VfEBD8aBjWU49OPQl1mnaXbZ\nauNj045tLZOM1nFzUXCuSQrKO220m87RtepWxrMbTR5fzYsRafBVlobBqGRlVimtS72VUkWlFBIV\ngdu87fFalakp53HJ4abdmblleUeJdX0vW1LhsbTVFAu8w7U+CxVjNjWY1V9eRaU449yPl44NTkMO\nqO3Y7ZZWka5WzrbjabX0Tet7vT4LFOPZi241N1WRfw6ePctmXjL03YHfITYH8o8dZrcnwHk9v7KR\nEuGfg63TVbbdiYNa462vYhwvCRyFrozDp9tvlFqNzVrkg1lwhHYnfb1pF6lqWdRddRdThpdi22U3\nJ92eHn4W1sUdOSYpViGBHYkdpaGltYS3KxUahxn37kfp+EHS222zo4+PxD2hOq7WWBzXTVVyXqWM\nK7G9ZlAFbEkHiGtH2k+Ha8fOvx6co33adp+mm6p6OgbKk0nT2NtHG6wWcazzZG4kAMRzAYje0vxl\nbfVelOj6LZkC1LqMJtKw3rsTpdO0YtLb/wBo3rqYqpJYM2w9UCW37Y9ftp1DU1v0zTaqbdNpx6Mt\ntNxk1G+7K0XGpK4+SdnIqa9t3HZUoCbglFbjjpqXxkpcndP/AMdDrhoaPwW581ffrqcQiInpHnCI\niAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAJKat6mNgVezem7Lce0i3JvaoE+5TjY\nmIwH/qJ/ORck/E39/X+7T9L2/wCel4J/+pMo/wBS939fZdYP2X39EZERLlBERAJiq5zh1W1syX6P\nlKEtX9tachmvpZSP2VqyqrTyP+LOQb+wTx8RUoLEuqQJRqFYyK0UbLUxZkvx1G3qrXelige3pio/\nnM/Co6l7Yx221LHtxFB7b3vxtwxy/wAIOZTi7n2bct+28+aaetiX0f48fln43vIVFTMrHs7mhK7S\nTvsNPIA9YzHB94P+zbFd4r+iLiImxiJadV0bUnvxr8bByrVGBo703JiZF1TNVpmn9wVrKuA6EEdx\nupBlWmJQe4fpKRQtuaLwxJKTOofa34h8ReIHwmy9FvoGm1Wqi04epHlZkdE3WE2ISFPQr2X8tj3b\neUY+FtVPt0zNP+/DzfpyI4L7h+ixwX3D9FmdFQqjhUMEklo/5NKSltxWopt+qLHjaPm4+LqT5OJf\njpZg1Vo19F9KPYdU0qwVq1qgM/Ct24jvsjH8jK5wX3D9FgKB7AB/ymU0hTU5mcUScpGPBfcP0WfQ\nNp9iXKiIiCBERAEx4j27D9JlEEnxhuCPeNpYsrxhmvbZaVqV7snWMklFtHGzW8JMHJ4b2EhVqrUr\n7SG3JLjtK9EpHRQx/q7mTDSOHDuRavE3j3Uc+u+vI4bZldyXkWalaSb8rAy2dBlZVi4/4mDWBXUF\nUCy0BRuCvnb40ynGWl9GPfRqWbnZ2VjWLmpU92fdp177NRkpYiJZp1JXi4IDWglwdhWYmaq1GlJI\n1dPG3NsuWleOrvvKnUcsB7MLFzq6a6qkNV75V2oZC03rZaBXji3ULBuoYhKagASOYpzuWJZyXdyW\nZ2LFmZjuzMT3LEkkn98+RLw0UMLml32ykVJFEpPvuR8Ybgj3ywfaDmi/O6wx6Mbq6fpTGrGS2uhS\ndLwmAVHscqACF239ijfc7kwEk/Fh/GT/AIbpP+kafDh/NPR/RKf4P1X2RkTsmR4c0p8hEGGqY6vi\nYorBRbuPnPs/V7Gya0VnsevWMpSzAkdVyCNwFh6vs6wnOIOvfU+dk4RNYryLhRTm6yNLOMzDEWpL\n6lfc2vaN7KbE4KdmPNDXoJTc+/Q3iqcaclI5nE6LpXgPT8qjDspyrqPvToWUNaluQlFV2rppXTva\njEWouCWt6hur2Zq0K9xaYHxvp2PRRpTUYt+KcnFy3tTK4+ZZq8/KpU2staA+rWANkXbbbvtyOkFa\nhjisrOW0/wCDKKrxQw2n67lYiInSYCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAkl4lP4\n9f8Aw/Sv+2l4EjZI+I/75P8A2Gl/6Xgyv/S9H9Fv+fdfZHRESxAiIggypuetlsrYpZUyvU6nYo6E\nMjA/kQwB/wCUmNVv8rqLXY6qE6teZj0kMK/L5daZdeNYoP8Ad9G9aivsILD2GQslNVPUxcC380TI\nwm39rNjWjIVt/wA1FOfTWAe46Hu2lIlevdF4Xc/ZnhreGlN7pWWalgluM7ftPjXILaWfYAdTpuoI\nHsZXH5TSkqg6+GVHe7S+TqPzfBus3sA2H/g5Fhf8yRm2nsEkVJgd0mIlfNCImJce8frLFTKJjzX3\nj9VjmvvH6rEwZRMQwPsI/WZQBETKtCzKqgszsFVVDMzMx2VVUd2YkgAD3wDGIiAImQQ8S2x4qyqW\n2biGcMUUt7AxCOQPz4N7jMYAiIggREQBERAEREEiSnio/j1/8N0n/SNPkXJPxYN7kHv0zSf9I0+U\neK9H9Fv+X6r7JDI8K60rBLKLAX6nPlfj8KunVTkP5qzq8MMClKbfxym6JWRuApi7QNc2YNVkE15Z\nfom7lbZlFqEOTTj9TnksWyaB16lYfj19+4k1rnj2hrs5MbBXyWrWctQWyzIW7L41BK2VgxGIyMWs\nHHkC7tuCu1a64+0KwVW1+SpKP0Fppa7PfGqrxaMLGxz0Gt/+JrowakGRW1bd2JLELx5VFTOX4o6W\nqJN/kzU0zwfqVobF5cVZupVQltWRi32DC1i9WWzHtak2E6TfTvuSG5AkcWE9cfwhqOSXbJtcWVjc\nGxmyhanHT2ranISxluV0zq2BQkFQCD3E98z7Q7LHZnw67FsRUt6t2RbfkKMPXMNvM5Y42Xsa9ZtU\nOTyC41A3J3YqPtGvrVK68WpaMZVrxqy9rGqlfI7I1ntscth8yx23a+zYKAqis6xPBfHuWXgc2/n2\nNDUPA+dUK9gtz5D1LTVUyM3G19TTe/1tsVlGm2uQ/YLyJI4uBoW+GNRRbHbHASlDYW62IVtrFAym\nfGYW7ZqDHItJo6gCMCdgQZPVfaNkK1bpjVrYtjmx1sylNlT/AH6jVqUYNS3T1zJUWIdwa6iO4O/z\nI+0O90yq2oNqZVBoVcjM1HLQKaDSHvqyHavIsQs1quq1lbGJHb1JdRU65Io4aDk2UqIidhyiIiCB\nERAEREAREQBERAEREAREQBERAEREAREQBJHxF/fJ/wCw0z/TMGR0kfEX98n/ALDTP9MwZXnv9Fv+\ndvsjoiJYqIiIAkpg+vhZlf549mLmrv3ARXbCtVf/AJWZs3GP7xR+4SLkp4V9bJWn/b6rsMA/smzJ\nqerH5/8ApXJal9/y6YP5SkeE/fYvBjL23NLTst6bUtTiWrJ9VxyrsVlKWVWLuOdbozoV/NXYfnNn\nWMFECXUbtiZJboljyelxsbMS1tv76vkO+w5KUYAA7COU7gH3zd0vUXp5jgttN4C5GNZyNVwXcoW4\nkMlilmIsQgjk2x2LAok5zQhalJmnLPrGtalXdj0Y2Zl1r936QlOPTkZSLys0zAPGuutwAzO5PYdy\nx988NJ0bBy7lSrUasAWJaSNS6iJW1dD2qvm8epq7UYpw5stRJYAIxIB8/FFB89VXXYrN5TR66r1Z\nkrZvuzT1S1XsCslZOzBmCkDuQJk4oYopPJ4o0UMUMM1msPcsX2gaT4s0VsZdSzcqs51T2UFNRyrV\nJr4C2tilvZ06le/5euNi3farnxTqo9up5v8A1mb9SfpDXPCuth6cTXNQ03VxbjPbhZORp2PlahiW\n1miu2nH0+qg3Z+K/Knm9bVsOIYsu3CyFxta8N6DqluKPDd2blHTeeZk41NlyWGwK++Pg5V9iVYZQ\n7NZVYRyUgdQDkfMoq+nDKyo4sfxuT3lL95npUlSc52rK1xW2JxTF1fNyMbUkycq/JSvBqsRL777l\nSwappVYsRbWIV+Fli8h32dh7CZG4mkX2ItnqU02FuF+RbRjpZwJVzV1mDZHEjY9INsdgdt5NeE/L\nXWazY9KpjLpt2X5VXfgK69U0y6rEFg2YIzdOrmNiA+42M1dD0HVtXyf7PRZkO7BLMjg1eJjIijbq\nXKvSw8euvjsvbZQoA/ZWemo1DawS55YI89wOKzzf9s0MjRr1RnQ1ZFda8nfHuovKJ+dllKnq01g7\nAu6KN2Ub9xvl4UtqTUMB8ghcevUMR8hm5cVpXIrawtx77BA3s7ya8U5Wk4j1UaOLLcnT7gbNfa5/\n7ZYEZbFx8LbpVYnNuxPIsq9yQTvpatpWKbBYuZi4qZdNOSmM6asxq69S2PWpow3TppabEHrE8UG/\neFHaV6d++1/fIh0dl3NXbb3F/wBJ07TasOvKzMbEGCKdK8ub8TITIfJuw8izMNmQ1K+artyUZwFs\ncCta+PDawCL0rN0GxMJ7hgV22HDbXlsxuCvhr5mvKrwKqauONmFFRj5cVsWekqQOrtS206ghVOq4\nhVORVSniAqpYgtxX7v2Xcgb7e3aY/dWN/mmH8mvf06YKrq+bfz3qb+O7koV8d6FqwtX0oVujV4hG\nNiac2Ohx12vy08L6312v2TfJcavbjAmwn1mUDYbyc8O6fg5VfmK8fENRpufVnfCfpc6fDmBdxxXr\noNWC1ef5yxlRqe71ftKOI5z91Y3+aYfya9/TpkunUAMo1XEC2bc1CeIAr8TuvIDT9m2Pcb+yIquu\nUT+e58/UhUzUppfHehEL7B/un2Sn3Vjf5ph/Jr39Oj7qxv8ANMP5Ne/p07La1+Tk8N6fBFxJT7qx\nv80w/k17+nR91Y3+aYfya9/TotrX5Fh6fBFxJT7qxv8ANMP5Ne/p0fdWN/mmH8mvf06La1+RYenw\nRcSU+6sb/NMP5Ne/p0fdeN/mmH//AF69/TotrX5HhvT4IomSvig/2iv/AIbpH+k6fLr4Q0vHwMqm\np1q1p9VufT1v0i+6zIwQ1Nb9fBudERc7e5GBYbBca8EgM5WW+0XRq8GhNOuts1izNp1DNTxBbZZZ\nTX9205jLh6b+LYK7BZQq2hif2lG37DLyOuLxFCl6enN+0vfkdKqz8NtvL+l7z9jkERE7jjEREEiI\niCBERAEREAREQBERAEREAREQBERAEREARPbCxLbrFroqe62zfhTUj22PxUu3GtAWbZVYnYdgpP5T\nafQs9fMcsHJHkAGzN8bKHlQUNgORun4AKAtu+3Yb+yQ40sWWULeCI+JtWaZlLy5Y1y8GvVuVN68W\nxePmVbdfVarkvIH9nkN9t5qwmmQ00IiJJAkj4i/vk/8AYaZ/pmDI6SPiP++T/wBhpf8ApeDK/wDX\ns/ot/wA7fZHRESxUREQBMqLnRlsrJWyp1sqYe1XQhkYfvDAH/lMYgkkPEtKJl5HSAWmyzr46r2Ax\nslRk442/w/g219vy7j8pHyT1j16MG72k0PiXN7GNuHZ6gI/MLiX4ag/+jb8jIyVgw75Fo8e+Ykv4\niRWyqVZxWr4GjK1hDMEVtK04M5Ve7AAk7DudpESZydR0+3ptdiZJsTGxaGNedj11t5XGoxVda309\n2TdaVJBZu5PeVinOaWf0TBKTTeWep1z7Rhofh/y64uRfr+XqeE1bZtuqr1cHCRalSihsWkvTjXm2\nz9ixDxpYAkFw0X4U/wDxAapht+JhY2TVXjtRQhs1A317uHUtnZt2Rdcm++6M23s2K7TmQu0of+Uz\nP4jhf0uOvpf+yZn8RxP6XOGCoQOCUacb5t/xO66647I65HanBEoVkp9v3JPw3qq89avvqFi5mm3d\neiplxx/atU03fpMUcVBHdXAKsPw1E8Kdaxkosx0TU0xb353YiavUuNa+yrzehdO4WNsqjcgnZR7p\nqvqGIlWRXj416Nl1JS73ZdGQq1rk42UeNdeHUeZfGrG5YjYt23IIiZ1qiTm2u0kc0VK1JJ9tslOv\npf8AsmZ/EcT+lzX1nMS60MlbVV100U1VNYtrKlFKVbtYtaBmYqzkhV72N2mnE1UCTn9szcbal/Ai\nIligiIgCIiAIiIAiIgCIiATfhPxRlad5kUMwrz8dsfJFbCq/gQwD0X8GOPcAzqHAPaxwQd5NeLNQ\ns06rK0XDuL4dz4eVfYMnGzaXe3Cpu3w7aaEWmtlyDWzpuXWtQSByVqhg4j3W1UpsHyba6ULHYB7X\nWtST+Q3YT31/LS7Kyba9+lbfYaARtxoDEUpt/hC1BF2/IKJzxUMLjnL19VKT9rzdUsSglP09L573\nGlE3m0fOD0ocTID5fLyyHHyA+RwOz9FOG92x9vHfaebablBLbDjXCrGt6ORYabxXTduF6Nr8eNVm\n5A4MQdyO01tLMzsvI1Ykxi+GM93es41lV1QJOPdXfTewGJnZu6VOnJt6sG7b3koBvv2ywfC+db1R\n0+k+MXF9N3Om2vpnGDc63Tcd8qvt7fb27Q6aBcyVRRvkyFiTGo+GNQoWt7aGUZL1LjAK7Nk9azKq\nralQN3BfEcbdieVZ22YGab6Tmjr74t48jt5vejIHluX7PX3X8DfY7c9vZCpIXgyHBEsUacREuUER\nEAREQBERAEREAREQBERAEREA3dD1Hy1rWcOfPDzsXblw289g5eDz5bH9nzPPj+fDbce0WzB+0IVU\nVIcQc8OtBi2htOYrYNK0/SXLNk4VjohXArsC0PUdndSW9VkrHho1i63qMK1+7dX4sX6W933TqHQr\n5AjkXu6acP8AEXC7HfY9EwMnQ8a/U7cUY2McZtfwMW1MzNstsxkGAmJmVE5ZZrnV8oBqSoYMwAO2\n84az4du+FtyOyrqOU1Elf7lf137QrMrBsxWxVRrcbHpOSLN2FpvGVqWTwFYHPLuqx2I37DH29ffc\nUmdF13T9Ex6s60Y+E9uOM4aZi1ajkZNeXjrnaJRh5Vpx80v5g0ZOe/TVqwRQx4IFO+Os4/h/nk1U\n4+IijJ8RUUZK52o2MtWBpteRpmSvLLZHa7Ld0BKMrBOKjluxiipoIF+MLk5/C9S1LRRxP8olNSOd\nkz7Os6hpmiYy5K44wWusxtUpH9sRh08fI0K3FyKX+9bt7WqbPZXPRLinbpjur/MrR/D9mbvZZjNg\n351Zs1FtSbzVl1viJKMjHes5ey4o017bOt0weKK/M78TZV6HGTKupxJytI5PJLxKPx6/+H6V/wB9\nLwJn4gOK1eBbj1VY7ZOCz5ePTZkWpVkLn6hSoIybrLKnbHqx3Ks3+PcAAgTHxOPx6/8Ah2k/99Kw\nDOiGK017/ujncMk/VfsyMiImpmIiIAiIgEphfiYWXX7TiPTmpuNwqFxh5Cr+YZ2vw2P5bYv7hIub\n3h/JrryazaeNFgsoyWHIlcfJrfHudQASXWu1nHY+si9j7JrZmNZVZZVaONuPa9VqgqQLK2KOAR2O\nzKe8orm17l3ekzyiIlygiIgCIiCRERBAiIgCIiAIiIAiIgCIiAIiZ41L2OiVqXsuda60Ubs7uwVE\nUfmxJA/5wSSPh/8AD6+We3kqitJ9Xc5mSr1Y3E/4XTa3IB2/8kR2JBkS49Uge4gfpJXWrkVa8Wll\nerELNZch5JkZVgUW3IfzqARKlPsK1ctgXYSMlIcyzuuOm+IfHGB1tSpq8zkY+su3mcrqpzpHlFxl\nOCroPVIGzLZx5VqibgDqHwH2iY/Ft6Mp2qFCYha/FFtYx6dLpTJ88lAtrvb7trsaoi2trGUlRsef\nOYmCqdGjd1uOc++7zoj+PsIP+Hj3opVQ91TYuJazeT8TYzW1U0L0MOwNrisOkoBbHsbYFiBlifaN\nRXXVUmLa1eJXXRS7vULrKqk0tVe3YbdT+wuABuAjVLu3HkecxHk6PIebpFzOi0/aLQr12DFt59R+\nsC+OyrUw8TVh6hZWym1U14MA6kcsXvuD28snx9j2VXVWUW21jFfHxFZNKoVeeB5IWA4dFdmDx7ep\nU7hqlWtuQAec/iPJ0c5yI83SSlMRETqOcREQQIiIAiIgCIiAIiIAiIgCIiAIiIJEREAREQBJLxLY\nGvrKkMBp+lrupUjkml4COu4/MMrKR+RUiRsSJXzJndIRESSoiIgCIiAJK6x+NTTljuwCYmb7xfTX\ntRae5O1uPWO57mzGyT7pFTd0fNFTMtgZ8fJTpZVSlQzV8gwesnstyOqWKT25IAdwWU1iXNF4XyZp\nSZydO0+rprdl5IsfGxb2WvBx7K181jUZSotj6gjPstygkqvcHtNHVsE0OByFlVq9THyFDCvIpJKr\nYm/cHkrKVPdXR1OxBE3PERTzVHUDGsYGjdQIVDlPurTuYQnsGK77E/ntKNzak+T+i6Uk5rmvsw6G\nl/7Xmfw7E/qkdDS/9rzP4dif1SdS+0TQvDGq1U5Hhi/Cwq9NpKarXk/eeMQr9AY1rK2O68Vbqo17\nMByeoEn1ZH+DvsH17PJZmxsbGajq42Z16MqnL3OyioYzsQns3ZguwI2DeycirsFi1HE4NHJPaXud\nLqkdqzDCotVORz59PxHqyLMfJvsbEqS50uxKMdWrbJxsY8LK8y08w+TWdioGwbvuADEk7e3tLP4d\n09UbWqcpummHgMMpqiHb+z6xpSNXUT2Z3sC1htiAbFJ7byObxBkoSMXjg1/4VxgqW8fyFmXt17zu\nSfXcjcnYKNlHVDG5tK//ABHNHAkk3d/rIkGfZbs3Ss/ljVathPjNqnJcDMux68O/qhkQWX7KrZFB\nd6gxuVmCvup7FXqLAgkEFSCQVI2II7EEfkQZaCkUXdxSOBw93iIiaTKiIiJgRERMCIiJgRPhO3t7\nSTp0S/itl/HDpcBluyedfNSAQ1NIU3ZK+svepGA5DcqO8hxJYhJvAj6q3dlRFZ3sZUREVmd3YhVR\nFXuzEkAAdyTJfIKYaPUhV8y5GrybkZWTFqcFXxanU7Pc6kq1g7BSyAndyfKzUaalavCDLzUrbmWh\nFyrFZSr11qjMuHUQSpVGZiCwLkE1iLlZOLHAvNQ4YiIiXKCIiCBERAEREAREQBERAEREAREQBERA\nEREAREQBERAEREAREQBERAEREAREQBERAEREAREQCV8O+IMrCcGrpWoDY3lcmmvKxC9lTUtace0F\nepwbbkNj2AO43B9fFWRvmV2uqN/Y9ItergtVLf8A5Xp7tXwqCrXWe44oAADsNtpCy06ro2pPfjX4\n2Dk2qMDR3purxMi6pmq0zT+4K1lXAdCCO43UgzCOGGGKejN4W4oZar7O++kN+bfXqFPhi3TKNM08\n0+cuxsXH1J7bgGqxsB8uvy9eJwWxWtuXbp3uAUJCWaNfh3E13Vny6fFFuDadLKPgY2bhWZeGFZVd\nBkYOVbVZhswFjLzdiz7sdz25p9rfiHxF4gfCbL0W+gabVaqLTh6keVmR0TdYTYhIU9BNl/LY923l\nHPhbVT7dMzT/AL8PN+nPJoai5WnFYcpcopL1c55np0tbvlZtLHKfspSJDw/pnM63j02Lc1WnOuO6\nfsZRo1jSiOjt3JsrrJVe5LGsDuRNzwrrWlaZQmZR1czWyjeVFtNSYGlWEsoyQGdjn5SqAy7qqgvu\nQSomjhaXnYuPqNl+Nk4obCpSq22nIxwbfvXS7FWt7FG9nGt2AB32rY/kZovqlFpLZeL1LGO7ZGPb\n5S61j7WtBrspY+zdlqQk7kliSZ6To7c1ip8pX3LHR8/4PPtWZcnLnO694aoxwRbnZqnJuste+zq5\neXY722LTX+JfkPY55HhUrt7f8Ow77CbGoeKMi2623pYa9e6y3idM0NyvUcvxLticnI323PczUy9U\n3raqilMamwr1VQ2vbfwIZBffaxZ1DKrcF4LyVTxBAIj5uoE72vQxttXJ+pKff+T8PD/heg/ykff+\nT8PD/heg/wApIuJawsituLMlPv8Ayfh4f8L0H+Uj7/yfh4f8L0H+UkXEWFkPEizJT7/yfh4f8L0H\n+Uj7/wAn4eH/AAvQf5SRcRYWQ8SLMlPv/J+Hh/wvQf5SDr2T8PD/AIXoP8pIuIsLIm3FmSa6/mrv\n0rEx2PtfFx8LDt293WxKkfj+7faR1thdmdyzvYxZ3cszux7lmZu7N+8zGJKhSwRDibxYiIklRERA\nEREAREQBERAEREAREQBERAEREAROeenWX8On5bvqR6dZfw6flu+pPI41V83sevwan03OhxOeenWX\n8On5bvqR6dZfw6flu+pHGqvm9hwan03OhxOeenWX8On5bvqR6dZfw6flu+pHGqvm9hwan03OhxOe\nenWX8On5bvqR6dZfw6flu+pHGqvm9hwan03OhxOeenWX8On5bvqR6dZfw6flu+pHGqvm9hwan03O\nhxOeenWX8On5bvqR6dZfw6flu+pHGqvm9hwan03OhxOeenWX8On5bvqR6dZfw6flu+pHGqvm9hwa\nn03OhxOeenWX8On5bvqR6dZfw6flu+pHGqvm9hwan03OhxOeenWX8On5bvqR6dZfw6flu+pHGqvm\n9hwan03OhxOeenWX8On5bvqR6dZfw6flu+pHGqvm9hwan03OhxOeenWX8On5bvqR6dZfw6flu+pH\nGqvm9hwan03OhxOeenWX8On5bvqR6dZfw6flu+pHGqvm9hwan03OhzEoPcP0nPvTrL+HT8t31I9O\nsv4dPy3fUjjVX12HB6fTc6DwX3D9FjgvuH6LOfenWX8On5bvqR6dZfw6flu+pJ41V9dhwen03Ogh\nQPYAP+UynPPTrL+HT8t31I9Osv4dPy3fUkcaq+uw4PT6bnQ4nPPTrL+HT8t31I9Osv4dPy3fUjjV\nXzew4NT6bnQ4nPPTrL+HT8t31I9Osv4dPy3fUjjVXzew4NT6bnQ4nPPTrL+HT8t31I9Osv4dPy3f\nUjjVXzew4NT6bnQ4nPPTrL+HT8t31I9Osv4dPy3fUjjVXzew4NT6bnQ4nPPTrL+HT8t31I9Osv4d\nPy3fUjjVXzew4NT6bnQ4nPPTrL+HT8t31I9Osv4dPy3fUjjVXzew4NT6bnQ4nPPTrL+HT8t31I9O\nsv4dPy3fUjjVXzew4NT6bnQ4nPPTrL+HT8t31I9Osv4dPy3fUjjVXzew4NT6bnQ4nPPTrL+HT8t3\n1I9Osv4dPy3fUjjVXzew4NT6bnQ4nPPTrL+HT8t31I9Osv4dPy3fUjjVXzew4NT6bnQ4nPPTrL+H\nT8t31I9Osv4dPy3fUjjVXzew4NT6bnQ4nPPTrL+HT8t31I9Osv4dPy3fUjjVXzew4NT6bnQ4nPPT\nrL+HT8t31I9Osv4dPy3fUjjVXzew4NT6bnQ4nPPTrL+HT8t31I9Osv4dPy3fUjjVXzew4NT6bnQ4\nnPPTrL+HT8t31I9Osv4dPy3fUjjVXzew4NT6blTiInxx9WIiIAiIgCIiAIiIAiIgCIiAIiIAiIgC\nIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIg\nCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiI\ngCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAi\nIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgCIiAIiIAiIgH//Z\n",
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"800\"\n",
       "            height=\"600\"\n",
       "            src=\"https://www.youtube.com/embed/WCUNPb-5EYI\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.YouTubeVideo at 0x7f9868e6cf50>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import IPython\n",
    "IPython.display.YouTubeVideo('WCUNPb-5EYI', width=800, height=600)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Rekurencyjna sieć neuronowa – schemat\n",
    "\n",
    "<img style=\"margin: auto\" width=\"20%\" src=\"http://colah.github.io/posts/2015-08-Understanding-LSTMs/img/RNN-rolled.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Rekurencyjna sieć neuronowa – schemat\n",
    "\n",
    "<img style=\"margin: auto\" width=\"80%\" src=\"http://colah.github.io/posts/2015-08-Understanding-LSTMs/img/RNN-unrolled.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Zależności długodystansowe (_long-distance dependencies_) w sieciach rekurencyjnych\n",
    "\n",
    "<img style=\"margin: auto\" width=\"60%\" src=\"http://colah.github.io/posts/2015-08-Understanding-LSTMs/img/RNN-longtermdependencies.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### RNN – typy sekwencji\n",
    "\n",
    "<img style=\"margin: auto\" width=\"80%\" src=\"http://karpathy.github.io/assets/rnn/diags.jpeg\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Prosta sieć RNN – schemat\n",
    "\n",
    "<img src=\"rnn.png\" style=\"margin: auto;\" width=\"100%\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### LSTM – schemat\n",
    "\n",
    "<img src=\"lstm.jpg\" style=\"margin: auto;\" width=\"80%\" />\n",
    "\n",
    "* Rekurencyjne sieci neuronowe znajduja zastosowanie w przetwarzaniu sekwencji, np. szeregów czasowych i tekstów.\n",
    "* LSTM są rozwinięciem RNN, umożliwiają „zapamiętywanie” i „zapominanie”."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Co potrafią generować rekurencyjne sieci neuronowe?\n",
    "\n",
    "http://karpathy.github.io/2015/05/21/rnn-effectiveness/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Generowanie tekstu za pomocą LSTM – przykład\n",
    "\n",
    "https://github.com/keras-team/keras/blob/master/examples/lstm_text_generation.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Przewidywanie ciągów czasowych za pomocą LSTM – przykład\n",
    "\n",
    "https://machinelearningmastery.com/time-series-forecasting-long-short-term-memory-network-python/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## GRU – _Gated Recurrent Unit_\n",
    "\n",
    "* Rodzaj rekurencyjnej sieci neuronowej wprwadzony w 2014 roku\n",
    "* Ma prostszą budowę niż LSTM (2 bramki zamiast 3).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### GRU – schemat\n",
    "\n",
    "<img src=\"gru.png\" style=\"margin: auto;\" width=\"50%\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### GRU vs LSTM\n",
    "* LSTM – 3 bramki: wejścia (_input_), wyjścia (_output_) i zapomnienia (_forget_); GRU – 2 bramki: resetu (_reset_) i aktualizacji (_update_). Bramka resetu pełni podwójną funkcję: zastępuje bramki wyjścia i zapomnienia.\n",
    "* GRU i LSTM mają podobną skuteczność, ale GRU dzięki prostszej budowie bywa bardziej wydajna.\n",
    "* LSTM sprawdza się lepiej w przetwarzaniu tekstu, ponieważ lepiej zapamiętuje zależności długosystansowe."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "livereveal": {
   "start_slideshow_at": "selected",
   "theme": "amu"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}