{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# 11. Sieci neuronowe – propagacja wsteczna" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## 11.1. Metoda propagacji wstecznej – wprowadzenie" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "\"Rys." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Architektura sieci neuronowych\n", "\n", "* Budowa warstwowa, najczęściej sieci jednokierunkowe i gęste.\n", "* Liczbę i rozmiar warstw dobiera się do każdego problemu.\n", "* Rozmiary sieci określane poprzez liczbę neuronów lub parametrów." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### _Feedforward_\n", "\n", "Mając daną $n$-warstwową sieć neuronową oraz jej parametry $\\Theta^{(1)}, \\ldots, \\Theta^{(L)} $ oraz $\\beta^{(1)}, \\ldots, \\beta^{(L)} $, obliczamy:\n", "\n", "$$a^{(l)} = g^{(l)}\\left( a^{(l-1)} \\Theta^{(l)} + \\beta^{(l)} \\right). $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "\"Rys." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Funkcje $g^{(l)}$ to **funkcje aktywacji**.
\n", "Dla $i = 0$ przyjmujemy $a^{(0)} = x$ (wektor wierszowy cech) oraz $g^{(0)}(x) = x$ (identyczność)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Parametry $\\Theta$ to wagi na połączeniach miedzy neuronami dwóch warstw.
\n", "Rozmiar macierzy $\\Theta^{(l)}$, czyli macierzy wag na połączeniach warstw $a^{(l-1)}$ i $a^{(l)}$, to $\\dim(a^{(l-1)}) \\times \\dim(a^{(l)})$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Parametry $\\beta$ zastępują tutaj dodawanie kolumny z jedynkami do macierzy cech.
Macierz $\\beta^{(l)}$ ma rozmiar równy liczbie neuronów w odpowiedniej warstwie, czyli $1 \\times \\dim(a^{(l)})$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **Klasyfikacja**: dla ostatniej warstwy $L$ (o rozmiarze równym liczbie klas) przyjmuje się $g^{(L)}(x) = \\mathop{\\mathrm{softmax}}(x)$.\n", "* **Regresja**: pojedynczy neuron wyjściowy; funkcją aktywacji może wtedy być np. funkcja identycznościowa." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Jak uczyć sieci neuronowe?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* W poznanych do tej pory algorytmach (regresja liniowa, regresja logistyczna) do uczenia używaliśmy funkcji kosztu, jej gradientu oraz algorytmu gradientu prostego (GD/SGD)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Dla sieci neuronowych potrzebowalibyśmy również znaleźć gradient funkcji kosztu." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Sprowadza się to do bardziej ogólnego problemu:
jak obliczyć gradient $\\nabla f(x)$ dla danej funkcji $f$ i wektora wejściowego $x$?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Pochodna funkcji\n", "\n", "* **Pochodna** mierzy, jak szybko zmienia się wartość funkcji względem zmiany jej argumentów:\n", "\n", "$$ \\frac{d f(x)}{d x} = \\lim_{h \\to 0} \\frac{ f(x + h) - f(x) }{ h } $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Pochodna cząstkowa i gradient\n", "\n", "* **Pochodna cząstkowa** mierzy, jak szybko zmienia się wartość funkcji względem zmiany jej *pojedynczego argumentu*." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* **Gradient** to wektor pochodnych cząstkowych:\n", "\n", "$$ \\nabla f = \\left( \\frac{\\partial f}{\\partial x_1}, \\ldots, \\frac{\\partial f}{\\partial x_n} \\right) $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Gradient – przykłady" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$$ f(x_1, x_2) = x_1 + x_2 \\qquad \\to \\qquad \\frac{\\partial f}{\\partial x_1} = 1, \\quad \\frac{\\partial f}{\\partial x_2} = 1, \\quad \\nabla f = (1, 1) $$ " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$$ f(x_1, x_2) = x_1 \\cdot x_2 \\qquad \\to \\qquad \\frac{\\partial f}{\\partial x_1} = x_2, \\quad \\frac{\\partial f}{\\partial x_2} = x_1, \\quad \\nabla f = (x_2, x_1) $$ " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Własności pochodnych cząstkowych\n", "\n", "Jeżeli $f(x, y, z) = (x + y) \\, z$ oraz $x + y = q$, to:\n", "$$f = q z,\n", "\\quad \\frac{\\partial f}{\\partial q} = z,\n", "\\quad \\frac{\\partial f}{\\partial z} = q,\n", "\\quad \\frac{\\partial q}{\\partial x} = 1,\n", "\\quad \\frac{\\partial q}{\\partial y} = 1 $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### Reguła łańcuchowa\n", "\n", "$$ \\frac{\\partial f}{\\partial x} = \\frac{\\partial f}{\\partial q} \\, \\frac{\\partial q}{\\partial x},\n", "\\quad \\frac{\\partial f}{\\partial y} = \\frac{\\partial f}{\\partial q} \\, \\frac{\\partial q}{\\partial y} $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Propagacja wsteczna – prosty przykład" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "# Dla ustalonego wejścia\n", "x = -2\n", "y = 5\n", "z = -4" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "3 -12\n" ] } ], "source": [ "# Krok w przód\n", "q = x + y\n", "f = q * z\n", "print(q, f)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "slideshow": { "slide_type": "fragment" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[-4, -4, 3]\n" ] } ], "source": [ "# Propagacja wsteczna dla f = q * z\n", "# Oznaczmy symbolami `dfx`, `dfy`, `dfz`, `dfq` odpowiednio\n", "# pochodne cząstkowe ∂f/∂x, ∂f/∂y, ∂f/∂z, ∂f/∂q\n", "dfz = q\n", "dfq = z\n", "# Propagacja wsteczna dla q = x + y\n", "dfx = 1 * dfq # z reguły łańcuchowej\n", "dfy = 1 * dfq # z reguły łańcuchowej\n", "print([dfx, dfy, dfz])" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "\"Rys." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Właśnie tak wygląda obliczanie pochodnych metodą propagacji wstecznej!" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Spróbujmy czegoś bardziej skomplikowanego:
metodą propagacji wstecznej obliczmy pochodną funkcji sigmoidalnej." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Propagacja wsteczna – funkcja sigmoidalna" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Funkcja sigmoidalna:\n", "\n", "$$f(\\theta,x) = \\frac{1}{1+e^{-(\\theta_0 x_0 + \\theta_1 x_1 + \\theta_2)}}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "$$\n", "\\begin{array}{lcl}\n", "f(x) = \\frac{1}{x} \\quad & \\rightarrow & \\quad \\frac{df}{dx} = -\\frac{1}{x^2} \\\\\n", "f_c(x) = c + x \\quad & \\rightarrow & \\quad \\frac{df}{dx} = 1 \\\\\n", "f(x) = e^x \\quad & \\rightarrow & \\quad \\frac{df}{dx} = e^x \\\\\n", "f_a(x) = ax \\quad & \\rightarrow & \\quad \\frac{df}{dx} = a \\\\\n", "\\end{array}\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "\"Rys." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[0.3932238664829637, -0.5898357997244456]\n", "[-0.19661193324148185, -0.3932238664829637, 0.19661193324148185]\n" ] } ], "source": [ "from math import exp\n", "\n", "\n", "# Losowe wagi i dane\n", "w = [2, -3, -3]\n", "x = [-1, -2]\n", "\n", "# Krok w przód\n", "dot = w[0] * x[0] + w[1] * x[1] + w[2]\n", "f = 1.0 / (1 + exp(-dot)) # funkcja sigmoidalna\n", "\n", "# Krok w tył\n", "ddot = (1 - f) * f # pochodna funkcji sigmoidalnej\n", "dx = [w[0] * ddot, w[1] * ddot]\n", "dw = [x[0] * ddot, x[1] * ddot, 1.0 * ddot]\n", "\n", "print(dx)\n", "print(dw)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Obliczanie gradientów – podsumowanie" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Gradient $f$ dla $x$ mówi, jak zmieni się całe wyrażenie przy zmianie wartości $x$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Gradienty łączymy, korzystając z **reguły łańcuchowej**." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* W kroku \"wstecz\" gradienty informują, które części grafu powinny być zwiększone lub zmniejszone (i z jaką siłą), aby zwiększyć wartość na wyjściu." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* W kontekście implementacji chcemy dzielić funkcję $f$ na części, dla których można łatwo obliczyć gradienty." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## 11.2. Uczenie wielowarstwowych sieci neuronowych metodą propagacji wstecznej" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Mając algorytm SGD oraz gradienty wszystkich wag, moglibyśmy trenować każdą sieć." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Niech $\\Theta = (\\Theta^{(1)},\\Theta^{(2)},\\Theta^{(3)},\\beta^{(1)},\\beta^{(2)},\\beta^{(3)})$\n", "* Funkcja sieci neuronowej z grafiki:\n", "$$\\small h_\\Theta(x) = \\tanh(\\tanh(\\tanh(x\\Theta^{(1)}+\\beta^{(1)})\\Theta^{(2)} + \\beta^{(2)})\\Theta^{(3)} + \\beta^{(3)})$$\n", "* Funkcja kosztu dla regresji:\n", "$$J(\\Theta) = \\dfrac{1}{2m} \\sum_{i=1}^{m} (h_\\Theta(x^{(i)})- y^{(i)})^2 $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Jak obliczymy gradienty?\n", "\n", "$$\\nabla_{\\Theta^{(l)}} J(\\Theta) = ? \\quad \\nabla_{\\beta^{(l)}} J(\\Theta) = ?$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### W kierunku propagacji wstecznej\n", "\n", "* Pewna (niewielka) zmiana wagi $\\Delta z^l_j$ dla $j$-ego neuronu w warstwie $l$ pociąga za sobą (niewielką) zmianę kosztu: \n", "\n", "$$\\frac{\\partial J(\\Theta)}{\\partial z^{l}_j} \\Delta z^{l}_j$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Jeżeli $\\frac{\\partial J(\\Theta)}{\\partial z^{l}_j}$ jest duża, $\\Delta z^l_j$ ze znakiem przeciwnym zredukuje koszt.\n", "* Jeżeli $\\frac{\\partial J(\\Theta)}{\\partial z^l_j}$ jest bliska zeru, koszt nie będzie mocno poprawiony." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "* Definiujemy błąd $\\delta^l_j$ neuronu $j$ w warstwie $l$: \n", "\n", "$$\\delta^l_j := \\dfrac{\\partial J(\\Theta)}{\\partial z^l_j}$$ \n", "$$\\delta^l := \\nabla_{z^l} J(\\Theta) \\quad \\textrm{ (zapis wektorowy)} $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Podstawowe równania propagacji wstecznej\n", "\n", "$$\n", "\\begin{array}{rcll}\n", "\\delta^L & = & \\nabla_{a^L}J(\\Theta) \\odot { \\left( g^{L} \\right) }^{\\prime} \\left( z^L \\right) & (BP1) \\\\[2mm]\n", "\\delta^{l} & = & \\left( \\left( \\Theta^{l+1} \\right) \\! ^\\top \\, \\delta^{l+1} \\right) \\odot {{ \\left( g^{l} \\right) }^{\\prime}} \\left( z^{l} \\right) & (BP2)\\\\[2mm]\n", "\\nabla_{\\beta^l} J(\\Theta) & = & \\delta^l & (BP3)\\\\[2mm]\n", "\\nabla_{\\Theta^l} J(\\Theta) & = & a^{l-1} \\odot \\delta^l & (BP4)\\\\\n", "\\end{array}\n", "$$\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### (BP1)\n", "$$ \\delta^L_j \\; = \\; \\frac{ \\partial J }{ \\partial a^L_j } \\, g' \\!\\! \\left( z^L_j \\right) $$\n", "$$ \\delta^L \\; = \\; \\nabla_{a^L}J(\\Theta) \\odot { \\left( g^{L} \\right) }^{\\prime} \\left( z^L \\right) $$\n", "Błąd w ostatniej warstwie jest iloczynem szybkości zmiany kosztu względem $j$-tego wyjścia i szybkości zmiany funkcji aktywacji w punkcie $z^L_j$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### (BP2)\n", "$$ \\delta^{l} \\; = \\; \\left( \\left( \\Theta^{l+1} \\right) \\! ^\\top \\, \\delta^{l+1} \\right) \\odot {{ \\left( g^{l} \\right) }^{\\prime}} \\left( z^{l} \\right) $$\n", "Aby obliczyć błąd w $l$-tej warstwie, należy przemnożyć błąd z następnej ($(l+1)$-szej) warstwy przez transponowany wektor wag, a uzyskaną macierz pomnożyć po współrzędnych przez szybkość zmiany funkcji aktywacji w punkcie $z^l$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### (BP3)\n", "$$ \\nabla_{\\beta^l} J(\\Theta) \\; = \\; \\delta^l $$\n", "Błąd w $l$-tej warstwie jest równy wartości gradientu funkcji kosztu." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "#### (BP4)\n", "$$ \\nabla_{\\Theta^l} J(\\Theta) \\; = \\; a^{l-1} \\odot \\delta^l $$\n", "Gradient funkcji kosztu względem wag $l$-tej warstwy można obliczyć jako iloczyn po współrzędnych $a^{l-1}$ przez $\\delta^l$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Algorytm propagacji wstecznej" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "Dla pojedynczego przykładu $(x,y)$:\n", "1. **Wejście**: Ustaw aktywacje w warstwie cech $a^{(0)}=x$ \n", "2. **Feedforward:** dla $l=1,\\dots,L$ oblicz \n", "$z^{(l)} = a^{(l-1)} \\Theta^{(l)} + \\beta^{(l)}$ oraz $a^{(l)}=g^{(l)} \\!\\! \\left( z^{(l)} \\right)$\n", "3. **Błąd wyjścia $\\delta^{(L)}$:** oblicz wektor $$\\delta^{(L)}= \\nabla_{a^{(L)}}J(\\Theta) \\odot {g^{\\prime}}^{(L)} \\!\\! \\left( z^{(L)} \\right) $$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "4. **Propagacja wsteczna błędu:** dla $l = L-1,L-2,\\dots,1$ oblicz $$\\delta^{(l)} = \\delta^{(l+1)}(\\Theta^{(l+1)})^T \\odot {g^{\\prime}}^{(l)} \\!\\! \\left( z^{(l)} \\right) $$\n", "5. **Gradienty:** \n", " * $\\dfrac{\\partial}{\\partial \\Theta_{ij}^{(l)}} J(\\Theta) = a_i^{(l-1)}\\delta_j^{(l)} \\textrm{ oraz } \\dfrac{\\partial}{\\partial \\beta_{j}^{(l)}} J(\\Theta) = \\delta_j^{(l)}$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "W naszym przykładzie:\n", "\n", "$$\\small J(\\Theta) = \\frac{1}{2} \\left( a^{(L)} - y \\right) ^2 $$\n", "$$\\small \\dfrac{\\partial}{\\partial a^{(L)}} J(\\Theta) = a^{(L)} - y$$\n", "\n", "$$\\small \\tanh^{\\prime}(x) = 1 - \\tanh^2(x)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "\"Rys." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Algorytm SGD z propagacją wsteczną\n", "\n", "Pojedyncza iteracja:\n", "1. Dla parametrów $\\Theta = (\\Theta^{(1)},\\ldots,\\Theta^{(L)})$ utwórz pomocnicze macierze zerowe $\\Delta = (\\Delta^{(1)},\\ldots,\\Delta^{(L)})$ o takich samych wymiarach (dla uproszczenia opuszczono wagi $\\beta$)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "2. Dla $m$ przykładów we wsadzie (*batch*), $i = 1,\\ldots,m$:\n", " * Wykonaj algortym propagacji wstecznej dla przykładu $(x^{(i)}, y^{(i)})$ i przechowaj gradienty $\\nabla_{\\Theta}J^{(i)}(\\Theta)$ dla tego przykładu;\n", " * $\\Delta := \\Delta + \\dfrac{1}{m}\\nabla_{\\Theta}J^{(i)}(\\Theta)$\n", "3. Wykonaj aktualizację wag: $\\Theta := \\Theta - \\alpha \\Delta$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Propagacja wsteczna – podsumowanie\n", "\n", "* Algorytm pierwszy raz wprowadzony w latach 70. XX w.\n", "* W 1986 David Rumelhart, Geoffrey Hinton i Ronald Williams pokazali, że jest znacznie szybszy od wcześniejszych metod.\n", "* Obecnie najpopularniejszy algorytm uczenia sieci neuronowych." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## 11.3. Przykłady implementacji wielowarstwowych sieci neuronowych" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "notes" } }, "source": [ "### Uwaga!\n", "\n", "Poniższe przykłady wykorzystują interfejs [Keras](https://keras.io), który jest częścią biblioteki [TensorFlow](https://www.tensorflow.org).\n", "\n", "Aby uruchomić TensorFlow w środowisku Jupyter, należy wykonać następujące czynności:\n", "\n", "#### Przed pierwszym uruchomieniem (wystarczy wykonać tylko raz)\n", "\n", "Instalacja biblioteki TensorFlow w środowisku Anaconda:\n", "\n", "1. Uruchom *Anaconda Navigator*\n", "1. Wybierz kafelek *CMD.exe Prompt*\n", "1. Kliknij przycisk *Launch*\n", "1. Pojawi się konsola. Wpisz następujące polecenia, każde zatwierdzając wciśnięciem klawisza Enter:\n", "```\n", "conda create -n tf tensorflow\n", "conda activate tf\n", "conda install pandas matplotlib\n", "jupyter notebook\n", "```\n", "\n", "#### Przed każdym uruchomieniem\n", "\n", "Jeżeli chcemy korzystać z biblioteki TensorFlow, to środowisko Jupyter Notebook należy uruchomić w następujący sposób:\n", "\n", "1. Uruchom *Anaconda Navigator*\n", "1. Wybierz kafelek *CMD.exe Prompt*\n", "1. Kliknij przycisk *Launch*\n", "1. Pojawi się konsola. Wpisz następujące polecenia, każde zatwierdzając wciśnięciem klawisza Enter:\n", "```\n", "conda activate tf\n", "jupyter notebook\n", "```" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "### Przykład: MNIST\n", "\n", "_Modified National Institute of Standards and Technology database_" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "* Zbiór cyfr zapisanych pismem odręcznym\n", "* 60 000 przykładów uczących, 10 000 przykładów testowych\n", "* Rozdzielczość każdego przykładu: 28 × 28 = 784 piksele" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-01-26 10:52:17.922141: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA\n", "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n", "2023-01-26 10:52:18.163925: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n", "2023-01-26 10:52:18.163996: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n", "2023-01-26 10:52:19.577890: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n", "2023-01-26 10:52:19.578662: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n", "2023-01-26 10:52:19.578677: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n", "11490434/11490434 [==============================] - 1s 0us/step\n" ] } ], "source": [ "from tensorflow import keras\n", "from tensorflow.keras.datasets import mnist\n", "from tensorflow.keras.layers import Dense, Dropout\n", "\n", "# załaduj dane i podziel je na zbiory uczący i testowy\n", "(x_train, y_train), (x_test, y_test) = mnist.load_data()" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [], "source": [ "from matplotlib import pyplot as plt\n", "\n", "\n", "def draw_examples(examples, captions=None):\n", " plt.figure(figsize=(16, 4))\n", " m = len(examples)\n", " for i, example in enumerate(examples):\n", " plt.subplot(100 + m * 10 + i + 1)\n", " plt.imshow(example, cmap=plt.get_cmap(\"gray\"))\n", " plt.show()\n", " if captions is not None:\n", " print(6 * \" \" + (10 * \" \").join(str(captions[i]) for i in range(m)))" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABQcAAADFCAYAAADpJUQuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAomElEQVR4nO3de3RU1d3G8V8CJFyTyC2BQiQqFRQJGgFRFqBE0KoQoKIUBKwFCwFEK6X4oqCIQVBbrmK1kKIoaBFQvFKuVUPK1S5AIlKEIEkAJRcCJErO+4fLVNx78ExmJjOz9/ez1vnDh30m+4Qnh2Eznh3hOI4jAAAAAAAAAKwTGewJAAAAAAAAAAgOFgcBAAAAAAAAS7E4CAAAAAAAAFiKxUEAAAAAAADAUiwOAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABL1QzUC8+fP19mzZol+fn5kpycLHPnzpVOnTr97HkVFRVy9OhRadCggURERARqerCU4zhSUlIizZs3l8hI79bGq9ppEXqNwPGl0yLcqxF6gtVpEXqNwOFeDdNwr4aJuFfDNF512gmAZcuWOVFRUc6iRYucPXv2OCNGjHDi4uKcgoKCnz03NzfXEREOjoAeubm51dZpes1RHYe3nfa113SaI9BHdXeaXnNUx8G9msO0g3s1h4kH92oO0w43nQ7I4mCnTp2c9PT0yv8+d+6c07x5cycjI+Nnzy0sLAz6N47D/KOwsLDaOk2vOarj8LbTvvaaTnME+qjuTtNrjuo4uFdzmHZwr+Yw8eBezWHa4abTfn/mYHl5uWzfvl1SU1Mrs8jISElNTZWsrCxlfFlZmRQXF1ceJSUl/p4SoPDm49redlqEXqP6efu/IHCvRqgLdKdF6DWqH/dqmIZ7NUzEvRqmcdNpvy8OnjhxQs6dOyfx8fHn5fHx8ZKfn6+Mz8jIkNjY2MqjZcuW/p4S4BNvOy1CrxH6uFfDNNyrYSLu1TAN92qYiHs1TBD03YonTZokRUVFlUdubm6wpwT4jF7DNHQaJqLXMA2dhonoNUxDpxGK/L5bcePGjaVGjRpSUFBwXl5QUCAJCQnK+OjoaImOjvb3NAC/8bbTIvQaoY97NUzDvRom4l4N03Cvhom4V8MEfv/kYFRUlKSkpMi6desqs4qKClm3bp106dLF318OCDg6DRPRa5iGTsNE9BqmodMwEb2GEbzehseFZcuWOdHR0U5mZqazd+9eZ+TIkU5cXJyTn5//s+cWFRUFfScXDvOPoqKiaus0veaojsPbTvvaazrNEeijujtNrzmq4+BezWHawb2aw8SDezWHaYebTgdkcdBxHGfu3LlOYmKiExUV5XTq1MnZsmWLq/P4weCojqMqN/yqdppec1THUZVO+9JrOs0R6KO6O02vOarj4F7NYdrBvZrDxIN7NYdph5tORziO40gIKS4ultjY2GBPA4YrKiqSmJiYavt69BqBRqdhmurutAi9RuBxr4ZpuFfDRNyrYRo3nQ76bsUAAAAAAAAAgoPFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUi4MAAAAAAACApVgcBAAAAAAAACzF4iAAAAAAAABgKRYHAQAAAAAAAEuxOAgAAAAAAABYisVBAAAAAAAAwFIsDgIAAAAAAACWqhnsCQCAWykpKdp8zJgxSjZ06FDt2CVLlijZ3LlztWN37NjhxewAAAAAAFUxe/ZsbT5u3Dgl2717t5Ldfvvt2vMPHTrk28QswScHAQAAAAAAAEuxOAgAAAAAAABYisVBAAAAAAAAwFIsDgIAAAAAAACWYkOSEFWjRg1tHhsb69Pr6jZuqFu3rnbs5ZdfrmTp6enasc8884ySDRo0SMnOnj2rPX/GjBlK9vjjj2vHwg4dOnRQsrVr12rHxsTEKJnjONqx99xzj5L16dNHO7ZRo0YXmCEQfnr27KlkS5cuVbLu3btrz8/JyfH7nACdyZMnK5mn9wWRkeq/dffo0UPJNm3a5PO8AMAUDRo00Ob169dXsttuu007tkmTJkr23HPPaceWlZV5MTuYrlWrVko2ZMgQ7diKigola9u2rZK1adNGez4bkrjDJwcBAAAAAAAAS7E4CAAAAAAAAFiKxUEAAAAAAADAUiwOAgAAAAAAAJZicRAAAAAAAACwFLsV+ygxMVGbR0VFKdn111+vHdu1a1cli4uL044dMGCA+8n56MiRI0o2Z84c7dh+/fopWUlJiZJ9+umn2vPZQdBunTp1UrIVK1YomafdunU7E+v6JyJSXl6uZJ52Jb7uuuuUbMeOHa5eE1XXrVs3JfP0e7Ry5cpAT8coHTt2VLKtW7cGYSbA94YPH67NJ06cqGS63Qo98bRjPQCYTrcLrO6e2qVLF+357dq18+nrN2vWTJuPGzfOp9eFWY4fP65kmzdv1o7t06dPoKcD4ZODAAAAAAAAgLVYHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABLsSGJFzp06KBk69ev1471tHFCKPL0gO/Jkycr2alTp7Rjly5dqmR5eXlKdvLkSe35OTk5F5oiwlDdunWV7JprrtGOfeWVV5TM08OM3dq/f782nzlzppItW7ZMO/bjjz9WMt3PRUZGhpezw4X06NFDyVq3bq0dy4YkepGR+n/7S0pKUrKLL75YySIiIvw+J0BH1z8Rkdq1a1fzTGCyzp07a/MhQ4YoWffu3ZXsyiuvdP21Hn74YW1+9OhRJdNtSiiif1+UnZ3teg4wT5s2bZRs/Pjx2rGDBw9Wsjp16iiZpz/rc3NzlczTRn9t27ZVsoEDB2rHLliwQMn27dunHQvzlZaWKtmhQ4eCMBP8gE8OAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsxYYkXjh8+LCSff3119qx1bkhie4BxYWFhdqxN954o5KVl5drx7788ss+zQt2e+GFF5Rs0KBB1fb1PW1+Ur9+fSXbtGmTdqxuY4z27dv7NC/8vKFDhypZVlZWEGYSvjxt6DNixAgl0z34ngeEIxBSU1OVbOzYsa7P99TL22+/XckKCgrcTwxGueuuu5Rs9uzZ2rGNGzdWMt0mDRs3btSe36RJEyWbNWvWz8zwwl/L0+vefffdrl8X4UH398Wnn35aO1bX6wYNGvj09T1t3te7d28lq1Wrlnas7r6s+7m6UA47xcXFKVlycnL1TwSV+OQgAAAAAAAAYCkWBwEAAAAAAABLsTgIAAAAAAAAWIrFQQAAAAAAAMBSLA4CAAAAAAAAlmK3Yi988803SjZhwgTtWN3OeTt37tSOnTNnjus57Nq1S8luvvlmJSstLdWef+WVVyrZAw884PrrAz+VkpKizW+77TYl87Qrn45uB+G3335bO/aZZ55RsqNHj2rH6n4OT548qR170003KZk314CqiYzk36189dJLL7ke62m3QqCqunbtqs0XL16sZLrdOj3xtAvsoUOHXL8GwlPNmupfWa699lrt2BdffFHJ6tatqx27efNmJZs2bZqSffTRR9rzo6Ojlez111/Xju3Vq5c219m2bZvrsQhf/fr1U7Lf/e53AflaBw4cUDLd3yFFRHJzc5Xssssu8/ucYDfdfTkxMdGn1+zYsaM21+2qzXsHFX8DAwAAAAAAACzF4iAAAAAAAABgKRYHAQAAAAAAAEuxOAgAAAAAAABYyusNSTZv3iyzZs2S7du3S15enqxcuVLS0tIqf91xHJkyZYq8+OKLUlhYKDfccIM8//zz0rp1a3/OO2SsWrVKm69fv17JSkpKtGOTk5OV7L777tOO1W284GnzEZ09e/Yo2ciRI12fbyI67V6HDh2UbO3atdqxMTExSuY4jnbse++9p2SDBg1Ssu7du2vPnzx5spJ52pDh+PHjSvbpp59qx1ZUVCiZbqOVa665Rnv+jh07tHmghUun27dvr83j4+OrdR4m8maTB08/w6EmXHoNkWHDhmnz5s2bu36NjRs3KtmSJUuqOqWQRKfdGzJkiJJ5s/GSp/vcXXfdpWTFxcWuX1d3vjcbjxw5ckSb//3vf3f9GqGGXrt35513+nT+l19+qc23bt2qZBMnTlQy3cYjnrRt29b1WNPQ6cDQbR6ZmZmpHTt16lRXr+lpXGFhoZLNmzfP1WvaxOtPDpaWlkpycrLMnz9f++szZ86UOXPmyMKFCyU7O1vq1asnvXv3lrNnz/o8WSAQ6DRMQ6dhInoN09BpmIhewzR0Grbw+pODt956q9x6663aX3McR/7yl7/I5MmTpW/fviLy/b/yxsfHy6pVq+Tuu+/2bbZAANBpmIZOw0T0Gqah0zARvYZp6DRs4ddnDh48eFDy8/MlNTW1MouNjZXOnTtLVlaW9pyysjIpLi4+7wBCRVU6LUKvEbroNExEr2EaOg0T0WuYhk7DJH5dHMzPzxcR9XlR8fHxlb/2UxkZGRIbG1t5tGzZ0p9TAnxSlU6L0GuELjoNE9FrmIZOw0T0Gqah0zBJ0HcrnjRpkhQVFVUe3jwYFQhV9BqmodMwEb2Gaeg0TESvYRo6jVDk9TMHLyQhIUFERAoKCqRZs2aVeUFBgXaXUxGR6OhoiY6O9uc0QoI3Hw0uKipyPXbEiBFKtnz5ciXT7bIK71Wl0yLh3+tf/vKX2nzChAlK5mlX1BMnTihZXl6edqxuV75Tp04p2TvvvKM931MeCHXq1FGyP/zhD9qxgwcPDvR0vBZKnf7Vr36lzXXfY3im2905KSnJ9flfffWVP6cTFKHUa9s0btxYyX77299qx+rem+h2EBQRefLJJ32aV7iztdPTpk3T5o888oiSOY6jHbtgwQIlmzx5snasr/8r3//93//5dP64ceO0+fHjx3163VBla6890f29buTIkdqxH374oZJ98cUX2rHHjh3zbWIauvcaoNP+5unPALe7FcM3fv3kYFJSkiQkJMi6desqs+LiYsnOzpYuXbr480sB1YJOwzR0Giai1zANnYaJ6DVMQ6dhEq8/OXjq1Knz/pXi4MGDsmvXLmnYsKEkJibK+PHj5cknn5TWrVtLUlKSPProo9K8eXNJS0vz57wBv6HTMA2dhonoNUxDp2Eieg3T0GnYwuvFwW3btsmNN95Y+d8PPfSQiIgMGzZMMjMz5Y9//KOUlpbKyJEjpbCwULp27Srvv/++1K5d23+zBvyITsM0dBomotcwDZ2Gieg1TEOnYQuvFwd79Ojh8RkfIiIRERHyxBNPyBNPPOHTxIDqQqdhGjoNE9FrmIZOw0T0Gqah07CFXzckQdXoHrCZkpKiHdu9e3clS01NVTLdQ2sBHd3DcJ955hntWN0GEiUlJdqxQ4cOVbJt27Zpx4b7BhSJiYnBnkJYuvzyy12P3bNnTwBnEt50P6+eHhz++eefK5mnn2Hgp1q1aqVkK1as8Ok1586dq803bNjg0+si9D322GNKptt4RESkvLxcyT744APt2IkTJyrZmTNnXM9L92mfXr16acfq/vyPiIjQjtVtsrN69WrX84J5jh49qmShuvECz89DMEVGqltlsAGr//l1QxIAAAAAAAAA4YPFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUi4MAAAAAAACApditOASUlpYq2YgRI7Rjd+zYoWQvvviiknna5U+3W+z8+fO1Yy+0ZTvMcfXVVyuZbldiT/r27avNN23aVOU5AT+1devWYE8hIGJiYrT5LbfcomRDhgzRjvW0i6bOtGnTlKywsND1+bCbrpft27d3ff66deuUbPbs2T7NCeEhLi5OyUaPHq1knt576nYmTktL83VactlllynZ0qVLlSwlJcX1a/7jH//Q5jNnznQ/McAH48aNU7J69er59JpXXXWV67GffPKJNs/KyvJpDrCXbmdi1ir8j08OAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsxYYkIerAgQPafPjw4Uq2ePFiJbvnnnu05+tyTw+oXbJkiZLl5eVpxyJ8Pffcc0oWERGhHavbZMTkjUciI9V/P9E9EBeB17Bhw4C8bnJyspJ56n9qaqqStWjRQjs2KipKyQYPHqxkuo6JiJw5c0bJsrOztWPLysqUrGZN/R/v27dv1+bAj3na6GHGjBmuzv/oo4+0+bBhw5SsqKjI9bwQvnT3xMaNG7s+X7fBQtOmTbVj7733XiXr06ePdmy7du2UrH79+krm6cH3uvyVV17RjtVtQAj8VN26dbX5FVdcoWRTpkzRjnW7saCn9yDevNc9evSokul+BkVEzp075/p1AVQ/PjkIAAAAAAAAWIrFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUG5KEmZUrVyrZ/v37lUy3yYSISM+ePZXsqaee0o69+OKLlWz69OnasV999ZU2R2i5/fbblaxDhw5K5unB22+99Za/pxTSdA9k1n1vdu3aVQ2zMY9u0w0R/fd44cKF2rGPPPKIT3No3769knnakOS7775TstOnT2vH7t27V8kWLVqkZNu2bdOer9vop6CgQDv2yJEjSlanTh3t2H379mlz2KtVq1ZKtmLFCp9e87///a8299RhmK+8vFzJjh8/rmRNmjTRnn/w4EEl8/RexRu6zRSKi4uVrFmzZtrzT5w4oWRvv/22z/OCWWrVqqXNr776aiXzdP/VddDT+yhdr7OyspTslltu0Z7vaVMUHd0GaP3799eOnT17tpLp7g0AgoNPDgIAAAAAAACWYnEQAAAAAAAAsBSLgwAAAAAAAIClWBwEAAAAAAAALMXiIAAAAAAAAGApdis2wO7du5Vs4MCB2rF33HGHki1evFg79v7771ey1q1ba8fefPPNF5oiQoRuB9OoqCglO3bsmPb85cuX+31O1S06OlrJpk6d6vr89evXK9mkSZN8mZK1Ro8erc0PHTqkZNdff31A5nD48GElW7VqlXbsZ599pmRbtmzx95Q8GjlypDbX7e7pabdY4KcmTpyoZLqd2r0xY8YMn86HeQoLC5UsLS1NydasWaM9v2HDhkp24MAB7djVq1crWWZmpnbsN998o2TLli1TMk+7FevGwm6699WedgV+8803Xb/u448/rmS696QiIh9//LGS6X6GPJ3frl071/PSvQfJyMjQjnX7nqusrMz114cdIiPVz7R5816lW7duSjZv3jyf5mQiPjkIAAAAAAAAWIrFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUG5IYSvfgZxGRl19+Wcleeukl7diaNdV66B7mKSLSo0cPJdu4caPH+SG0eXoQcF5eXjXPpOp0G4+IiEyePFnJJkyYoB175MgRJXv22WeV7NSpU17ODhfy9NNPB3sKIalnz56ux65YsSKAM0E46tChgzbv1auXT6+r2/whJyfHp9eEHbKzs5VMt7lBIOne13bv3l3JPD34ns2f7FarVi0l020c4ul9ps57772nzefOnatknv6+p/s5evfdd5Xsqquu0p5fXl6uZDNnztSO1W1e0rdvX+3YpUuXKtk///lPJfP0PvDkyZPaXGfXrl2uxyL06e7BjuO4Pr9///5KdsUVV2jH7t271/3EDMMnBwEAAAAAAABLsTgIAAAAAAAAWIrFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUuxUboH379kr261//Wju2Y8eOSqbbldgTT7v3bN682fVrIPS99dZbwZ6CV3S7cHraGe6uu+5SMt1umyIiAwYM8GleQLCsXLky2FNAiPnwww+1+UUXXeT6NbZs2aJkw4cPr+qUgKCrU6eOknmzK+ayZcv8PieEnho1amjzadOmKdnDDz+sZKWlpdrz//SnPymZp07pdia+9tprtWPnzZunZFdffbWS7d+/X3v+qFGjlGzDhg3asTExMUp2/fXXa8cOHjxYyfr06aNka9eu1Z6vk5ubq82TkpJcvwZC38KFC5Xs/vvv9+k1R44cqc3Hjx/v0+uGMz45CAAAAAAAAFiKxUEAAAAAAADAUiwOAgAAAAAAAJZicRAAAAAAAACwFBuShKjLL79cm48ZM0bJ+vfvr2QJCQk+z+HcuXNKlpeXpx2re3gzQk9ERISrLC0tTXv+Aw884O8peeXBBx/U5o8++qiSxcbGascuXbpUyYYOHerbxAAgxDVq1Eibe/Pn94IFC5Ts1KlTVZ4TEGwffPBBsKeAMOBp4wLd5iOnT59WMk8bJ+g2irruuuu0Y++9914lu/XWW7VjdRvtPPHEE0q2ePFi7fmeNvnQKS4uVrL3339fO1aXDxo0SMl+85vfuP76nv5uALPs27cv2FOwAp8cBAAAAAAAACzF4iAAAAAAAABgKRYHAQAAAAAAAEuxOAgAAAAAAABYyqvFwYyMDOnYsaM0aNBAmjZtKmlpaZKTk3PemLNnz0p6ero0atRI6tevLwMGDJCCggK/ThrwJ3oN09BpmIhewzR0Gqah0zARvYYtIhzHcdwOvuWWW+Tuu++Wjh07ynfffSePPPKI7N69W/bu3Sv16tUTEZFRo0bJO++8I5mZmRIbGytjxoyRyMhI+fjjj119jeLiYo+7jIY7TzsI63Zp0u1KLCLSqlUrf05JRES2bdumzadPn65kb731lt+/fjAUFRVJTEyMiNjV6zvvvFPJXnvtNSXT7VQtIvLCCy8o2aJFi7Rjv/76ayXztAPbPffco2TJyclK1qJFC+35hw8fVrItW7Zox86ePdv12HBia6dtsnz5cm0+cOBAJRs2bJh27JIlS/w6p0D6cadF6LU3dDtQDh8+XDvWm92KL7nkEiU7dOiQ6/PBvTrU9O7dW8neffddJfP016VmzZop2fHjx32fWBip7k6LVH+v8/LytHmTJk2UrKysTMk87bT6w/fkxy677DIvZ6eaOnWqkmVkZCiZp/f74F4dDj7//HMlu/TSS12fHxmp/5yc7mfwwIED7icWon76vlqnpjcv+NPtxzMzM6Vp06ayfft26datmxQVFcnf/vY3efXVV+Wmm24Ske/foLZt21a2bNnicWEACCZ6DdPQaZiIXsM0dBqmodMwEb2GLXx65mBRUZGIiDRs2FBERLZv3y7ffvutpKamVo5p06aNJCYmSlZWlvY1ysrKpLi4+LwDCCZ6DdPQaZiIXsM0dBqm8UenReg1Qgv3apiqyouDFRUVMn78eLnhhhukXbt2IiKSn58vUVFREhcXd97Y+Ph4yc/P175ORkaGxMbGVh4tW7as6pQAn9FrmIZOw0T0Gqah0zCNvzotQq8ROrhXw2RVXhxMT0+X3bt3y7Jly3yawKRJk6SoqKjyyM3N9en1AF/Qa5iGTsNE9BqmodMwjb86LUKvETq4V8NkXj1z8AdjxoyRNWvWyObNm8/bICAhIUHKy8ulsLDwvJXzgoICj5txREdHS3R0dFWmERLi4+O1+RVXXKFk8+bN045t06aNX+ckIpKdna3NZ82apWSrV6/WjvXmIeUmoNf/U6NGDW0+evRoJRswYIB2rO7j8a1bt/ZpXp988ok237Bhg5I99thjPn0tE9BpO+gelO/pIcsmoNf/06FDB23+4/+16Qee/kwvLy9Xsvnz52vHsvNiYNDp4NFtsgPf+bPTIsHvtadPf+k2JNHNU7fJnie6DXFERDZv3qxkq1at0o798ssvlYzNR3zHvTq07NmzR8m8uafbttbhhld/e3AcR8aMGSMrV66U9evXS1JS0nm/npKSIrVq1ZJ169ZVZjk5OXL48GHp0qWLf2YM+Bm9hmnoNExEr2EaOg3T0GmYiF7DFl59cjA9PV1effVVWb16tTRo0KDyX1FiY2OlTp06EhsbK/fdd5889NBD0rBhQ4mJiZGxY8dKly5d2KUHIYtewzR0Giai1zANnYZp6DRMRK9hC68WB59//nkREenRo8d5+eLFi2X48OEiIvLnP/9ZIiMjZcCAAVJWVia9e/eWBQsW+GWyQCDQa5iGTsNE9BqmodMwDZ2Gieg1bOHV4qDuuUY/Vbt2bZk/f77H59UAoYZewzR0Giai1zANnYZp6DRMRK9hiyptSGKDhg0bKtkLL7ygZJ4eBh6oBxzrNmR49tlnleyDDz7Qnn/mzBm/zwnhIysrS8m2bt2qZB07dnT9mp4etOtpsx6dr7/+Wsl0u4A98MADrl8TsJmnZ9xkZmZW70QQUD9+8PmPXejB/j/11VdfKdnDDz9c1SkBYeVf//qXkuk2dOLB9Xbr1q2bNk9LS1Oya665RsmOHTumPX/RokVKdvLkSe1Y3eZRgM3++te/Ktkdd9wRhJmYw9ztDAEAAAAAAABcEIuDAAAAAAAAgKVYHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABLWbVbcefOnZVswoQJ2rGdOnVSsl/84hd+n5OIyOnTp5Vszpw52rFPPfWUkpWWlvp9TjDTkSNHlKx///5Kdv/992vPnzx5sk9ff/bs2dr8+eefV7IvvvjCp68F2CIiIiLYUwCAsLR7924l279/v5Jdcskl2vMvvfRSJTt+/LjvE0NIKSkp0eYvv/yyqwyA/+3du1fJPvvsM+3Ytm3bBno6RuCTgwAAAAAAAIClWBwEAAAAAAAALMXiIAAAAAAAAGApFgcBAAAAAAAAS1m1IUm/fv1cZd7QPQhTRGTNmjVK9t1332nHPvvss0pWWFjo07wAt/Ly8pRs6tSp2rGecgCB995772nzO++8s5pnglCxb98+bf7JJ58oWdeuXQM9HcAIus3/XnrpJe3Y6dOnK9nYsWO1Yz39nQEA4L1Dhw4p2VVXXRWEmZiDTw4CAAAAAAAAlmJxEAAAAAAAALAUi4MAAAAAAACApVgcBAAAAAAAACzF4iAAAAAAAABgqQjHcZxgT+LHiouLJTY2NtjTgOGKiookJiam2r4evUag0WmYpro7LUKvEXjcq0Of7vfn9ddf145NTU1VsjfffFM79t5771Wy0tJSL2cXerhXw0Tcq2EaN53mk4MAAAAAAACApVgcBAAAAAAAACzF4iAAAAAAAABgKRYHAQAAAAAAAEvVDPYEAAAAACAUFBcXK9nAgQO1Y6dPn65ko0aN0o6dOnWqku3du9e7yQEAECB8chAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABLsVsxAAAAAHig28FYRGTs2LGuMgAAQh2fHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABLhdzioOM4wZ4CLFDdPaPXCDQ6DdMEo2P0GoHGvRqm4V4NE3GvhmncdCzkFgdLSkqCPQVYoLp7Rq8RaHQapglGx+g1Ao17NUzDvRom4l4N07jpWIQTYsvUFRUVcvToUWnQoIGUlJRIy5YtJTc3V2JiYoI9Nb8pLi7muoLEcRwpKSmR5s2bS2Rk9a2N0+vwFerXRacDJ9R/76sq1K8rWJ0W+V+vHceRxMTEkP0eVVWo/95XVThcF/fqwAmH3/+qCPXr4l4dOKH+e19V4XBd3KsDJxx+/6si1K/Lm07XrKY5uRYZGSktWrQQEZGIiAgREYmJiQnJb7SvuK7giI2NrfavSa/DXyhfF50OLK6r+gWj0yL/63VxcbGIhPb3yBdcV3Bwrw4srqv6ca8OLK4rOLhXBxbXVf3cdjrk/rdiAAAAAAAAANWDxUEAAAAAAADAUiG9OBgdHS1TpkyR6OjoYE/Fr7guu5n6feK67GXq94jrspep3yOuy26mfp+4LnuZ+j3iuuxm6veJ6wp9IbchCQAAAAAAAIDqEdKfHAQAAAAAAAAQOCwOAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsFdKLg/Pnz5dWrVpJ7dq1pXPnzvLvf/872FPyyubNm+WOO+6Q5s2bS0REhKxateq8X3ccRx577DFp1qyZ1KlTR1JTU2X//v3BmaxLGRkZ0rFjR2nQoIE0bdpU0tLSJCcn57wxZ8+elfT0dGnUqJHUr19fBgwYIAUFBUGacWih06GJXldduHdaxMxe02nfhHuvTey0CL32Rbh3WsTMXtNp34R7r03stAi99kW4d1rEzF7b0umQXRxcvny5PPTQQzJlyhTZsWOHJCcnS+/eveXYsWPBnpprpaWlkpycLPPnz9f++syZM2XOnDmycOFCyc7Olnr16knv3r3l7Nmz1TxT9zZt2iTp6emyZcsWWbt2rXz77bfSq1cvKS0trRzz4IMPyttvvy1vvPGGbNq0SY4ePSr9+/cP4qxDA50OXfS6akzotIiZvabTVWdCr03stAi9rioTOi1iZq/pdNWZ0GsTOy1Cr6vKhE6LmNlrazrthKhOnTo56enplf997tw5p3nz5k5GRkYQZ1V1IuKsXLmy8r8rKiqchIQEZ9asWZVZYWGhEx0d7bz22mtBmGHVHDt2zBERZ9OmTY7jfH8NtWrVct54443KMZ999pkjIk5WVlawphkS6HT4oNfumNZpxzG313TaPdN6bWqnHYdeu2Vapx3H3F7TafdM67WpnXYceu2WaZ12HHN7bWqnQ/KTg+Xl5bJ9+3ZJTU2tzCIjIyU1NVWysrKCODP/OXjwoOTn5593jbGxsdK5c+ewusaioiIREWnYsKGIiGzfvl2+/fbb866rTZs2kpiYGFbX5W90OryukV7/PBs6LWJOr+m0Ozb02pROi9BrN2zotIg5vabT7tjQa1M6LUKv3bCh0yLm9NrUTofk4uCJEyfk3LlzEh8ff14eHx8v+fn5QZqVf/1wHeF8jRUVFTJ+/Hi54YYbpF27diLy/XVFRUVJXFzceWPD6boCgU6HzzXSa3ds6LSIGb2m0+7Z0GsTOi1Cr92yodMiZvSaTrtnQ69N6LQIvXbLhk6LmNFrkztdM9gTQPhKT0+X3bt3y0cffRTsqQB+Q69hGjoNE9FrmIZOw0T0GqYxudMh+cnBxo0bS40aNZTdXQoKCiQhISFIs/KvH64jXK9xzJgxsmbNGtmwYYO0aNGiMk9ISJDy8nIpLCw8b3y4XFeg0OnwuEZ67Z4NnRYJ/17Tae/Y0Otw77QIvfaGDZ0WCf9e02nv2NDrcO+0CL32hg2dFgn/Xpve6ZBcHIyKipKUlBRZt25dZVZRUSHr1q2TLl26BHFm/pOUlCQJCQnnXWNxcbFkZ2eH9DU6jiNjxoyRlStXyvr16yUpKem8X09JSZFatWqdd105OTly+PDhkL6uQKPToX2N9Np7NnRaJHx7TaerxoZeh2unReh1VdjQaZHw7TWdrhobeh2unRah11VhQ6dFwrfX1nQ6iJuhXNCyZcuc6OhoJzMz09m7d68zcuRIJy4uzsnPzw/21FwrKSlxdu7c6ezcudMREee5555zdu7c6Rw6dMhxHMeZMWOGExcX56xevdr5z3/+4/Tt29dJSkpyzpw5E+SZezZq1CgnNjbW2bhxo5OXl1d5nD59unLM73//eycxMdFZv369s23bNqdLly5Oly5dgjjr0ECnQxe9rhoTOu04ZvaaTledCb02sdOOQ6+ryoROO46ZvabTVWdCr03stOPQ66oyodOOY2avbel0yC4OOo7jzJ0710lMTHSioqKcTp06OVu2bAn2lLyyYcMGR0SUY9iwYY7jfL+V96OPPurEx8c70dHRTs+ePZ2cnJzgTvpn6K5HRJzFixdXjjlz5owzevRo56KLLnLq1q3r9OvXz8nLywvepEMInQ5N9Lrqwr3TjmNmr+m0b8K91yZ22nHotS/CvdOOY2av6bRvwr3XJnbacei1L8K9045jZq9t6XSE4ziOd581BAAAAAAAAGCCkHzmIAAAAAAAAIDAY3EQAAAAAAAAsBSLgwAAAAAAAIClWBwEAAAAAAAALMXiIAAAAAAAAGApFgcBAAAAAAAAS7E4CAAAAAAAAFiKxUEAAAAAAADAUiwOAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKX+H9kDRDMQW71hAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ " 5 0 4 1 9 2 1\n" ] } ], "source": [ "draw_examples(x_train[:7], captions=y_train)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "60000 przykładów uczących\n", "10000 przykładów testowych\n" ] } ], "source": [ "num_classes = 10\n", "\n", "x_train = x_train.reshape(60000, 784) # 784 = 28 * 28\n", "x_test = x_test.reshape(10000, 784)\n", "x_train = x_train.astype(\"float32\")\n", "x_test = x_test.astype(\"float32\")\n", "x_train /= 255\n", "x_test /= 255\n", "print(\"{} przykładów uczących\".format(x_train.shape[0]))\n", "print(\"{} przykładów testowych\".format(x_test.shape[0]))\n", "\n", "# przekonwertuj wektory klas na binarne macierze klas\n", "y_train = keras.utils.to_categorical(y_train, num_classes)\n", "y_test = keras.utils.to_categorical(y_test, num_classes)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "scrolled": true, "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-01-26 10:52:27.077963: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\n", "2023-01-26 10:52:27.078089: W tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:265] failed call to cuInit: UNKNOWN ERROR (303)\n", "2023-01-26 10:52:27.078807: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ELLIOT): /proc/driver/nvidia/version does not exist\n", "2023-01-26 10:52:27.095828: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA\n", "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " dense (Dense) (None, 512) 401920 \n", " \n", " dense_1 (Dense) (None, 512) 262656 \n", " \n", " dense_2 (Dense) (None, 10) 5130 \n", " \n", "=================================================================\n", "Total params: 669,706\n", "Trainable params: 669,706\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model = keras.Sequential()\n", "model.add(Dense(512, activation=\"relu\", input_shape=(784,)))\n", "model.add(Dropout(0.2))\n", "model.add(Dense(512, activation=\"relu\"))\n", "model.add(Dropout(0.2))\n", "model.add(Dense(num_classes, activation=\"softmax\"))\n", "model.summary()" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(60000, 784) (60000, 10)\n" ] } ], "source": [ "print(x_train.shape, y_train.shape)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-01-26 10:52:27.713204: W tensorflow/tsl/framework/cpu_allocator_impl.cc:82] Allocation of 188160000 exceeds 10% of free system memory.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/5\n", "469/469 [==============================] - 13s 25ms/step - loss: 0.2303 - accuracy: 0.9290 - val_loss: 0.1023 - val_accuracy: 0.9684\n", "Epoch 2/5\n", "469/469 [==============================] - 9s 20ms/step - loss: 0.0840 - accuracy: 0.9742 - val_loss: 0.0794 - val_accuracy: 0.9754\n", "Epoch 3/5\n", "469/469 [==============================] - 9s 20ms/step - loss: 0.0548 - accuracy: 0.9826 - val_loss: 0.0603 - val_accuracy: 0.9828\n", "Epoch 4/5\n", "469/469 [==============================] - 9s 20ms/step - loss: 0.0367 - accuracy: 0.9883 - val_loss: 0.0707 - val_accuracy: 0.9796\n", "Epoch 5/5\n", "469/469 [==============================] - 9s 19ms/step - loss: 0.0278 - accuracy: 0.9912 - val_loss: 0.0765 - val_accuracy: 0.9785\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.compile(\n", " loss=\"categorical_crossentropy\",\n", " optimizer=keras.optimizers.RMSprop(),\n", " metrics=[\"accuracy\"],\n", ")\n", "\n", "model.fit(\n", " x_train,\n", " y_train,\n", " batch_size=128,\n", " epochs=5,\n", " verbose=1,\n", " validation_data=(x_test, y_test),\n", ")" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Test loss: 0.07645954936742783\n", "Test accuracy: 0.9785000085830688\n" ] } ], "source": [ "score = model.evaluate(x_test, y_test, verbose=0)\n", "\n", "print(\"Test loss: {}\".format(score[0]))\n", "print(\"Test accuracy: {}\".format(score[1]))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "Warstwa _dropout_ to metoda regularyzacji, służy zapobieganiu nadmiernemu dopasowaniu sieci. Polega na tym, że część węzłów sieci jest usuwana w sposób losowy." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential_1\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " dense_3 (Dense) (None, 512) 401920 \n", " \n", " dense_4 (Dense) (None, 512) 262656 \n", " \n", " dense_5 (Dense) (None, 10) 5130 \n", " \n", "=================================================================\n", "Total params: 669,706\n", "Trainable params: 669,706\n", "Non-trainable params: 0\n", "_________________________________________________________________\n", "Epoch 1/5\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2023-01-26 10:53:20.710986: W tensorflow/tsl/framework/cpu_allocator_impl.cc:82] Allocation of 188160000 exceeds 10% of free system memory.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "469/469 [==============================] - 10s 19ms/step - loss: 0.2283 - accuracy: 0.9302 - val_loss: 0.0983 - val_accuracy: 0.9685\n", "Epoch 2/5\n", "469/469 [==============================] - 10s 22ms/step - loss: 0.0849 - accuracy: 0.9736 - val_loss: 0.0996 - val_accuracy: 0.9673\n", "Epoch 3/5\n", "469/469 [==============================] - 10s 22ms/step - loss: 0.0549 - accuracy: 0.9829 - val_loss: 0.0704 - val_accuracy: 0.9777\n", "Epoch 4/5\n", "469/469 [==============================] - 10s 21ms/step - loss: 0.0380 - accuracy: 0.9877 - val_loss: 0.0645 - val_accuracy: 0.9797\n", "Epoch 5/5\n", "469/469 [==============================] - 20s 43ms/step - loss: 0.0276 - accuracy: 0.9910 - val_loss: 0.0637 - val_accuracy: 0.9825\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Bez warstw Dropout\n", "\n", "num_classes = 10\n", "\n", "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n", "\n", "x_train = x_train.reshape(60000, 784) # 784 = 28 * 28\n", "x_test = x_test.reshape(10000, 784)\n", "x_train = x_train.astype(\"float32\")\n", "x_test = x_test.astype(\"float32\")\n", "x_train /= 255\n", "x_test /= 255\n", "\n", "y_train = keras.utils.to_categorical(y_train, num_classes)\n", "y_test = keras.utils.to_categorical(y_test, num_classes)\n", "\n", "model_no_dropout = keras.Sequential()\n", "model_no_dropout.add(Dense(512, activation=\"relu\", input_shape=(784,)))\n", "model_no_dropout.add(Dense(512, activation=\"relu\"))\n", "model_no_dropout.add(Dense(num_classes, activation=\"softmax\"))\n", "model_no_dropout.summary()\n", "\n", "model_no_dropout.compile(\n", " loss=\"categorical_crossentropy\",\n", " optimizer=keras.optimizers.RMSprop(),\n", " metrics=[\"accuracy\"],\n", ")\n", "\n", "model_no_dropout.fit(\n", " x_train,\n", " y_train,\n", " batch_size=128,\n", " epochs=5,\n", " verbose=1,\n", " validation_data=(x_test, y_test),\n", ")" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Test loss (no dropout): 0.06374581903219223\n", "Test accuracy (no dropout): 0.9825000166893005\n" ] } ], "source": [ "# Bez warstw Dropout\n", "\n", "score = model_no_dropout.evaluate(x_test, y_test, verbose=0)\n", "\n", "print(\"Test loss (no dropout): {}\".format(score[0]))\n", "print(\"Test accuracy (no dropout): {}\".format(score[1]))" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential_3\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " dense_6 (Dense) (None, 2500) 1962500 \n", " \n", " dense_7 (Dense) (None, 2000) 5002000 \n", " \n", " dense_8 (Dense) (None, 1500) 3001500 \n", " \n", " dense_9 (Dense) (None, 1000) 1501000 \n", " \n", " dense_10 (Dense) (None, 500) 500500 \n", " \n", " dense_11 (Dense) (None, 10) 5010 \n", " \n", "=================================================================\n", "Total params: 11,972,510\n", "Trainable params: 11,972,510\n", "Non-trainable params: 0\n", "_________________________________________________________________\n", "Epoch 1/10\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2023-01-26 11:06:02.193383: W tensorflow/tsl/framework/cpu_allocator_impl.cc:82] Allocation of 188160000 exceeds 10% of free system memory.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "469/469 [==============================] - 140s 294ms/step - loss: 0.6488 - accuracy: 0.8175 - val_loss: 0.2686 - val_accuracy: 0.9211\n", "Epoch 2/10\n", "469/469 [==============================] - 147s 313ms/step - loss: 0.2135 - accuracy: 0.9367 - val_loss: 0.2251 - val_accuracy: 0.9363\n", "Epoch 3/10\n", "469/469 [==============================] - 105s 224ms/step - loss: 0.1549 - accuracy: 0.9535 - val_loss: 0.1535 - val_accuracy: 0.9533\n", "Epoch 4/10\n", "469/469 [==============================] - 94s 200ms/step - loss: 0.1210 - accuracy: 0.9635 - val_loss: 0.1412 - val_accuracy: 0.9599\n", "Epoch 5/10\n", "469/469 [==============================] - 93s 199ms/step - loss: 0.0985 - accuracy: 0.9704 - val_loss: 0.1191 - val_accuracy: 0.9650\n", "Epoch 6/10\n", "469/469 [==============================] - 105s 224ms/step - loss: 0.0834 - accuracy: 0.9746 - val_loss: 0.0959 - val_accuracy: 0.9732\n", "Epoch 7/10\n", "469/469 [==============================] - 111s 236ms/step - loss: 0.0664 - accuracy: 0.9797 - val_loss: 0.1071 - val_accuracy: 0.9685\n", "Epoch 8/10\n", "469/469 [==============================] - 184s 392ms/step - loss: 0.0562 - accuracy: 0.9824 - val_loss: 0.0951 - val_accuracy: 0.9737\n", "Epoch 9/10\n", "469/469 [==============================] - 161s 344ms/step - loss: 0.0475 - accuracy: 0.9852 - val_loss: 0.1377 - val_accuracy: 0.9631\n", "Epoch 10/10\n", "469/469 [==============================] - 146s 311ms/step - loss: 0.0399 - accuracy: 0.9873 - val_loss: 0.1093 - val_accuracy: 0.9736\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Więcej warstw, inna funkcja aktywacji\n", "\n", "num_classes = 10\n", "\n", "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n", "\n", "x_train = x_train.reshape(60000, 784) # 784 = 28 * 28\n", "x_test = x_test.reshape(10000, 784)\n", "x_train = x_train.astype(\"float32\")\n", "x_test = x_test.astype(\"float32\")\n", "x_train /= 255\n", "x_test /= 255\n", "\n", "y_train = keras.utils.to_categorical(y_train, num_classes)\n", "y_test = keras.utils.to_categorical(y_test, num_classes)\n", "\n", "model3 = keras.Sequential()\n", "model3.add(Dense(2500, activation=\"tanh\", input_shape=(784,)))\n", "model3.add(Dense(2000, activation=\"tanh\"))\n", "model3.add(Dense(1500, activation=\"tanh\"))\n", "model3.add(Dense(1000, activation=\"tanh\"))\n", "model3.add(Dense(500, activation=\"tanh\"))\n", "model3.add(Dense(num_classes, activation=\"softmax\"))\n", "model3.summary()\n", "\n", "model3.compile(\n", " loss=\"categorical_crossentropy\",\n", " optimizer=keras.optimizers.RMSprop(),\n", " metrics=[\"accuracy\"],\n", ")\n", "\n", "model3.fit(\n", " x_train,\n", " y_train,\n", " batch_size=128,\n", " epochs=10,\n", " verbose=1,\n", " validation_data=(x_test, y_test),\n", ")" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Test loss: 0.10930903255939484\n", "Test accuracy: 0.9735999703407288\n" ] } ], "source": [ "# Więcej warstw, inna funkcja aktywacji\n", "\n", "score = model3.evaluate(x_test, y_test, verbose=0)\n", "\n", "print(\"Test loss: {}\".format(score[0]))\n", "print(\"Test accuracy: {}\".format(score[1]))" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Przykład: 4-pikselowy aparat fotograficzny\n", "\n", "https://www.youtube.com/watch?v=ILsA4nyG7I0" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "def generate_example(description):\n", " variant = random.choice([1, -1])\n", " if description == \"s\": # solid\n", " return (\n", " np.array([[1.0, 1.0], [1.0, 1.0]])\n", " if variant == 1\n", " else np.array([[-1.0, -1.0], [-1.0, -1.0]])\n", " )\n", " elif description == \"v\": # vertical\n", " return (\n", " np.array([[1.0, -1.0], [1.0, -1.0]])\n", " if variant == 1\n", " else np.array([[-1.0, 1.0], [-1.0, 1.0]])\n", " )\n", " elif description == \"d\": # diagonal\n", " return (\n", " np.array([[1.0, -1.0], [-1.0, 1.0]])\n", " if variant == 1\n", " else np.array([[-1.0, 1.0], [1.0, -1.0]])\n", " )\n", " elif description == \"h\": # horizontal\n", " return (\n", " np.array([[1.0, 1.0], [-1.0, -1.0]])\n", " if variant == 1\n", " else np.array([[-1.0, -1.0], [1.0, 1.0]])\n", " )\n", " else:\n", " return np.array(\n", " [\n", " [random.uniform(-1, 1), random.uniform(-1, 1)],\n", " [random.uniform(-1, 1), random.uniform(-1, 1)],\n", " ]\n", " )" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "import numpy as np\n", "import random\n", "\n", "num_classes = 4\n", "\n", "trainset_size = 4000\n", "testset_size = 1000\n", "\n", "y4_train = np.array([random.choice([\"s\", \"v\", \"d\", \"h\"]) for i in range(trainset_size)])\n", "x4_train = np.array([generate_example(desc) for desc in y4_train])\n", "\n", "y4_test = np.array([random.choice([\"s\", \"v\", \"d\", \"h\"]) for i in range(testset_size)])\n", "x4_test = np.array([generate_example(desc) for desc in y4_test])" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABRcAAADICAYAAAByFdYYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbGklEQVR4nO3df2xddf0/8Fe30c7FtHNja2nY4DMRjEM2GHapJmzEmolkiQbjRANz0QGGf3REZAScEOMi8gcJmZFg5lQw/IgMjBoQO36EOQYMGwi/ksniBq6d2+SWDe10Pd8/+LassM17D7295777eCSvhHt6zr3vnvvci3tfub2nIcuyLAAAAAAAKjSh1gsAAAAAAOqT4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJBL1YaL+/fvj69+9avR3NwcU6dOja9//etx4MCB4x6zePHiaGhoGFFXXHFFtZYIFZNrUiPTpEiuSY1MkyK5JjUyzXjWkGVZVo07vuCCC2L37t1x2223xX/+859YsWJFfOITn4hf//rXxzxm8eLFcfrpp8eNN944vG3KlCnR3NxcjSVCxeSa1Mg0KZJrUiPTpEiuSY1MM55NqsadvvTSS/Hggw/G008/Heeee25ERNx6663xuc99Lm6++eZob28/5rFTpkyJtra2aiwL3he5JjUyTYrkmtTINCmSa1Ij04x3VRkubtmyJaZOnTr8jyoioqurKyZMmBBbt26NL3zhC8c89s4774w77rgj2traYunSpXH99dfHlClTjrn/wMBADAwMDN8eHByM/fv3x/Tp06OhoWF0fiGIiO7u7mhpaYlzzjlneJtcU882bdoULS0t0dbWFoODgzFhwgSZpu7p1aRGryZFejWp0atJUZZl8eabb0Z7e3tMmHD8b1WsynCxt7c3Zs6cOfKBJk2KadOmRW9v7zGP+8pXvhKnnHJKtLe3x3PPPRff/e5345VXXon77rvvmMesXbs2brjhhlFbO/wvf//73+Pkk0+OCLkmDbNmzYpdu3bFySefLNMkQ68mNXo1KdKrSY1eTYqGMn08FX3n4jXXXBM/+tGPjrvPSy+9FPfdd1/84he/iFdeeWXEz2bOnBk33HBDfPOb3yzr8TZt2hSf/vSnY/v27fHhD3/4qPu8e2pfKpVi9uzZsWvXLt9TUKaWlpZaL6GuvPHGGyPO2VjmGqrlyFzr1RTRmjVr4pZbbjnuPk8//XTce++9cdNNN+nVdaJUKtV6CTVTbqZ/+9vfxp133hmvvvpqzXo1VIteXR/06luOu49eTere3auPpqJPLl511VXxta997bj7zJkzJ9ra2mLPnj0jtv/3v/+N/fv3V/RdAgsXLoyIOO4/rKampmhqanrP9ubmZm9YqYojP2o+1rmGahnKtV5NUV177bVx+eWXH3efOXPmxNatWyNCr64X4/nff7mZfu6552Lfvn0RUbteDdWiV9cHvVqvZnwr58/tKxouzpgxI2bMmPE/9+vs7Iw33ngjtm3bFgsWLIiItyfwg4ODw/9YytHT0xMRESeddFIly4Sq+stf/hKLFi2KCLkmPTJNUZX7GqSjoyMi9GqKr5LX1e/+1JBMkwq9mqLTq6FMWZV89rOfzc4+++xs69at2RNPPJF95CMfyS6++OLhn7/22mvZGWeckW3dujXLsizbvn17duONN2bPPPNMtmPHjuyBBx7I5syZk5133nkVPW6pVMoiIiuVSqP6+6QsIlQFddZZZ9Us10pVqzZt2qRXk4ShbOnV9VGUp6urq6a9WqlqlV5dH0V59GqVapXznq1qnWLfvn3ZxRdfnH3wgx/MmpubsxUrVmRvvvnm8M937NiRRUT2yCOPZFmWZTt37szOO++8bNq0aVlTU1N22mmnZd/5zncqfuPpDWvlah3UeqsvfvGLNcu1UtUqvZpUDGVLr66PojxDmZVplVrp1fVRlEevVqlWOZms6IIu9aC/vz9aWlqiVCqN6++GqITL1VemFtkayjVUy1jnWq+mWmqZLb26com9DK2aWuVapqk2vbo+6NXl0atJVTmZnjBGawEAAAAAEmO4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAOQyJsPFdevWxamnnhqTJ0+OhQsXxlNPPXXc/e+999746Ec/GpMnT46Pf/zj8Yc//GEslgllk2lSc/vtt8s0ydGrSY1eTYr0alKjVzMuZVV21113ZY2Njdn69euzF154IVu5cmU2derUrK+v76j7b968OZs4cWJ20003ZS+++GJ23XXXZSeccEL2/PPPl/V4pVIpi4isVCqN5q+RtIhQFdT69evHNNNZ9k6ulapW1SrTejWjbShbenV9FOUZypZMq9RKr66Pojx6tUq1ynnPVvVO0dHRkV155ZXDtw8fPpy1t7dna9euPer+X/rSl7ILL7xwxLaFCxdml19+eVmP5w1r5Wod1HqrBQsWjGmms8z/MFT1a+XKlTXJtF7NaBvKll5dH0V5hrJVq16tVLVKr66Pojx6tUq1ynnPVtU/iz506FBs27Yturq6hrdNmDAhurq6YsuWLUc9ZsuWLSP2j4hYsmTJMfcfGBiI/v7+EQXV1NPTU9VMR8g1Y2/x4sXD/y3TpECvJiWHDh2KCL2a9OjVpESvZjyr6nBx7969cfjw4WhtbR2xvbW1NXp7e496TG9vb0X7r127NlpaWoZr1qxZo7N4OIZqZzpCrhl7M2fOHHFbpql3ejUp2bdvX0To1aRHryYlejXjWd1fLXr16tVRKpWGa9euXbVeErxvck1qZJoUyTWpkWlSJNekRqYpoknVvPMTTzwxJk6cGH19fSO29/X1RVtb21GPaWtrq2j/pqamaGpqGp0FQxmqnekIuWbs7dmzZ8Rtmabe6dWkZPr06RGhV5MevZqU6NWMZ1X95GJjY2MsWLAguru7h7cNDg5Gd3d3dHZ2HvWYzs7OEftHRDz88MPH3B/G2vz582Wa5Dz22GPD/y3TpECvJiWNjY0RoVeTHr2alOjVjGtlX4Iop7vuuitramrKNmzYkL344ovZZZddlk2dOjXr7e3NsizLLrnkkuyaa64Z3n/z5s3ZpEmTsptvvjl76aWXsjVr1lR0KXZXIK1cFODqQ/VU69evH9NMZ5krgKnqV60yrVcz2oaypVfXR1GeoWzJtEqt9Or6KMqjV6tUq5z3bGPSKW699dZs9uzZWWNjY9bR0ZE9+eSTwz9btGhRtnz58hH733PPPdnpp5+eNTY2ZnPnzs1+//vfl/1Y3rBWrtZBrbcqlUpjmuks8z8MVf368Y9/XJNM69WMtiOzpVcXvyjPULZq1auVqlbp1fVRlEevVqlWOe/ZGrIsyyIh/f390dLSEqVSKZqbm2u9nLrQ0NBQ6yXUlVpkayjXUC1jnWu9mmqpZbb06sol9jK0amqVa5mm2vTq+qBXl0evJlXlZLrurxYNAAAAANSG4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQy5gMF9etWxennnpqTJ48ORYuXBhPPfXUMffdsGFDNDQ0jKjJkyePxTKhbDJNam6//XaZJjl6NanRq0mRXk1q9GrGo6oPF+++++5YtWpVrFmzJp599tmYN29eLFmyJPbs2XPMY5qbm2P37t3D9be//a3ay4Sy/eY3v5FpknPttdfKNEnRq0mRXk1q9GpSpFczLmVV1tHRkV155ZXDtw8fPpy1t7dna9euPer+P//5z7OWlpbcj1cqlbKIyEqlUu77GG8iQlVQCxYsGNNMZ9k7uVaqWrVy5cqaZFqvZrQNZUuvro+iPEPZqlWvVqpapVfXR1EevVqlWuW8Z5sUVXTo0KHYtm1brF69enjbhAkToqurK7Zs2XLM4w4cOBCnnHJKDA4OxjnnnBM//OEPY+7cuUfdd2BgIAYGBoZv9/f3R0RES0vLKP0W6cuyrNZLqAv9/f3R0tISPT09cd111w1vH+1MRxw716VSKZqbm0fht4G37d27N2bMmBGLFy8e3jaWmdarqRa9uj40NDTUegl1pVa9WqYZbV5X1xe9ujJ6NakY6tXlqOqfRe/duzcOHz4cra2tI7a3trZGb2/vUY8544wzYv369fHAAw/EHXfcEYODg/HJT34yXnvttaPuv3bt2mhpaRmuWbNmjfrvAUeqdqYj5Jqxs2/fvoiImDlz5ojtMk2906tJkV5NavRqUqRXMx4V7mrRnZ2dcemll8b8+fNj0aJFcd9998WMGTPitttuO+r+q1evjlKpNFy7du0a4xXD8VWa6Qi5pthkmhTJNamRaVIk16RGpklFVf8s+sQTT4yJEydGX1/fiO19fX3R1tZW1n2ccMIJcfbZZ8f27duP+vOmpqZoamp632uFclU70xFyzdiZPn16RMR7vmRapql3ejUp0qtJjV5NivRqxqOqfnKxsbExFixYEN3d3cPbBgcHo7u7Ozo7O8u6j8OHD8fzzz8fJ510UrWWCRWZP3++TJOMxsbGiIh47LHHhrfJNCnQq0mRXk1q9GpSpFczLr2vyxKV4a677sqampqyDRs2ZC+++GJ22WWXZVOnTs16e3uzLMuySy65JLvmmmuG97/hhhuyhx56KPvrX/+abdu2Lfvyl7+cTZ48OXvhhRfKejxXSqq8KM9QttavXz+mmT7ysV1Zl9E2lK1aZVqpapVeXR9qnZN6K5kmFV5X15da9756K5kmFZVkq6p/Fh0RsWzZsvjHP/4R3/ve96K3tzfmz58fDz744PAX9+7cuTMmTHjnA5T//Oc/Y+XKldHb2xsf+tCHYsGCBfHnP/85Pvaxj1V7qVCWiy66KA4ePCjTJOUHP/iBTJMUvZoU6dWkRq8mRXo141FDlmVZrRcxmiq5VDZvSywCVTOUrVKpFM3NzePmsUlbrbKlV1NtenV9aGhoqPUS6kqterVMM9q8rq4venVl9GpSUUm2Cne1aAAAAACgPhguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5VHW4+Pjjj8fSpUujvb09Ghoa4v777/+fxzz66KNxzjnnRFNTU5x22mmxYcOGai4RKrZ582a5JjnLli2TaZIi06RIrkmNTJMiuWY8qupw8eDBgzFv3rxYt25dWfvv2LEjLrzwwjj//POjp6cnvvWtb8U3vvGNeOihh6q5TKjIW2+9Jdck58wzz5RpkiLTpEiuSY1MkyK5ZlzKxkhEZBs3bjzuPldffXU2d+7cEduWLVuWLVmypOzHKZVKWUSoCoryDGWrVCoNbxvrXB/52DAa3p0tvVqlUmOd6SNzrVeXr9Y5qbeqVa+WaUZbrV5/HO2x+d9q3fvqrfRqUlFJtgr1nYtbtmyJrq6uEduWLFkSW7ZsOeYxAwMD0d/fP6KgSOSa1Mg0qcmT6Qi5ptj0alKjV5MivZpUFGq42NvbG62trSO2tba2Rn9/f/zrX/866jFr166NlpaW4Zo1a9ZYLBXKJtekRqZJTZ5MR8g1xaZXkxq9mhTp1aSiUMPFPFavXh2lUmm4du3aVeslwfsm16RGpkmRXJMamSZFck1qZJoimlTrBRypra0t+vr6Rmzr6+uL5ubm+MAHPnDUY5qamqKpqWkslge5yDWpkWlSkyfTEXJNsenVpEavJkV6Nako1CcXOzs7o7u7e8S2hx9+ODo7O2u0Inj/5JrUyDSpkWlSJNekRqZJkVyTiqoOFw8cOBA9PT3R09MTEW9fZr2npyd27twZEW9/nPfSSy8d3v+KK66IV199Na6++up4+eWX4yc/+Uncc8898e1vf7uay4SKyDUpeu6552SapMg0KZJrUiPTpEiuGZeqednqRx555KiXZl++fHmWZVm2fPnybNGiRe85Zv78+VljY2M2Z86c7Oc//3lFjzl0qWxVflGeoWz97ne/q1muy7kEPFTieD1Tr1apVTUzfWSu9ery1ToTKZTXH9SjWr3+OPKx5bp8te5zKZReTT2qJFsNWZZlkZD+/v5oaWmp9TLqSmIRqJqhbJVKpWhubh43j03aapUtvZpq06vrQ0NDQ62XUFdq1atlmtHmdXV90asro1eTikqyVajvXAQAAAAA6ofhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJBLVYeLjz/+eCxdujTa29ujoaEh7r///uPu/+ijj0ZDQ8N7qre3t5rLhIps3rxZrknOsmXLZJqkyDQpkmtSI9OkSK4Zj6o6XDx48GDMmzcv1q1bV9Fxr7zySuzevXu4Zs6cWaUVQuXeeustuSY5Z555pkyTFJkmRXJNamSaFMk149Gkat75BRdcEBdccEHFx82cOTOmTp06+guCUfCZz3wmLrroooqPk2uK7Prrr4/m5uaKjpFpikymSZFckxqZJkVyzXhU1eFiXvPnz4+BgYE488wz4/vf/3586lOfOua+AwMDMTAwMHy7VCqNxRKT0t/fX+sl1IWh85RlWa7jRyPXnitG2/vJtV5NkVU70xF6NWOvVr1aphltY/X6I0KuGXt6NamoqFdnYyQiso0bNx53n5dffjn76U9/mj3zzDPZ5s2bsxUrVmSTJk3Ktm3bdsxj1qxZk0WEUmNWu3btkmuVXA3lOkKmVRpV7UzLtapF6dUqtdKrVYqlV6vU6sgZyLE0/P/QV11DQ0Ns3LgxPv/5z1d03KJFi2L27Nnxq1/96qg/f/fUfnBwMPbv3x/Tp0+PhoaG97PkUdXf3x+zZs2KXbt2VfwR6fGoiOcry7J48803o729PSZMePvrSsdzrov4HBVZUc/Xu3M9njMdUdznqaiKeL7GKtMR9ZHrIj5HRVbU86VXj1TU56mIinqu9OqRivo8FVVRz5dePVJRn6ciKuq5OtoM5FgK+WfRR+ro6IgnnnjimD9vamqKpqamEduK/F0Fzc3NhQpL0RXtfLW0tIzK/aSU66I9R0VXxPM1GrlOKdMRxXyeiqxo52ssMh1RX7ku2nNUdEU8X3r1exXxeSqqIp4rvfq9ivg8FVkRz5de/V5FfJ6KqojnqtxMV/Vq0aOhp6cnTjrppFovA0aVXJMamSY1Mk2K5JrUyDQpkmvqUVU/uXjgwIHYvn378O0dO3ZET09PTJs2LWbPnh2rV6+O119/PX75y19GRMQtt9wS//d//xdz586Nf//73/Gzn/0sNm3aFH/84x+ruUyoiFyTGpkmNTJNiuSa1Mg0KZJrxquqDhefeeaZOP/884dvr1q1KiIili9fHhs2bIjdu3fHzp07h39+6NChuOqqq+L111+PKVOmxFlnnRV/+tOfRtxHvWpqaoo1a9a85+PLHF2Rz5dcv63Iz1ERFfl8yfQ7ivw8FVFRz5dMv6Ooz1FRFfl8yfU7ivw8FU2Rz5VMv6PIz1MRFfl8yfU7ivw8FU0K52rMLugCAAAAAKSl8N+5CAAAAAAUk+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+HiGFm3bl2ceuqpMXny5Fi4cGE89dRTtV5SIT3++OOxdOnSaG9vj4aGhrj//vtrvSSOQabLI9P1Q6bLJ9f1Q67LI9P1Q6bLJ9f1Q67LI9P1Q6bLl0quDRfHwN133x2rVq2KNWvWxLPPPhvz5s2LJUuWxJ49e2q9tMI5ePBgzJs3L9atW1frpXAcMl0+ma4PMl0Zua4Pcl0+ma4PMl0Zua4Pcl0+ma4PMl2ZZHKdUXUdHR3ZlVdeOXz78OHDWXt7e7Z27doarqr4IiLbuHFjrZfBUch0PjJdXDKdn1wXl1znI9PFJdP5yXVxyXU+Ml1cMp1fPefaJxer7NChQ7Ft27bo6uoa3jZhwoTo6uqKLVu21HBlkI9MkxqZJkVyTWpkmhTJNamR6fHLcLHK9u7dG4cPH47W1tYR21tbW6O3t7dGq4L8ZJrUyDQpkmtSI9OkSK5JjUyPX4aLAAAAAEAuhotVduKJJ8bEiROjr69vxPa+vr5oa2ur0aogP5kmNTJNiuSa1Mg0KZJrUiPT45fhYpU1NjbGggULoru7e3jb4OBgdHd3R2dnZw1XBvnINKmRaVIk16RGpkmRXJMamR6/JtV6AePBqlWrYvny5XHuuedGR0dH3HLLLXHw4MFYsWJFrZdWOAcOHIjt27cP396xY0f09PTEtGnTYvbs2TVcGUeS6fLJdH2Q6crIdX2Q6/LJdH2Q6crIdX2Q6/LJdH2Q6cokk+taX656vLj11luz2bNnZ42NjVlHR0f25JNP1npJhfTII49kEfGeWr58ea2XxrvIdHlkun7IdPnkun7IdXlkun7IdPnkun7IdXlkun7IdPlSyXVDlmVZleeXAAAAAECCfOciAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQy/8DtjV1ROlkKSAAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ " d h h d h d h\n" ] } ], "source": [ "draw_examples(x4_train[:7], captions=y4_train)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "x4_train = x4_train.reshape(trainset_size, 4)\n", "x4_test = x4_test.reshape(testset_size, 4)\n", "x4_train = x4_train.astype(\"float32\")\n", "x4_test = x4_test.astype(\"float32\")\n", "\n", "y4_train = np.array([{\"s\": 0, \"v\": 1, \"d\": 2, \"h\": 3}[desc] for desc in y4_train])\n", "y4_test = np.array([{\"s\": 0, \"v\": 1, \"d\": 2, \"h\": 3}[desc] for desc in y4_test])\n", "\n", "y4_train = keras.utils.to_categorical(y4_train, num_classes)\n", "y4_test = keras.utils.to_categorical(y4_test, num_classes)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential_4\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " dense_12 (Dense) (None, 4) 20 \n", " \n", " dense_13 (Dense) (None, 4) 20 \n", " \n", " dense_14 (Dense) (None, 8) 40 \n", " \n", " dense_15 (Dense) (None, 4) 36 \n", " \n", "=================================================================\n", "Total params: 116\n", "Trainable params: 116\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model4 = keras.Sequential()\n", "model4.add(Dense(4, activation=\"tanh\", input_shape=(4,)))\n", "model4.add(Dense(4, activation=\"tanh\"))\n", "model4.add(Dense(8, activation=\"relu\"))\n", "model4.add(Dense(num_classes, activation=\"softmax\"))\n", "model4.summary()" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "model4.layers[0].set_weights(\n", " [\n", " np.array(\n", " [\n", " [1.0, 0.0, 1.0, 0.0],\n", " [0.0, 1.0, 0.0, 1.0],\n", " [1.0, 0.0, -1.0, 0.0],\n", " [0.0, 1.0, 0.0, -1.0],\n", " ],\n", " dtype=np.float32,\n", " ),\n", " np.array([0.0, 0.0, 0.0, 0.0], dtype=np.float32),\n", " ]\n", ")\n", "model4.layers[1].set_weights(\n", " [\n", " np.array(\n", " [\n", " [1.0, -1.0, 0.0, 0.0],\n", " [1.0, 1.0, 0.0, 0.0],\n", " [0.0, 0.0, 1.0, -1.0],\n", " [0.0, 0.0, -1.0, -1.0],\n", " ],\n", " dtype=np.float32,\n", " ),\n", " np.array([0.0, 0.0, 0.0, 0.0], dtype=np.float32),\n", " ]\n", ")\n", "model4.layers[2].set_weights(\n", " [\n", " np.array(\n", " [\n", " [1.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],\n", " [0.0, 0.0, 1.0, -1.0, 0.0, 0.0, 0.0, 0.0],\n", " [0.0, 0.0, 0.0, 0.0, 1.0, -1.0, 0.0, 0.0],\n", " [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, -1.0],\n", " ],\n", " dtype=np.float32,\n", " ),\n", " np.array([0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], dtype=np.float32),\n", " ]\n", ")" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "model4.layers[3].set_weights(\n", " [\n", " np.array(\n", " [\n", " [1.0, 0.0, 0.0, 0.0],\n", " [1.0, 0.0, 0.0, 0.0],\n", " [0.0, 1.0, 0.0, 0.0],\n", " [0.0, 1.0, 0.0, 0.0],\n", " [0.0, 0.0, 1.0, 0.0],\n", " [0.0, 0.0, 1.0, 0.0],\n", " [0.0, 0.0, 0.0, 1.0],\n", " [0.0, 0.0, 0.0, 1.0],\n", " ],\n", " dtype=np.float32,\n", " ),\n", " np.array([0.0, 0.0, 0.0, 0.0], dtype=np.float32),\n", " ]\n", ")\n", "\n", "model4.compile(\n", " loss=\"categorical_crossentropy\",\n", " optimizer=keras.optimizers.Adagrad(),\n", " metrics=[\"accuracy\"],\n", ")" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[array([[ 1., 0., 1., 0.],\n", " [ 0., 1., 0., 1.],\n", " [ 1., 0., -1., 0.],\n", " [ 0., 1., 0., -1.]], dtype=float32), array([0., 0., 0., 0.], dtype=float32)]\n", "[array([[ 1., -1., 0., 0.],\n", " [ 1., 1., 0., 0.],\n", " [ 0., 0., 1., -1.],\n", " [ 0., 0., -1., -1.]], dtype=float32), array([0., 0., 0., 0.], dtype=float32)]\n", "[array([[ 1., -1., 0., 0., 0., 0., 0., 0.],\n", " [ 0., 0., 1., -1., 0., 0., 0., 0.],\n", " [ 0., 0., 0., 0., 1., -1., 0., 0.],\n", " [ 0., 0., 0., 0., 0., 0., 1., -1.]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)]\n", "[array([[1., 0., 0., 0.],\n", " [1., 0., 0., 0.],\n", " [0., 1., 0., 0.],\n", " [0., 1., 0., 0.],\n", " [0., 0., 1., 0.],\n", " [0., 0., 1., 0.],\n", " [0., 0., 0., 1.],\n", " [0., 0., 0., 1.]], dtype=float32), array([0., 0., 0., 0.], dtype=float32)]\n" ] } ], "source": [ "for layer in model4.layers:\n", " print(layer.get_weights())" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1/1 [==============================] - 1s 872ms/step\n" ] }, { "data": { "text/plain": [ "array([[0.17831734, 0.17831734, 0.17831734, 0.465048 ]], dtype=float32)" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model4.predict([np.array([[1.0, 1.0], [-1.0, -1.0]]).reshape(1, 4)])" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Test loss: 0.7656148672103882\n", "Test accuracy: 1.0\n" ] } ], "source": [ "score = model4.evaluate(x4_test, y4_test, verbose=0)\n", "\n", "print(\"Test loss: {}\".format(score[0]))\n", "print(\"Test accuracy: {}\".format(score[1]))" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential_5\"\n", "_________________________________________________________________\n", " Layer (type) Output Shape Param # \n", "=================================================================\n", " dense_16 (Dense) (None, 4) 20 \n", " \n", " dense_17 (Dense) (None, 4) 20 \n", " \n", " dense_18 (Dense) (None, 8) 40 \n", " \n", " dense_19 (Dense) (None, 4) 36 \n", " \n", "=================================================================\n", "Total params: 116\n", "Trainable params: 116\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model5 = keras.Sequential()\n", "model5.add(Dense(4, activation=\"tanh\", input_shape=(4,)))\n", "model5.add(Dense(4, activation=\"tanh\"))\n", "model5.add(Dense(8, activation=\"relu\"))\n", "model5.add(Dense(num_classes, activation=\"softmax\"))\n", "model5.compile(\n", " loss=\"categorical_crossentropy\",\n", " optimizer=keras.optimizers.RMSprop(),\n", " metrics=[\"accuracy\"],\n", ")\n", "model5.summary()" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "scrolled": true, "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/8\n", "125/125 [==============================] - 3s 8ms/step - loss: 1.3014 - accuracy: 0.4947 - val_loss: 1.1876 - val_accuracy: 0.6040\n", "Epoch 2/8\n", "125/125 [==============================] - 1s 6ms/step - loss: 1.0779 - accuracy: 0.7395 - val_loss: 0.9865 - val_accuracy: 0.8730\n", "Epoch 3/8\n", "125/125 [==============================] - 1s 4ms/step - loss: 0.8925 - accuracy: 0.8382 - val_loss: 0.8114 - val_accuracy: 0.7460\n", "Epoch 4/8\n", "125/125 [==============================] - 0s 4ms/step - loss: 0.7266 - accuracy: 0.8060 - val_loss: 0.6622 - val_accuracy: 0.8730\n", "Epoch 5/8\n", "125/125 [==============================] - 0s 4ms/step - loss: 0.5890 - accuracy: 0.8765 - val_loss: 0.5392 - val_accuracy: 0.8730\n", "Epoch 6/8\n", "125/125 [==============================] - 1s 4ms/step - loss: 0.4738 - accuracy: 0.8838 - val_loss: 0.4293 - val_accuracy: 0.8730\n", "Epoch 7/8\n", "125/125 [==============================] - 1s 5ms/step - loss: 0.3636 - accuracy: 0.9337 - val_loss: 0.3191 - val_accuracy: 1.0000\n", "Epoch 8/8\n", "125/125 [==============================] - 1s 5ms/step - loss: 0.2606 - accuracy: 1.0000 - val_loss: 0.2202 - val_accuracy: 1.0000\n" ] }, { "data": { "text/plain": [ "" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model5.fit(x4_train, y4_train, epochs=8, validation_data=(x4_test, y4_test))" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1/1 [==============================] - 0s 106ms/step\n" ] }, { "data": { "text/plain": [ "array([[1.5366691e-01, 4.4674356e-04, 4.7448810e-02, 7.9843748e-01]],\n", " dtype=float32)" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model5.predict([np.array([[1.0, 1.0], [-1.0, -1.0]]).reshape(1, 4)])" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Test loss: 0.22015966475009918\n", "Test accuracy: 1.0\n" ] } ], "source": [ "score = model5.evaluate(x4_test, y4_test, verbose=0)\n", "\n", "print(\"Test loss: {}\".format(score[0]))\n", "print(\"Test accuracy: {}\".format(score[1]))" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "slideshow": { "slide_type": "notes" } }, "outputs": [], "source": [ "import contextlib\n", "\n", "\n", "@contextlib.contextmanager\n", "def printoptions(*args, **kwargs):\n", " original = np.get_printoptions()\n", " np.set_printoptions(*args, **kwargs)\n", " try:\n", " yield\n", " finally:\n", " np.set_printoptions(**original)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[array([[-0.8, 0.1, -0.6, 0.1],\n", " [-0.9, -0.7, -1. , 0.6],\n", " [-0.3, 0.5, 0.5, 0.3],\n", " [ 0.4, 0.3, -0.9, -0.8]], dtype=float32), array([ 0., -0., 0., 0.], dtype=float32)]\n", "[array([[-1.1, 1.2, -0.6, -0.6],\n", " [-1.1, -0.2, -0.7, -1.3],\n", " [ 0.6, 0.9, 0.3, -1.3],\n", " [ 0.8, 0.3, 0.7, 0.4]], dtype=float32), array([ 0.3, 0.5, -0.4, 0.5], dtype=float32)]\n", "[array([[ 0.5, 0.4, -0.4, 0.3, 0.8, -1.4, -1.1, 0.8],\n", " [ 0.5, -1.3, 0.3, 0.4, -1.3, 0.2, 0.9, 0.7],\n", " [-0.2, -0.1, -0.5, -0.2, 1.2, -0.4, -0.4, 1.1],\n", " [-1.1, 0.4, 1.3, -1.1, 1. , -1.1, -0.8, 0.3]], dtype=float32), array([ 0.2, 0.2, 0.1, 0.1, 0.2, 0.1, -0.2, 0. ], dtype=float32)]\n", "[array([[ 0.7, 0.8, -1.5, -0.2],\n", " [ 0.7, -0.9, -1.2, 0.2],\n", " [-0.4, 1.1, -0.1, -1.6],\n", " [ 0.3, 0.8, -1.4, 0.4],\n", " [ 0.2, -1.4, -0.3, 0.5],\n", " [-0.2, -1.2, 0.6, 0.7],\n", " [-0.1, -1.5, 0.3, -0.1],\n", " [-1.4, 0.1, 1.2, -0. ]], dtype=float32), array([-0.2, 0.5, 0.5, -0.5], dtype=float32)]\n" ] } ], "source": [ "with printoptions(precision=1, suppress=True):\n", " for layer in model5.layers:\n", " print(layer.get_weights())" ] } ], "metadata": { "author": "Paweł Skórzewski", "celltoolbar": "Slideshow", "email": "pawel.skorzewski@amu.edu.pl", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "lang": "pl", "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" }, "livereveal": { "start_slideshow_at": "selected", "theme": "white" }, "subtitle": "10.Sieci neuronowe – propagacja wsteczna[wykład]", "title": "Uczenie maszynowe", "vscode": { "interpreter": { "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6" } }, "year": "2021" }, "nbformat": 4, "nbformat_minor": 4 }