2099 lines
84 KiB
Plaintext
2099 lines
84 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "slide"
|
||
}
|
||
},
|
||
"source": [
|
||
"# 11. Sieci neuronowe – propagacja wsteczna"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "notes"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"%matplotlib inline"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "slide"
|
||
}
|
||
},
|
||
"source": [
|
||
"## 11.1. Metoda propagacji wstecznej – wprowadzenie"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"<img src=\"nn1.png\" alt=\"Rys. 12.1. Wielowarstwowa sieć neuronowa\" style=\"height: 100%\"/>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Architektura sieci neuronowych\n",
|
||
"\n",
|
||
"* Budowa warstwowa, najczęściej sieci jednokierunkowe i gęste.\n",
|
||
"* Liczbę i rozmiar warstw dobiera się do każdego problemu.\n",
|
||
"* Rozmiary sieci określane poprzez liczbę neuronów lub parametrów."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### _Feedforward_\n",
|
||
"\n",
|
||
"Mając daną $n$-warstwową sieć neuronową oraz jej parametry $\\Theta^{(1)}, \\ldots, \\Theta^{(L)} $ oraz $\\beta^{(1)}, \\ldots, \\beta^{(L)} $, obliczamy:\n",
|
||
"\n",
|
||
"$$a^{(l)} = g^{(l)}\\left( a^{(l-1)} \\Theta^{(l)} + \\beta^{(l)} \\right). $$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"<img src=\"nn2.png\" alt=\"Rys. 12.2. Wielowarstwowa sieć neuronowa - feedforward\" style=\"height:100%\"/>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Funkcje $g^{(l)}$ to **funkcje aktywacji**.<br/>\n",
|
||
"Dla $i = 0$ przyjmujemy $a^{(0)} = x$ (wektor wierszowy cech) oraz $g^{(0)}(x) = x$ (identyczność)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Parametry $\\Theta$ to wagi na połączeniach miedzy neuronami dwóch warstw.<br/>\n",
|
||
"Rozmiar macierzy $\\Theta^{(l)}$, czyli macierzy wag na połączeniach warstw $a^{(l-1)}$ i $a^{(l)}$, to $\\dim(a^{(l-1)}) \\times \\dim(a^{(l)})$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Parametry $\\beta$ zastępują tutaj dodawanie kolumny z jedynkami do macierzy cech.<br/>Macierz $\\beta^{(l)}$ ma rozmiar równy liczbie neuronów w odpowiedniej warstwie, czyli $1 \\times \\dim(a^{(l)})$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* **Klasyfikacja**: dla ostatniej warstwy $L$ (o rozmiarze równym liczbie klas) przyjmuje się $g^{(L)}(x) = \\mathop{\\mathrm{softmax}}(x)$.\n",
|
||
"* **Regresja**: pojedynczy neuron wyjściowy; funkcją aktywacji może wtedy być np. funkcja identycznościowa."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Jak uczyć sieci neuronowe?"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* W poznanych do tej pory algorytmach (regresja liniowa, regresja logistyczna) do uczenia używaliśmy funkcji kosztu, jej gradientu oraz algorytmu gradientu prostego (GD/SGD)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Dla sieci neuronowych potrzebowalibyśmy również znaleźć gradient funkcji kosztu."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Sprowadza się to do bardziej ogólnego problemu:<br/>jak obliczyć gradient $\\nabla f(x)$ dla danej funkcji $f$ i wektora wejściowego $x$?"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Pochodna funkcji\n",
|
||
"\n",
|
||
"* **Pochodna** mierzy, jak szybko zmienia się wartość funkcji względem zmiany jej argumentów:\n",
|
||
"\n",
|
||
"$$ \\frac{d f(x)}{d x} = \\lim_{h \\to 0} \\frac{ f(x + h) - f(x) }{ h } $$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Pochodna cząstkowa i gradient\n",
|
||
"\n",
|
||
"* **Pochodna cząstkowa** mierzy, jak szybko zmienia się wartość funkcji względem zmiany jej *pojedynczego argumentu*."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* **Gradient** to wektor pochodnych cząstkowych:\n",
|
||
"\n",
|
||
"$$ \\nabla f = \\left( \\frac{\\partial f}{\\partial x_1}, \\ldots, \\frac{\\partial f}{\\partial x_n} \\right) $$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"#### Gradient – przykłady"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"$$ f(x_1, x_2) = x_1 + x_2 \\qquad \\to \\qquad \\frac{\\partial f}{\\partial x_1} = 1, \\quad \\frac{\\partial f}{\\partial x_2} = 1, \\quad \\nabla f = (1, 1) $$ "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"$$ f(x_1, x_2) = x_1 \\cdot x_2 \\qquad \\to \\qquad \\frac{\\partial f}{\\partial x_1} = x_2, \\quad \\frac{\\partial f}{\\partial x_2} = x_1, \\quad \\nabla f = (x_2, x_1) $$ "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Własności pochodnych cząstkowych\n",
|
||
"\n",
|
||
"Jeżeli $f(x, y, z) = (x + y) \\, z$ oraz $x + y = q$, to:\n",
|
||
"$$f = q z,\n",
|
||
"\\quad \\frac{\\partial f}{\\partial q} = z,\n",
|
||
"\\quad \\frac{\\partial f}{\\partial z} = q,\n",
|
||
"\\quad \\frac{\\partial q}{\\partial x} = 1,\n",
|
||
"\\quad \\frac{\\partial q}{\\partial y} = 1 $$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"#### Reguła łańcuchowa\n",
|
||
"\n",
|
||
"$$ \\frac{\\partial f}{\\partial x} = \\frac{\\partial f}{\\partial q} \\, \\frac{\\partial q}{\\partial x},\n",
|
||
"\\quad \\frac{\\partial f}{\\partial y} = \\frac{\\partial f}{\\partial q} \\, \\frac{\\partial q}{\\partial y} $$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Propagacja wsteczna – prosty przykład"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Dla ustalonego wejścia\n",
|
||
"x = -2\n",
|
||
"y = 5\n",
|
||
"z = -4"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"3 -12\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Krok w przód\n",
|
||
"q = x + y\n",
|
||
"f = q * z\n",
|
||
"print(q, f)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[-4, -4, 3]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Propagacja wsteczna dla f = q * z\n",
|
||
"# Oznaczmy symbolami `dfx`, `dfy`, `dfz`, `dfq` odpowiednio\n",
|
||
"# pochodne cząstkowe ∂f/∂x, ∂f/∂y, ∂f/∂z, ∂f/∂q\n",
|
||
"dfz = q\n",
|
||
"dfq = z\n",
|
||
"# Propagacja wsteczna dla q = x + y\n",
|
||
"dfx = 1 * dfq # z reguły łańcuchowej\n",
|
||
"dfy = 1 * dfq # z reguły łańcuchowej\n",
|
||
"print([dfx, dfy, dfz])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"<img src=\"exp1.png\" alt=\"Rys. 12.3. Propagacja wsteczna - przykład 1\" style=\"height:100%\"/>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Właśnie tak wygląda obliczanie pochodnych metodą propagacji wstecznej!"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Spróbujmy czegoś bardziej skomplikowanego:<br/>metodą propagacji wstecznej obliczmy pochodną funkcji sigmoidalnej."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Propagacja wsteczna – funkcja sigmoidalna"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"Funkcja sigmoidalna:\n",
|
||
"\n",
|
||
"$$f(\\theta,x) = \\frac{1}{1+e^{-(\\theta_0 x_0 + \\theta_1 x_1 + \\theta_2)}}$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"$$\n",
|
||
"\\begin{array}{lcl}\n",
|
||
"f(x) = \\frac{1}{x} \\quad & \\rightarrow & \\quad \\frac{df}{dx} = -\\frac{1}{x^2} \\\\\n",
|
||
"f_c(x) = c + x \\quad & \\rightarrow & \\quad \\frac{df}{dx} = 1 \\\\\n",
|
||
"f(x) = e^x \\quad & \\rightarrow & \\quad \\frac{df}{dx} = e^x \\\\\n",
|
||
"f_a(x) = ax \\quad & \\rightarrow & \\quad \\frac{df}{dx} = a \\\\\n",
|
||
"\\end{array}\n",
|
||
"$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"<img src=\"exp2.png\" alt=\"Rys. 12.4. Propagacja wsteczna - przykład 2\" style=\"height:100%\"/>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[0.3932238664829637, -0.5898357997244456]\n",
|
||
"[-0.19661193324148185, -0.3932238664829637, 0.19661193324148185]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from math import exp\n",
|
||
"\n",
|
||
"\n",
|
||
"# Losowe wagi i dane\n",
|
||
"w = [2, -3, -3]\n",
|
||
"x = [-1, -2]\n",
|
||
"\n",
|
||
"# Krok w przód\n",
|
||
"dot = w[0] * x[0] + w[1] * x[1] + w[2]\n",
|
||
"f = 1.0 / (1 + exp(-dot)) # funkcja sigmoidalna\n",
|
||
"\n",
|
||
"# Krok w tył\n",
|
||
"ddot = (1 - f) * f # pochodna funkcji sigmoidalnej\n",
|
||
"dx = [w[0] * ddot, w[1] * ddot]\n",
|
||
"dw = [x[0] * ddot, x[1] * ddot, 1.0 * ddot]\n",
|
||
"\n",
|
||
"print(dx)\n",
|
||
"print(dw)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Obliczanie gradientów – podsumowanie"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Gradient $f$ dla $x$ mówi, jak zmieni się całe wyrażenie przy zmianie wartości $x$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Gradienty łączymy, korzystając z **reguły łańcuchowej**."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* W kroku \"wstecz\" gradienty informują, które części grafu powinny być zwiększone lub zmniejszone (i z jaką siłą), aby zwiększyć wartość na wyjściu."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* W kontekście implementacji chcemy dzielić funkcję $f$ na części, dla których można łatwo obliczyć gradienty."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "slide"
|
||
}
|
||
},
|
||
"source": [
|
||
"## 11.2. Uczenie wielowarstwowych sieci neuronowych metodą propagacji wstecznej"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"Mając algorytm SGD oraz gradienty wszystkich wag, moglibyśmy trenować każdą sieć."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Niech $\\Theta = (\\Theta^{(1)},\\Theta^{(2)},\\Theta^{(3)},\\beta^{(1)},\\beta^{(2)},\\beta^{(3)})$\n",
|
||
"* Funkcja sieci neuronowej z grafiki:\n",
|
||
"$$\\small h_\\Theta(x) = \\tanh(\\tanh(\\tanh(x\\Theta^{(1)}+\\beta^{(1)})\\Theta^{(2)} + \\beta^{(2)})\\Theta^{(3)} + \\beta^{(3)})$$\n",
|
||
"* Funkcja kosztu dla regresji:\n",
|
||
"$$J(\\Theta) = \\dfrac{1}{2m} \\sum_{i=1}^{m} (h_\\Theta(x^{(i)})- y^{(i)})^2 $$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Jak obliczymy gradienty?\n",
|
||
"\n",
|
||
"$$\\nabla_{\\Theta^{(l)}} J(\\Theta) = ? \\quad \\nabla_{\\beta^{(l)}} J(\\Theta) = ?$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### W kierunku propagacji wstecznej\n",
|
||
"\n",
|
||
"* Pewna (niewielka) zmiana wagi $\\Delta z^l_j$ dla $j$-ego neuronu w warstwie $l$ pociąga za sobą (niewielką) zmianę kosztu: \n",
|
||
"\n",
|
||
"$$\\frac{\\partial J(\\Theta)}{\\partial z^{l}_j} \\Delta z^{l}_j$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Jeżeli $\\frac{\\partial J(\\Theta)}{\\partial z^{l}_j}$ jest duża, $\\Delta z^l_j$ ze znakiem przeciwnym zredukuje koszt.\n",
|
||
"* Jeżeli $\\frac{\\partial J(\\Theta)}{\\partial z^l_j}$ jest bliska zeru, koszt nie będzie mocno poprawiony."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Definiujemy błąd $\\delta^l_j$ neuronu $j$ w warstwie $l$: \n",
|
||
"\n",
|
||
"$$\\delta^l_j := \\dfrac{\\partial J(\\Theta)}{\\partial z^l_j}$$ \n",
|
||
"$$\\delta^l := \\nabla_{z^l} J(\\Theta) \\quad \\textrm{ (zapis wektorowy)} $$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Podstawowe równania propagacji wstecznej\n",
|
||
"\n",
|
||
"$$\n",
|
||
"\\begin{array}{rcll}\n",
|
||
"\\delta^L & = & \\nabla_{a^L}J(\\Theta) \\odot { \\left( g^{L} \\right) }^{\\prime} \\left( z^L \\right) & (BP1) \\\\[2mm]\n",
|
||
"\\delta^{l} & = & \\left( \\left( \\Theta^{l+1} \\right) \\! ^\\top \\, \\delta^{l+1} \\right) \\odot {{ \\left( g^{l} \\right) }^{\\prime}} \\left( z^{l} \\right) & (BP2)\\\\[2mm]\n",
|
||
"\\nabla_{\\beta^l} J(\\Theta) & = & \\delta^l & (BP3)\\\\[2mm]\n",
|
||
"\\nabla_{\\Theta^l} J(\\Theta) & = & a^{l-1} \\odot \\delta^l & (BP4)\\\\\n",
|
||
"\\end{array}\n",
|
||
"$$\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"#### (BP1)\n",
|
||
"$$ \\delta^L_j \\; = \\; \\frac{ \\partial J }{ \\partial a^L_j } \\, g' \\!\\! \\left( z^L_j \\right) $$\n",
|
||
"$$ \\delta^L \\; = \\; \\nabla_{a^L}J(\\Theta) \\odot { \\left( g^{L} \\right) }^{\\prime} \\left( z^L \\right) $$\n",
|
||
"Błąd w ostatniej warstwie jest iloczynem szybkości zmiany kosztu względem $j$-tego wyjścia i szybkości zmiany funkcji aktywacji w punkcie $z^L_j$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"#### (BP2)\n",
|
||
"$$ \\delta^{l} \\; = \\; \\left( \\left( \\Theta^{l+1} \\right) \\! ^\\top \\, \\delta^{l+1} \\right) \\odot {{ \\left( g^{l} \\right) }^{\\prime}} \\left( z^{l} \\right) $$\n",
|
||
"Aby obliczyć błąd w $l$-tej warstwie, należy przemnożyć błąd z następnej ($(l+1)$-szej) warstwy przez transponowany wektor wag, a uzyskaną macierz pomnożyć po współrzędnych przez szybkość zmiany funkcji aktywacji w punkcie $z^l$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"#### (BP3)\n",
|
||
"$$ \\nabla_{\\beta^l} J(\\Theta) \\; = \\; \\delta^l $$\n",
|
||
"Błąd w $l$-tej warstwie jest równy wartości gradientu funkcji kosztu."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"#### (BP4)\n",
|
||
"$$ \\nabla_{\\Theta^l} J(\\Theta) \\; = \\; a^{l-1} \\odot \\delta^l $$\n",
|
||
"Gradient funkcji kosztu względem wag $l$-tej warstwy można obliczyć jako iloczyn po współrzędnych $a^{l-1}$ przez $\\delta^l$."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Algorytm propagacji wstecznej"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"Dla pojedynczego przykładu $(x,y)$:\n",
|
||
"1. **Wejście**: Ustaw aktywacje w warstwie cech $a^{(0)}=x$ \n",
|
||
"2. **Feedforward:** dla $l=1,\\dots,L$ oblicz \n",
|
||
"$z^{(l)} = a^{(l-1)} \\Theta^{(l)} + \\beta^{(l)}$ oraz $a^{(l)}=g^{(l)} \\!\\! \\left( z^{(l)} \\right)$\n",
|
||
"3. **Błąd wyjścia $\\delta^{(L)}$:** oblicz wektor $$\\delta^{(L)}= \\nabla_{a^{(L)}}J(\\Theta) \\odot {g^{\\prime}}^{(L)} \\!\\! \\left( z^{(L)} \\right) $$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"4. **Propagacja wsteczna błędu:** dla $l = L-1,L-2,\\dots,1$ oblicz $$\\delta^{(l)} = \\delta^{(l+1)}(\\Theta^{(l+1)})^T \\odot {g^{\\prime}}^{(l)} \\!\\! \\left( z^{(l)} \\right) $$\n",
|
||
"5. **Gradienty:** \n",
|
||
" * $\\dfrac{\\partial}{\\partial \\Theta_{ij}^{(l)}} J(\\Theta) = a_i^{(l-1)}\\delta_j^{(l)} \\textrm{ oraz } \\dfrac{\\partial}{\\partial \\beta_{j}^{(l)}} J(\\Theta) = \\delta_j^{(l)}$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"W naszym przykładzie:\n",
|
||
"\n",
|
||
"$$\\small J(\\Theta) = \\frac{1}{2} \\left( a^{(L)} - y \\right) ^2 $$\n",
|
||
"$$\\small \\dfrac{\\partial}{\\partial a^{(L)}} J(\\Theta) = a^{(L)} - y$$\n",
|
||
"\n",
|
||
"$$\\small \\tanh^{\\prime}(x) = 1 - \\tanh^2(x)$$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"<img src=\"nn3.png\" alt=\"Rys. 12.5. Propagacja wsteczna - schemat\" style=\"height:100%\"/>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Algorytm SGD z propagacją wsteczną\n",
|
||
"\n",
|
||
"Pojedyncza iteracja:\n",
|
||
"1. Dla parametrów $\\Theta = (\\Theta^{(1)},\\ldots,\\Theta^{(L)})$ utwórz pomocnicze macierze zerowe $\\Delta = (\\Delta^{(1)},\\ldots,\\Delta^{(L)})$ o takich samych wymiarach (dla uproszczenia opuszczono wagi $\\beta$)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"2. Dla $m$ przykładów we wsadzie (*batch*), $i = 1,\\ldots,m$:\n",
|
||
" * Wykonaj algortym propagacji wstecznej dla przykładu $(x^{(i)}, y^{(i)})$ i przechowaj gradienty $\\nabla_{\\Theta}J^{(i)}(\\Theta)$ dla tego przykładu;\n",
|
||
" * $\\Delta := \\Delta + \\dfrac{1}{m}\\nabla_{\\Theta}J^{(i)}(\\Theta)$\n",
|
||
"3. Wykonaj aktualizację wag: $\\Theta := \\Theta - \\alpha \\Delta$"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Propagacja wsteczna – podsumowanie\n",
|
||
"\n",
|
||
"* Algorytm pierwszy raz wprowadzony w latach 70. XX w.\n",
|
||
"* W 1986 David Rumelhart, Geoffrey Hinton i Ronald Williams pokazali, że jest znacznie szybszy od wcześniejszych metod.\n",
|
||
"* Obecnie najpopularniejszy algorytm uczenia sieci neuronowych."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "slide"
|
||
}
|
||
},
|
||
"source": [
|
||
"## 11.3. Przykłady implementacji wielowarstwowych sieci neuronowych"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "notes"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Uwaga!\n",
|
||
"\n",
|
||
"Poniższe przykłady wykorzystują interfejs [Keras](https://keras.io), który jest częścią biblioteki [TensorFlow](https://www.tensorflow.org).\n",
|
||
"\n",
|
||
"Aby uruchomić TensorFlow w środowisku Jupyter, należy wykonać następujące czynności:\n",
|
||
"\n",
|
||
"#### Przed pierwszym uruchomieniem (wystarczy wykonać tylko raz)\n",
|
||
"\n",
|
||
"Instalacja biblioteki TensorFlow w środowisku Anaconda:\n",
|
||
"\n",
|
||
"1. Uruchom *Anaconda Navigator*\n",
|
||
"1. Wybierz kafelek *CMD.exe Prompt*\n",
|
||
"1. Kliknij przycisk *Launch*\n",
|
||
"1. Pojawi się konsola. Wpisz następujące polecenia, każde zatwierdzając wciśnięciem klawisza Enter:\n",
|
||
"```\n",
|
||
"conda create -n tf tensorflow\n",
|
||
"conda activate tf\n",
|
||
"conda install pandas matplotlib\n",
|
||
"jupyter notebook\n",
|
||
"```\n",
|
||
"\n",
|
||
"#### Przed każdym uruchomieniem\n",
|
||
"\n",
|
||
"Jeżeli chcemy korzystać z biblioteki TensorFlow, to środowisko Jupyter Notebook należy uruchomić w następujący sposób:\n",
|
||
"\n",
|
||
"1. Uruchom *Anaconda Navigator*\n",
|
||
"1. Wybierz kafelek *CMD.exe Prompt*\n",
|
||
"1. Kliknij przycisk *Launch*\n",
|
||
"1. Pojawi się konsola. Wpisz następujące polecenia, każde zatwierdzając wciśnięciem klawisza Enter:\n",
|
||
"```\n",
|
||
"conda activate tf\n",
|
||
"jupyter notebook\n",
|
||
"```"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Przykład: MNIST\n",
|
||
"\n",
|
||
"_Modified National Institute of Standards and Technology database_"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "fragment"
|
||
}
|
||
},
|
||
"source": [
|
||
"* Zbiór cyfr zapisanych pismem odręcznym\n",
|
||
"* 60 000 przykładów uczących, 10 000 przykładów testowych\n",
|
||
"* Rozdzielczość każdego przykładu: 28 × 28 = 784 piksele"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"2023-01-26 10:52:17.922141: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA\n",
|
||
"To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
|
||
"2023-01-26 10:52:18.163925: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n",
|
||
"2023-01-26 10:52:18.163996: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.\n",
|
||
"2023-01-26 10:52:19.577890: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n",
|
||
"2023-01-26 10:52:19.578662: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n",
|
||
"2023-01-26 10:52:19.578677: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
|
||
"11490434/11490434 [==============================] - 1s 0us/step\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from tensorflow import keras\n",
|
||
"from tensorflow.keras.datasets import mnist\n",
|
||
"from tensorflow.keras.layers import Dense, Dropout\n",
|
||
"\n",
|
||
"# załaduj dane i podziel je na zbiory uczący i testowy\n",
|
||
"(x_train, y_train), (x_test, y_test) = mnist.load_data()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "notes"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"from matplotlib import pyplot as plt\n",
|
||
"\n",
|
||
"\n",
|
||
"def draw_examples(examples, captions=None):\n",
|
||
" plt.figure(figsize=(16, 4))\n",
|
||
" m = len(examples)\n",
|
||
" for i, example in enumerate(examples):\n",
|
||
" plt.subplot(100 + m * 10 + i + 1)\n",
|
||
" plt.imshow(example, cmap=plt.get_cmap(\"gray\"))\n",
|
||
" plt.show()\n",
|
||
" if captions is not None:\n",
|
||
" print(6 * \" \" + (10 * \" \").join(str(captions[i]) for i in range(m)))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABQcAAADFCAYAAADpJUQuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAomElEQVR4nO3de3RU1d3G8V8CJFyTyC2BQiQqFRQJGgFRFqBE0KoQoKIUBKwFCwFEK6X4oqCIQVBbrmK1kKIoaBFQvFKuVUPK1S5AIlKEIEkAJRcCJErO+4fLVNx78ExmJjOz9/ez1vnDh30m+4Qnh2Eznh3hOI4jAAAAAAAAAKwTGewJAAAAAAAAAAgOFgcBAAAAAAAAS7E4CAAAAAAAAFiKxUEAAAAAAADAUiwOAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABL1QzUC8+fP19mzZol+fn5kpycLHPnzpVOnTr97HkVFRVy9OhRadCggURERARqerCU4zhSUlIizZs3l8hI79bGq9ppEXqNwPGl0yLcqxF6gtVpEXqNwOFeDdNwr4aJuFfDNF512gmAZcuWOVFRUc6iRYucPXv2OCNGjHDi4uKcgoKCnz03NzfXEREOjoAeubm51dZpes1RHYe3nfa113SaI9BHdXeaXnNUx8G9msO0g3s1h4kH92oO0w43nQ7I4mCnTp2c9PT0yv8+d+6c07x5cycjI+Nnzy0sLAz6N47D/KOwsLDaOk2vOarj8LbTvvaaTnME+qjuTtNrjuo4uFdzmHZwr+Yw8eBezWHa4abTfn/mYHl5uWzfvl1SU1Mrs8jISElNTZWsrCxlfFlZmRQXF1ceJSUl/p4SoPDm49redlqEXqP6efu/IHCvRqgLdKdF6DWqH/dqmIZ7NUzEvRqmcdNpvy8OnjhxQs6dOyfx8fHn5fHx8ZKfn6+Mz8jIkNjY2MqjZcuW/p4S4BNvOy1CrxH6uFfDNNyrYSLu1TAN92qYiHs1TBD03YonTZokRUVFlUdubm6wpwT4jF7DNHQaJqLXMA2dhonoNUxDpxGK/L5bcePGjaVGjRpSUFBwXl5QUCAJCQnK+OjoaImOjvb3NAC/8bbTIvQaoY97NUzDvRom4l4N03Cvhom4V8MEfv/kYFRUlKSkpMi6desqs4qKClm3bp106dLF318OCDg6DRPRa5iGTsNE9BqmodMwEb2GEbzehseFZcuWOdHR0U5mZqazd+9eZ+TIkU5cXJyTn5//s+cWFRUFfScXDvOPoqKiaus0veaojsPbTvvaazrNEeijujtNrzmq4+BezWHawb2aw8SDezWHaYebTgdkcdBxHGfu3LlOYmKiExUV5XTq1MnZsmWLq/P4weCojqMqN/yqdppec1THUZVO+9JrOs0R6KO6O02vOarj4F7NYdrBvZrDxIN7NYdph5tORziO40gIKS4ultjY2GBPA4YrKiqSmJiYavt69BqBRqdhmurutAi9RuBxr4ZpuFfDRNyrYRo3nQ76bsUAAAAAAAAAgoPFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUi4MAAAAAAACApVgcBAAAAAAAACzF4iAAAAAAAABgKRYHAQAAAAAAAEuxOAgAAAAAAABYisVBAAAAAAAAwFIsDgIAAAAAAACWqhnsCQCAWykpKdp8zJgxSjZ06FDt2CVLlijZ3LlztWN37NjhxewAAAAAAFUxe/ZsbT5u3Dgl2717t5Ldfvvt2vMPHTrk28QswScHAQAAAAAAAEuxOAgAAAAAAABYisVBAAAAAAAAwFIsDgIAAAAAAACWYkOSEFWjRg1tHhsb69Pr6jZuqFu3rnbs5ZdfrmTp6enasc8884ySDRo0SMnOnj2rPX/GjBlK9vjjj2vHwg4dOnRQsrVr12rHxsTEKJnjONqx99xzj5L16dNHO7ZRo0YXmCEQfnr27KlkS5cuVbLu3btrz8/JyfH7nACdyZMnK5mn9wWRkeq/dffo0UPJNm3a5PO8AMAUDRo00Ob169dXsttuu007tkmTJkr23HPPaceWlZV5MTuYrlWrVko2ZMgQ7diKigola9u2rZK1adNGez4bkrjDJwcBAAAAAAAAS7E4CAAAAAAAAFiKxUEAAAAAAADAUiwOAgAAAAAAAJZicRAAAAAAAACwFLsV+ygxMVGbR0VFKdn111+vHdu1a1cli4uL044dMGCA+8n56MiRI0o2Z84c7dh+/fopWUlJiZJ9+umn2vPZQdBunTp1UrIVK1YomafdunU7E+v6JyJSXl6uZJ52Jb7uuuuUbMeOHa5eE1XXrVs3JfP0e7Ry5cpAT8coHTt2VLKtW7cGYSbA94YPH67NJ06cqGS63Qo98bRjPQCYTrcLrO6e2qVLF+357dq18+nrN2vWTJuPGzfOp9eFWY4fP65kmzdv1o7t06dPoKcD4ZODAAAAAAAAgLVYHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABLsSGJFzp06KBk69ev1471tHFCKPL0gO/Jkycr2alTp7Rjly5dqmR5eXlKdvLkSe35OTk5F5oiwlDdunWV7JprrtGOfeWVV5TM08OM3dq/f782nzlzppItW7ZMO/bjjz9WMt3PRUZGhpezw4X06NFDyVq3bq0dy4YkepGR+n/7S0pKUrKLL75YySIiIvw+J0BH1z8Rkdq1a1fzTGCyzp07a/MhQ4YoWffu3ZXsyiuvdP21Hn74YW1+9OhRJdNtSiiif1+UnZ3teg4wT5s2bZRs/Pjx2rGDBw9Wsjp16iiZpz/rc3NzlczTRn9t27ZVsoEDB2rHLliwQMn27dunHQvzlZaWKtmhQ4eCMBP8gE8OAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsxYYkXjh8+LCSff3119qx1bkhie4BxYWFhdqxN954o5KVl5drx7788ss+zQt2e+GFF5Rs0KBB1fb1PW1+Ur9+fSXbtGmTdqxuY4z27dv7NC/8vKFDhypZVlZWEGYSvjxt6DNixAgl0z34ngeEIxBSU1OVbOzYsa7P99TL22+/XckKCgrcTwxGueuuu5Rs9uzZ2rGNGzdWMt0mDRs3btSe36RJEyWbNWvWz8zwwl/L0+vefffdrl8X4UH398Wnn35aO1bX6wYNGvj09T1t3te7d28lq1Wrlnas7r6s+7m6UA47xcXFKVlycnL1TwSV+OQgAAAAAAAAYCkWBwEAAAAAAABLsTgIAAAAAAAAWIrFQQAAAAAAAMBSLA4CAAAAAAAAlmK3Yi988803SjZhwgTtWN3OeTt37tSOnTNnjus57Nq1S8luvvlmJSstLdWef+WVVyrZAw884PrrAz+VkpKizW+77TYl87Qrn45uB+G3335bO/aZZ55RsqNHj2rH6n4OT548qR170003KZk314CqiYzk36189dJLL7ke62m3QqCqunbtqs0XL16sZLrdOj3xtAvsoUOHXL8GwlPNmupfWa699lrt2BdffFHJ6tatqx27efNmJZs2bZqSffTRR9rzo6Ojlez111/Xju3Vq5c219m2bZvrsQhf/fr1U7Lf/e53AflaBw4cUDLd3yFFRHJzc5Xssssu8/ucYDfdfTkxMdGn1+zYsaM21+2qzXsHFX8DAwAAAAAAACzF4iAAAAAAAABgKRYHAQAAAAAAAEuxOAgAAAAAAABYyusNSTZv3iyzZs2S7du3S15enqxcuVLS0tIqf91xHJkyZYq8+OKLUlhYKDfccIM8//zz0rp1a3/OO2SsWrVKm69fv17JSkpKtGOTk5OV7L777tOO1W284GnzEZ09e/Yo2ciRI12fbyI67V6HDh2UbO3atdqxMTExSuY4jnbse++9p2SDBg1Ssu7du2vPnzx5spJ52pDh+PHjSvbpp59qx1ZUVCiZbqOVa665Rnv+jh07tHmghUun27dvr83j4+OrdR4m8maTB08/w6EmXHoNkWHDhmnz5s2bu36NjRs3KtmSJUuqOqWQRKfdGzJkiJJ5s/GSp/vcXXfdpWTFxcWuX1d3vjcbjxw5ckSb//3vf3f9GqGGXrt35513+nT+l19+qc23bt2qZBMnTlQy3cYjnrRt29b1WNPQ6cDQbR6ZmZmpHTt16lRXr+lpXGFhoZLNmzfP1WvaxOtPDpaWlkpycrLMnz9f++szZ86UOXPmyMKFCyU7O1vq1asnvXv3lrNnz/o8WSAQ6DRMQ6dhInoN09BpmIhewzR0Grbw+pODt956q9x6663aX3McR/7yl7/I5MmTpW/fviLy/b/yxsfHy6pVq+Tuu+/2bbZAANBpmIZOw0T0Gqah0zARvYZp6DRs4ddnDh48eFDy8/MlNTW1MouNjZXOnTtLVlaW9pyysjIpLi4+7wBCRVU6LUKvEbroNExEr2EaOg0T0WuYhk7DJH5dHMzPzxcR9XlR8fHxlb/2UxkZGRIbG1t5tGzZ0p9TAnxSlU6L0GuELjoNE9FrmIZOw0T0Gqah0zBJ0HcrnjRpkhQVFVUe3jwYFQhV9BqmodMwEb2Gaeg0TESvYRo6jVDk9TMHLyQhIUFERAoKCqRZs2aVeUFBgXaXUxGR6OhoiY6O9uc0QoI3Hw0uKipyPXbEiBFKtnz5ciXT7bIK71Wl0yLh3+tf/vKX2nzChAlK5mlX1BMnTihZXl6edqxuV75Tp04p2TvvvKM931MeCHXq1FGyP/zhD9qxgwcPDvR0vBZKnf7Vr36lzXXfY3im2905KSnJ9flfffWVP6cTFKHUa9s0btxYyX77299qx+rem+h2EBQRefLJJ32aV7iztdPTpk3T5o888oiSOY6jHbtgwQIlmzx5snasr/8r3//93//5dP64ceO0+fHjx3163VBla6890f29buTIkdqxH374oZJ98cUX2rHHjh3zbWIauvcaoNP+5unPALe7FcM3fv3kYFJSkiQkJMi6desqs+LiYsnOzpYuXbr480sB1YJOwzR0Giai1zANnYaJ6DVMQ6dhEq8/OXjq1Knz/pXi4MGDsmvXLmnYsKEkJibK+PHj5cknn5TWrVtLUlKSPProo9K8eXNJS0vz57wBv6HTMA2dhonoNUxDp2Eieg3T0GnYwuvFwW3btsmNN95Y+d8PPfSQiIgMGzZMMjMz5Y9//KOUlpbKyJEjpbCwULp27Srvv/++1K5d23+zBvyITsM0dBomotcwDZ2Gieg1TEOnYQuvFwd79Ojh8RkfIiIRERHyxBNPyBNPPOHTxIDqQqdhGjoNE9FrmIZOw0T0Gqah07CFXzckQdXoHrCZkpKiHdu9e3clS01NVTLdQ2sBHd3DcJ955hntWN0GEiUlJdqxQ4cOVbJt27Zpx4b7BhSJiYnBnkJYuvzyy12P3bNnTwBnEt50P6+eHhz++eefK5mnn2Hgp1q1aqVkK1as8Ok1586dq803bNjg0+si9D322GNKptt4RESkvLxcyT744APt2IkTJyrZmTNnXM9L92mfXr16acfq/vyPiIjQjtVtsrN69WrX84J5jh49qmShuvECz89DMEVGqltlsAGr//l1QxIAAAAAAAAA4YPFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUi4MAAAAAAACApditOASUlpYq2YgRI7Rjd+zYoWQvvviiknna5U+3W+z8+fO1Yy+0ZTvMcfXVVyuZbldiT/r27avNN23aVOU5AT+1devWYE8hIGJiYrT5LbfcomRDhgzRjvW0i6bOtGnTlKywsND1+bCbrpft27d3ff66deuUbPbs2T7NCeEhLi5OyUaPHq1knt576nYmTktL83VactlllynZ0qVLlSwlJcX1a/7jH//Q5jNnznQ/McAH48aNU7J69er59JpXXXWV67GffPKJNs/KyvJpDrCXbmdi1ir8j08OAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsxYYkIerAgQPafPjw4Uq2ePFiJbvnnnu05+tyTw+oXbJkiZLl5eVpxyJ8Pffcc0oWERGhHavbZMTkjUciI9V/P9E9EBeB17Bhw4C8bnJyspJ56n9qaqqStWjRQjs2KipKyQYPHqxkuo6JiJw5c0bJsrOztWPLysqUrGZN/R/v27dv1+bAj3na6GHGjBmuzv/oo4+0+bBhw5SsqKjI9bwQvnT3xMaNG7s+X7fBQtOmTbVj7733XiXr06ePdmy7du2UrH79+krm6cH3uvyVV17RjtVtQAj8VN26dbX5FVdcoWRTpkzRjnW7saCn9yDevNc9evSokul+BkVEzp075/p1AVQ/PjkIAAAAAAAAWIrFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUG5KEmZUrVyrZ/v37lUy3yYSISM+ePZXsqaee0o69+OKLlWz69OnasV999ZU2R2i5/fbblaxDhw5K5unB22+99Za/pxTSdA9k1n1vdu3aVQ2zMY9u0w0R/fd44cKF2rGPPPKIT3No3769knnakOS7775TstOnT2vH7t27V8kWLVqkZNu2bdOer9vop6CgQDv2yJEjSlanTh3t2H379mlz2KtVq1ZKtmLFCp9e87///a8299RhmK+8vFzJjh8/rmRNmjTRnn/w4EEl8/RexRu6zRSKi4uVrFmzZtrzT5w4oWRvv/22z/OCWWrVqqXNr776aiXzdP/VddDT+yhdr7OyspTslltu0Z7vaVMUHd0GaP3799eOnT17tpLp7g0AgoNPDgIAAAAAAACWYnEQAAAAAAAAsBSLgwAAAAAAAIClWBwEAAAAAAAALMXiIAAAAAAAAGApdis2wO7du5Vs4MCB2rF33HGHki1evFg79v7771ey1q1ba8fefPPNF5oiQoRuB9OoqCglO3bsmPb85cuX+31O1S06OlrJpk6d6vr89evXK9mkSZN8mZK1Ro8erc0PHTqkZNdff31A5nD48GElW7VqlXbsZ599pmRbtmzx95Q8GjlypDbX7e7pabdY4KcmTpyoZLqd2r0xY8YMn86HeQoLC5UsLS1NydasWaM9v2HDhkp24MAB7djVq1crWWZmpnbsN998o2TLli1TMk+7FevGwm6699WedgV+8803Xb/u448/rmS696QiIh9//LGS6X6GPJ3frl071/PSvQfJyMjQjnX7nqusrMz114cdIiPVz7R5816lW7duSjZv3jyf5mQiPjkIAAAAAAAAWIrFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUG5IYSvfgZxGRl19+Wcleeukl7diaNdV66B7mKSLSo0cPJdu4caPH+SG0eXoQcF5eXjXPpOp0G4+IiEyePFnJJkyYoB175MgRJXv22WeV7NSpU17ODhfy9NNPB3sKIalnz56ux65YsSKAM0E46tChgzbv1auXT6+r2/whJyfHp9eEHbKzs5VMt7lBIOne13bv3l3JPD34ns2f7FarVi0l020c4ul9ps57772nzefOnatknv6+p/s5evfdd5Xsqquu0p5fXl6uZDNnztSO1W1e0rdvX+3YpUuXKtk///lPJfP0PvDkyZPaXGfXrl2uxyL06e7BjuO4Pr9///5KdsUVV2jH7t271/3EDMMnBwEAAAAAAABLsTgIAAAAAAAAWIrFQQAAAAAAAMBSLA4CAAAAAAAAlmJxEAAAAAAAALAUuxUboH379kr261//Wju2Y8eOSqbbldgTT7v3bN682fVrIPS99dZbwZ6CV3S7cHraGe6uu+5SMt1umyIiAwYM8GleQLCsXLky2FNAiPnwww+1+UUXXeT6NbZs2aJkw4cPr+qUgKCrU6eOknmzK+ayZcv8PieEnho1amjzadOmKdnDDz+sZKWlpdrz//SnPymZp07pdia+9tprtWPnzZunZFdffbWS7d+/X3v+qFGjlGzDhg3asTExMUp2/fXXa8cOHjxYyfr06aNka9eu1Z6vk5ubq82TkpJcvwZC38KFC5Xs/vvv9+k1R44cqc3Hjx/v0+uGMz45CAAAAAAAAFiKxUEAAAAAAADAUiwOAgAAAAAAAJZicRAAAAAAAACwFBuShKjLL79cm48ZM0bJ+vfvr2QJCQk+z+HcuXNKlpeXpx2re3gzQk9ERISrLC0tTXv+Aw884O8peeXBBx/U5o8++qiSxcbGascuXbpUyYYOHerbxAAgxDVq1Eibe/Pn94IFC5Ts1KlTVZ4TEGwffPBBsKeAMOBp4wLd5iOnT59WMk8bJ+g2irruuuu0Y++9914lu/XWW7VjdRvtPPHEE0q2ePFi7fmeNvnQKS4uVrL3339fO1aXDxo0SMl+85vfuP76nv5uALPs27cv2FOwAp8cBAAAAAAAACzF4iAAAAAAAABgKRYHAQAAAAAAAEuxOAgAAAAAAABYyqvFwYyMDOnYsaM0aNBAmjZtKmlpaZKTk3PemLNnz0p6ero0atRI6tevLwMGDJCCggK/ThrwJ3oN09BpmIhewzR0Gqah0zARvYYtIhzHcdwOvuWWW+Tuu++Wjh07ynfffSePPPKI7N69W/bu3Sv16tUTEZFRo0bJO++8I5mZmRIbGytjxoyRyMhI+fjjj119jeLiYo+7jIY7TzsI63Zp0u1KLCLSqlUrf05JRES2bdumzadPn65kb731lt+/fjAUFRVJTEyMiNjV6zvvvFPJXnvtNSXT7VQtIvLCCy8o2aJFi7Rjv/76ayXztAPbPffco2TJyclK1qJFC+35hw8fVrItW7Zox86ePdv12HBia6dtsnz5cm0+cOBAJRs2bJh27JIlS/w6p0D6cadF6LU3dDtQDh8+XDvWm92KL7nkEiU7dOiQ6/PBvTrU9O7dW8neffddJfP016VmzZop2fHjx32fWBip7k6LVH+v8/LytHmTJk2UrKysTMk87bT6w/fkxy677DIvZ6eaOnWqkmVkZCiZp/f74F4dDj7//HMlu/TSS12fHxmp/5yc7mfwwIED7icWon76vlqnpjcv+NPtxzMzM6Vp06ayfft26datmxQVFcnf/vY3efXVV+Wmm24Ske/foLZt21a2bNnicWEACCZ6DdPQaZiIXsM0dBqmodMwEb2GLXx65mBRUZGIiDRs2FBERLZv3y7ffvutpKamVo5p06aNJCYmSlZWlvY1ysrKpLi4+LwDCCZ6DdPQaZiIXsM0dBqm8UenReg1Qgv3apiqyouDFRUVMn78eLnhhhukXbt2IiKSn58vUVFREhcXd97Y+Ph4yc/P175ORkaGxMbGVh4tW7as6pQAn9FrmIZOw0T0Gqah0zCNvzotQq8ROrhXw2RVXhxMT0+X3bt3y7Jly3yawKRJk6SoqKjyyM3N9en1AF/Qa5iGTsNE9BqmodMwjb86LUKvETq4V8NkXj1z8AdjxoyRNWvWyObNm8/bICAhIUHKy8ulsLDwvJXzgoICj5txREdHS3R0dFWmERLi4+O1+RVXXKFk8+bN045t06aNX+ckIpKdna3NZ82apWSrV6/WjvXmIeUmoNf/U6NGDW0+evRoJRswYIB2rO7j8a1bt/ZpXp988ok237Bhg5I99thjPn0tE9BpO+gelO/pIcsmoNf/06FDB23+4/+16Qee/kwvLy9Xsvnz52vHsvNiYNDp4NFtsgPf+bPTIsHvtadPf+k2JNHNU7fJnie6DXFERDZv3qxkq1at0o798ssvlYzNR3zHvTq07NmzR8m8uafbttbhhld/e3AcR8aMGSMrV66U9evXS1JS0nm/npKSIrVq1ZJ169ZVZjk5OXL48GHp0qWLf2YM+Bm9hmnoNExEr2EaOg3T0GmYiF7DFl59cjA9PV1effVVWb16tTRo0KDyX1FiY2OlTp06EhsbK/fdd5889NBD0rBhQ4mJiZGxY8dKly5d2KUHIYtewzR0Giai1zANnYZp6DRMRK9hC68WB59//nkREenRo8d5+eLFi2X48OEiIvLnP/9ZIiMjZcCAAVJWVia9e/eWBQsW+GWyQCDQa5iGTsNE9BqmodMwDZ2Gieg1bOHV4qDuuUY/Vbt2bZk/f77H59UAoYZewzR0Giai1zANnYZp6DRMRK9hiyptSGKDhg0bKtkLL7ygZJ4eBh6oBxzrNmR49tlnleyDDz7Qnn/mzBm/zwnhIysrS8m2bt2qZB07dnT9mp4etOtpsx6dr7/+Wsl0u4A98MADrl8TsJmnZ9xkZmZW70QQUD9+8PmPXejB/j/11VdfKdnDDz9c1SkBYeVf//qXkuk2dOLB9Xbr1q2bNk9LS1Oya665RsmOHTumPX/RokVKdvLkSe1Y3eZRgM3++te/Ktkdd9wRhJmYw9ztDAEAAAAAAABcEIuDAAAAAAAAgKVYHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABLWbVbcefOnZVswoQJ2rGdOnVSsl/84hd+n5OIyOnTp5Vszpw52rFPPfWUkpWWlvp9TjDTkSNHlKx///5Kdv/992vPnzx5sk9ff/bs2dr8+eefV7IvvvjCp68F2CIiIiLYUwCAsLR7924l279/v5Jdcskl2vMvvfRSJTt+/LjvE0NIKSkp0eYvv/yyqwyA/+3du1fJPvvsM+3Ytm3bBno6RuCTgwAAAAAAAIClWBwEAAAAAAAALMXiIAAAAAAAAGApFgcBAAAAAAAAS1m1IUm/fv1cZd7QPQhTRGTNmjVK9t1332nHPvvss0pWWFjo07wAt/Ly8pRs6tSp2rGecgCB995772nzO++8s5pnglCxb98+bf7JJ58oWdeuXQM9HcAIus3/XnrpJe3Y6dOnK9nYsWO1Yz39nQEA4L1Dhw4p2VVXXRWEmZiDTw4CAAAAAAAAlmJxEAAAAAAAALAUi4MAAAAAAACApVgcBAAAAAAAACzF4iAAAAAAAABgqQjHcZxgT+LHiouLJTY2NtjTgOGKiookJiam2r4evUag0WmYpro7LUKvEXjcq0Of7vfn9ddf145NTU1VsjfffFM79t5771Wy0tJSL2cXerhXw0Tcq2EaN53mk4MAAAAAAACApVgcBAAAAAAAACzF4iAAAAAAAABgKRYHAQAAAAAAAEvVDPYEAAAAACAUFBcXK9nAgQO1Y6dPn65ko0aN0o6dOnWqku3du9e7yQEAECB8chAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABLsVsxAAAAAHig28FYRGTs2LGuMgAAQh2fHAQAAAAAAAAsxeIgAAAAAAAAYCkWBwEAAAAAAABLhdzioOM4wZ4CLFDdPaPXCDQ6DdMEo2P0GoHGvRqm4V4NE3GvhmncdCzkFgdLSkqCPQVYoLp7Rq8RaHQapglGx+g1Ao17NUzDvRom4l4N07jpWIQTYsvUFRUVcvToUWnQoIGUlJRIy5YtJTc3V2JiYoI9Nb8pLi7muoLEcRwpKSmR5s2bS2Rk9a2N0+vwFerXRacDJ9R/76sq1K8rWJ0W+V+vHceRxMTEkP0eVVWo/95XVThcF/fqwAmH3/+qCPXr4l4dOKH+e19V4XBd3KsDJxx+/6si1K/Lm07XrKY5uRYZGSktWrQQEZGIiAgREYmJiQnJb7SvuK7giI2NrfavSa/DXyhfF50OLK6r+gWj0yL/63VxcbGIhPb3yBdcV3Bwrw4srqv6ca8OLK4rOLhXBxbXVf3cdjrk/rdiAAAAAAAAANWDxUEAAAAAAADAUiG9OBgdHS1TpkyR6OjoYE/Fr7guu5n6feK67GXq94jrspep3yOuy26mfp+4LnuZ+j3iuuxm6veJ6wp9IbchCQAAAAAAAIDqEdKfHAQAAAAAAAAQOCwOAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKVYHAQAAAAAAAAsFdKLg/Pnz5dWrVpJ7dq1pXPnzvLvf/872FPyyubNm+WOO+6Q5s2bS0REhKxateq8X3ccRx577DFp1qyZ1KlTR1JTU2X//v3BmaxLGRkZ0rFjR2nQoIE0bdpU0tLSJCcn57wxZ8+elfT0dGnUqJHUr19fBgwYIAUFBUGacWih06GJXldduHdaxMxe02nfhHuvTey0CL32Rbh3WsTMXtNp34R7r03stAi99kW4d1rEzF7b0umQXRxcvny5PPTQQzJlyhTZsWOHJCcnS+/eveXYsWPBnpprpaWlkpycLPPnz9f++syZM2XOnDmycOFCyc7Olnr16knv3r3l7Nmz1TxT9zZt2iTp6emyZcsWWbt2rXz77bfSq1cvKS0trRzz4IMPyttvvy1vvPGGbNq0SY4ePSr9+/cP4qxDA50OXfS6akzotIiZvabTVWdCr03stAi9rioTOi1iZq/pdNWZ0GsTOy1Cr6vKhE6LmNlrazrthKhOnTo56enplf997tw5p3nz5k5GRkYQZ1V1IuKsXLmy8r8rKiqchIQEZ9asWZVZYWGhEx0d7bz22mtBmGHVHDt2zBERZ9OmTY7jfH8NtWrVct54443KMZ999pkjIk5WVlawphkS6HT4oNfumNZpxzG313TaPdN6bWqnHYdeu2Vapx3H3F7TafdM67WpnXYceu2WaZ12HHN7bWqnQ/KTg+Xl5bJ9+3ZJTU2tzCIjIyU1NVWysrKCODP/OXjwoOTn5593jbGxsdK5c+ewusaioiIREWnYsKGIiGzfvl2+/fbb866rTZs2kpiYGFbX5W90OryukV7/PBs6LWJOr+m0Ozb02pROi9BrN2zotIg5vabT7tjQa1M6LUKv3bCh0yLm9NrUTofk4uCJEyfk3LlzEh8ff14eHx8v+fn5QZqVf/1wHeF8jRUVFTJ+/Hi54YYbpF27diLy/XVFRUVJXFzceWPD6boCgU6HzzXSa3ds6LSIGb2m0+7Z0GsTOi1Cr92yodMiZvSaTrtnQ69N6LQIvXbLhk6LmNFrkztdM9gTQPhKT0+X3bt3y0cffRTsqQB+Q69hGjoNE9FrmIZOw0T0GqYxudMh+cnBxo0bS40aNZTdXQoKCiQhISFIs/KvH64jXK9xzJgxsmbNGtmwYYO0aNGiMk9ISJDy8nIpLCw8b3y4XFeg0OnwuEZ67Z4NnRYJ/17Tae/Y0Otw77QIvfaGDZ0WCf9e02nv2NDrcO+0CL32hg2dFgn/Xpve6ZBcHIyKipKUlBRZt25dZVZRUSHr1q2TLl26BHFm/pOUlCQJCQnnXWNxcbFkZ2eH9DU6jiNjxoyRlStXyvr16yUpKem8X09JSZFatWqdd105OTly+PDhkL6uQKPToX2N9Np7NnRaJHx7TaerxoZeh2unReh1VdjQaZHw7TWdrhobeh2unRah11VhQ6dFwrfX1nQ6iJuhXNCyZcuc6OhoJzMz09m7d68zcuRIJy4uzsnPzw/21FwrKSlxdu7c6ezcudMREee5555zdu7c6Rw6dMhxHMeZMWOGExcX56xevdr5z3/+4/Tt29dJSkpyzpw5E+SZezZq1CgnNjbW2bhxo5OXl1d5nD59unLM73//eycxMdFZv369s23bNqdLly5Oly5dgjjr0ECnQxe9rhoTOu04ZvaaTledCb02sdOOQ6+ryoROO46ZvabTVWdCr03stOPQ66oyodOOY2avbel0yC4OOo7jzJ0710lMTHSioqKcTp06OVu2bAn2lLyyYcMGR0SUY9iwYY7jfL+V96OPPurEx8c70dHRTs+ePZ2cnJzgTvpn6K5HRJzFixdXjjlz5owzevRo56KLLnLq1q3r9OvXz8nLywvepEMInQ5N9Lrqwr3TjmNmr+m0b8K91yZ22nHotS/CvdOOY2av6bRvwr3XJnbacei1L8K9045jZq9t6XSE4ziOd581BAAAAAAAAGCCkHzmIAAAAAAAAIDAY3EQAAAAAAAAsBSLgwAAAAAAAIClWBwEAAAAAAAALMXiIAAAAAAAAGApFgcBAAAAAAAAS7E4CAAAAAAAAFiKxUEAAAAAAADAUiwOAgAAAAAAAJZicRAAAAAAAACwFIuDAAAAAAAAgKX+H9kDRDMQW71hAAAAAElFTkSuQmCC",
|
||
"text/plain": [
|
||
"<Figure size 1600x400 with 7 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
" 5 0 4 1 9 2 1\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"draw_examples(x_train[:7], captions=y_train)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"60000 przykładów uczących\n",
|
||
"10000 przykładów testowych\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"num_classes = 10\n",
|
||
"\n",
|
||
"x_train = x_train.reshape(60000, 784) # 784 = 28 * 28\n",
|
||
"x_test = x_test.reshape(10000, 784)\n",
|
||
"x_train = x_train.astype(\"float32\")\n",
|
||
"x_test = x_test.astype(\"float32\")\n",
|
||
"x_train /= 255\n",
|
||
"x_test /= 255\n",
|
||
"print(\"{} przykładów uczących\".format(x_train.shape[0]))\n",
|
||
"print(\"{} przykładów testowych\".format(x_test.shape[0]))\n",
|
||
"\n",
|
||
"# przekonwertuj wektory klas na binarne macierze klas\n",
|
||
"y_train = keras.utils.to_categorical(y_train, num_classes)\n",
|
||
"y_test = keras.utils.to_categorical(y_test, num_classes)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"metadata": {
|
||
"scrolled": true,
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"2023-01-26 10:52:27.077963: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\n",
|
||
"2023-01-26 10:52:27.078089: W tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:265] failed call to cuInit: UNKNOWN ERROR (303)\n",
|
||
"2023-01-26 10:52:27.078807: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ELLIOT): /proc/driver/nvidia/version does not exist\n",
|
||
"2023-01-26 10:52:27.095828: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA\n",
|
||
"To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Model: \"sequential\"\n",
|
||
"_________________________________________________________________\n",
|
||
" Layer (type) Output Shape Param # \n",
|
||
"=================================================================\n",
|
||
" dense (Dense) (None, 512) 401920 \n",
|
||
" \n",
|
||
" dense_1 (Dense) (None, 512) 262656 \n",
|
||
" \n",
|
||
" dense_2 (Dense) (None, 10) 5130 \n",
|
||
" \n",
|
||
"=================================================================\n",
|
||
"Total params: 669,706\n",
|
||
"Trainable params: 669,706\n",
|
||
"Non-trainable params: 0\n",
|
||
"_________________________________________________________________\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"model = keras.Sequential()\n",
|
||
"model.add(Dense(512, activation=\"relu\", input_shape=(784,)))\n",
|
||
"model.add(Dropout(0.2))\n",
|
||
"model.add(Dense(512, activation=\"relu\"))\n",
|
||
"model.add(Dropout(0.2))\n",
|
||
"model.add(Dense(num_classes, activation=\"softmax\"))\n",
|
||
"model.summary()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"(60000, 784) (60000, 10)\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(x_train.shape, y_train.shape)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"2023-01-26 10:52:27.713204: W tensorflow/tsl/framework/cpu_allocator_impl.cc:82] Allocation of 188160000 exceeds 10% of free system memory.\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Epoch 1/5\n",
|
||
"469/469 [==============================] - 13s 25ms/step - loss: 0.2303 - accuracy: 0.9290 - val_loss: 0.1023 - val_accuracy: 0.9684\n",
|
||
"Epoch 2/5\n",
|
||
"469/469 [==============================] - 9s 20ms/step - loss: 0.0840 - accuracy: 0.9742 - val_loss: 0.0794 - val_accuracy: 0.9754\n",
|
||
"Epoch 3/5\n",
|
||
"469/469 [==============================] - 9s 20ms/step - loss: 0.0548 - accuracy: 0.9826 - val_loss: 0.0603 - val_accuracy: 0.9828\n",
|
||
"Epoch 4/5\n",
|
||
"469/469 [==============================] - 9s 20ms/step - loss: 0.0367 - accuracy: 0.9883 - val_loss: 0.0707 - val_accuracy: 0.9796\n",
|
||
"Epoch 5/5\n",
|
||
"469/469 [==============================] - 9s 19ms/step - loss: 0.0278 - accuracy: 0.9912 - val_loss: 0.0765 - val_accuracy: 0.9785\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"<keras.callbacks.History at 0x7f8642785120>"
|
||
]
|
||
},
|
||
"execution_count": 12,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"model.compile(\n",
|
||
" loss=\"categorical_crossentropy\",\n",
|
||
" optimizer=keras.optimizers.RMSprop(),\n",
|
||
" metrics=[\"accuracy\"],\n",
|
||
")\n",
|
||
"\n",
|
||
"model.fit(\n",
|
||
" x_train,\n",
|
||
" y_train,\n",
|
||
" batch_size=128,\n",
|
||
" epochs=5,\n",
|
||
" verbose=1,\n",
|
||
" validation_data=(x_test, y_test),\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Test loss: 0.07645954936742783\n",
|
||
"Test accuracy: 0.9785000085830688\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"score = model.evaluate(x_test, y_test, verbose=0)\n",
|
||
"\n",
|
||
"print(\"Test loss: {}\".format(score[0]))\n",
|
||
"print(\"Test accuracy: {}\".format(score[1]))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"source": [
|
||
"Warstwa _dropout_ to metoda regularyzacji, służy zapobieganiu nadmiernemu dopasowaniu sieci. Polega na tym, że część węzłów sieci jest usuwana w sposób losowy."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 14,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "notes"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Model: \"sequential_1\"\n",
|
||
"_________________________________________________________________\n",
|
||
" Layer (type) Output Shape Param # \n",
|
||
"=================================================================\n",
|
||
" dense_3 (Dense) (None, 512) 401920 \n",
|
||
" \n",
|
||
" dense_4 (Dense) (None, 512) 262656 \n",
|
||
" \n",
|
||
" dense_5 (Dense) (None, 10) 5130 \n",
|
||
" \n",
|
||
"=================================================================\n",
|
||
"Total params: 669,706\n",
|
||
"Trainable params: 669,706\n",
|
||
"Non-trainable params: 0\n",
|
||
"_________________________________________________________________\n",
|
||
"Epoch 1/5\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"2023-01-26 10:53:20.710986: W tensorflow/tsl/framework/cpu_allocator_impl.cc:82] Allocation of 188160000 exceeds 10% of free system memory.\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"469/469 [==============================] - 10s 19ms/step - loss: 0.2283 - accuracy: 0.9302 - val_loss: 0.0983 - val_accuracy: 0.9685\n",
|
||
"Epoch 2/5\n",
|
||
"469/469 [==============================] - 10s 22ms/step - loss: 0.0849 - accuracy: 0.9736 - val_loss: 0.0996 - val_accuracy: 0.9673\n",
|
||
"Epoch 3/5\n",
|
||
"469/469 [==============================] - 10s 22ms/step - loss: 0.0549 - accuracy: 0.9829 - val_loss: 0.0704 - val_accuracy: 0.9777\n",
|
||
"Epoch 4/5\n",
|
||
"469/469 [==============================] - 10s 21ms/step - loss: 0.0380 - accuracy: 0.9877 - val_loss: 0.0645 - val_accuracy: 0.9797\n",
|
||
"Epoch 5/5\n",
|
||
"469/469 [==============================] - 20s 43ms/step - loss: 0.0276 - accuracy: 0.9910 - val_loss: 0.0637 - val_accuracy: 0.9825\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"<keras.callbacks.History at 0x7f86301a3f40>"
|
||
]
|
||
},
|
||
"execution_count": 14,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# Bez warstw Dropout\n",
|
||
"\n",
|
||
"num_classes = 10\n",
|
||
"\n",
|
||
"(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
|
||
"\n",
|
||
"x_train = x_train.reshape(60000, 784) # 784 = 28 * 28\n",
|
||
"x_test = x_test.reshape(10000, 784)\n",
|
||
"x_train = x_train.astype(\"float32\")\n",
|
||
"x_test = x_test.astype(\"float32\")\n",
|
||
"x_train /= 255\n",
|
||
"x_test /= 255\n",
|
||
"\n",
|
||
"y_train = keras.utils.to_categorical(y_train, num_classes)\n",
|
||
"y_test = keras.utils.to_categorical(y_test, num_classes)\n",
|
||
"\n",
|
||
"model_no_dropout = keras.Sequential()\n",
|
||
"model_no_dropout.add(Dense(512, activation=\"relu\", input_shape=(784,)))\n",
|
||
"model_no_dropout.add(Dense(512, activation=\"relu\"))\n",
|
||
"model_no_dropout.add(Dense(num_classes, activation=\"softmax\"))\n",
|
||
"model_no_dropout.summary()\n",
|
||
"\n",
|
||
"model_no_dropout.compile(\n",
|
||
" loss=\"categorical_crossentropy\",\n",
|
||
" optimizer=keras.optimizers.RMSprop(),\n",
|
||
" metrics=[\"accuracy\"],\n",
|
||
")\n",
|
||
"\n",
|
||
"model_no_dropout.fit(\n",
|
||
" x_train,\n",
|
||
" y_train,\n",
|
||
" batch_size=128,\n",
|
||
" epochs=5,\n",
|
||
" verbose=1,\n",
|
||
" validation_data=(x_test, y_test),\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 15,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Test loss (no dropout): 0.06374581903219223\n",
|
||
"Test accuracy (no dropout): 0.9825000166893005\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Bez warstw Dropout\n",
|
||
"\n",
|
||
"score = model_no_dropout.evaluate(x_test, y_test, verbose=0)\n",
|
||
"\n",
|
||
"print(\"Test loss (no dropout): {}\".format(score[0]))\n",
|
||
"print(\"Test accuracy (no dropout): {}\".format(score[1]))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 18,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "notes"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Model: \"sequential_3\"\n",
|
||
"_________________________________________________________________\n",
|
||
" Layer (type) Output Shape Param # \n",
|
||
"=================================================================\n",
|
||
" dense_6 (Dense) (None, 2500) 1962500 \n",
|
||
" \n",
|
||
" dense_7 (Dense) (None, 2000) 5002000 \n",
|
||
" \n",
|
||
" dense_8 (Dense) (None, 1500) 3001500 \n",
|
||
" \n",
|
||
" dense_9 (Dense) (None, 1000) 1501000 \n",
|
||
" \n",
|
||
" dense_10 (Dense) (None, 500) 500500 \n",
|
||
" \n",
|
||
" dense_11 (Dense) (None, 10) 5010 \n",
|
||
" \n",
|
||
"=================================================================\n",
|
||
"Total params: 11,972,510\n",
|
||
"Trainable params: 11,972,510\n",
|
||
"Non-trainable params: 0\n",
|
||
"_________________________________________________________________\n",
|
||
"Epoch 1/10\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"2023-01-26 11:06:02.193383: W tensorflow/tsl/framework/cpu_allocator_impl.cc:82] Allocation of 188160000 exceeds 10% of free system memory.\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"469/469 [==============================] - 140s 294ms/step - loss: 0.6488 - accuracy: 0.8175 - val_loss: 0.2686 - val_accuracy: 0.9211\n",
|
||
"Epoch 2/10\n",
|
||
"469/469 [==============================] - 147s 313ms/step - loss: 0.2135 - accuracy: 0.9367 - val_loss: 0.2251 - val_accuracy: 0.9363\n",
|
||
"Epoch 3/10\n",
|
||
"469/469 [==============================] - 105s 224ms/step - loss: 0.1549 - accuracy: 0.9535 - val_loss: 0.1535 - val_accuracy: 0.9533\n",
|
||
"Epoch 4/10\n",
|
||
"469/469 [==============================] - 94s 200ms/step - loss: 0.1210 - accuracy: 0.9635 - val_loss: 0.1412 - val_accuracy: 0.9599\n",
|
||
"Epoch 5/10\n",
|
||
"469/469 [==============================] - 93s 199ms/step - loss: 0.0985 - accuracy: 0.9704 - val_loss: 0.1191 - val_accuracy: 0.9650\n",
|
||
"Epoch 6/10\n",
|
||
"469/469 [==============================] - 105s 224ms/step - loss: 0.0834 - accuracy: 0.9746 - val_loss: 0.0959 - val_accuracy: 0.9732\n",
|
||
"Epoch 7/10\n",
|
||
"469/469 [==============================] - 111s 236ms/step - loss: 0.0664 - accuracy: 0.9797 - val_loss: 0.1071 - val_accuracy: 0.9685\n",
|
||
"Epoch 8/10\n",
|
||
"469/469 [==============================] - 184s 392ms/step - loss: 0.0562 - accuracy: 0.9824 - val_loss: 0.0951 - val_accuracy: 0.9737\n",
|
||
"Epoch 9/10\n",
|
||
"469/469 [==============================] - 161s 344ms/step - loss: 0.0475 - accuracy: 0.9852 - val_loss: 0.1377 - val_accuracy: 0.9631\n",
|
||
"Epoch 10/10\n",
|
||
"469/469 [==============================] - 146s 311ms/step - loss: 0.0399 - accuracy: 0.9873 - val_loss: 0.1093 - val_accuracy: 0.9736\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"<keras.callbacks.History at 0x7f8640136f50>"
|
||
]
|
||
},
|
||
"execution_count": 18,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# Więcej warstw, inna funkcja aktywacji\n",
|
||
"\n",
|
||
"num_classes = 10\n",
|
||
"\n",
|
||
"(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
|
||
"\n",
|
||
"x_train = x_train.reshape(60000, 784) # 784 = 28 * 28\n",
|
||
"x_test = x_test.reshape(10000, 784)\n",
|
||
"x_train = x_train.astype(\"float32\")\n",
|
||
"x_test = x_test.astype(\"float32\")\n",
|
||
"x_train /= 255\n",
|
||
"x_test /= 255\n",
|
||
"\n",
|
||
"y_train = keras.utils.to_categorical(y_train, num_classes)\n",
|
||
"y_test = keras.utils.to_categorical(y_test, num_classes)\n",
|
||
"\n",
|
||
"model3 = keras.Sequential()\n",
|
||
"model3.add(Dense(2500, activation=\"tanh\", input_shape=(784,)))\n",
|
||
"model3.add(Dense(2000, activation=\"tanh\"))\n",
|
||
"model3.add(Dense(1500, activation=\"tanh\"))\n",
|
||
"model3.add(Dense(1000, activation=\"tanh\"))\n",
|
||
"model3.add(Dense(500, activation=\"tanh\"))\n",
|
||
"model3.add(Dense(num_classes, activation=\"softmax\"))\n",
|
||
"model3.summary()\n",
|
||
"\n",
|
||
"model3.compile(\n",
|
||
" loss=\"categorical_crossentropy\",\n",
|
||
" optimizer=keras.optimizers.RMSprop(),\n",
|
||
" metrics=[\"accuracy\"],\n",
|
||
")\n",
|
||
"\n",
|
||
"model3.fit(\n",
|
||
" x_train,\n",
|
||
" y_train,\n",
|
||
" batch_size=128,\n",
|
||
" epochs=10,\n",
|
||
" verbose=1,\n",
|
||
" validation_data=(x_test, y_test),\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 19,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Test loss: 0.10930903255939484\n",
|
||
"Test accuracy: 0.9735999703407288\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Więcej warstw, inna funkcja aktywacji\n",
|
||
"\n",
|
||
"score = model3.evaluate(x_test, y_test, verbose=0)\n",
|
||
"\n",
|
||
"print(\"Test loss: {}\".format(score[0]))\n",
|
||
"print(\"Test accuracy: {}\".format(score[1]))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "slide"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Przykład: 4-pikselowy aparat fotograficzny\n",
|
||
"\n",
|
||
"https://www.youtube.com/watch?v=ILsA4nyG7I0"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 20,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def generate_example(description):\n",
|
||
" variant = random.choice([1, -1])\n",
|
||
" if description == \"s\": # solid\n",
|
||
" return (\n",
|
||
" np.array([[1.0, 1.0], [1.0, 1.0]])\n",
|
||
" if variant == 1\n",
|
||
" else np.array([[-1.0, -1.0], [-1.0, -1.0]])\n",
|
||
" )\n",
|
||
" elif description == \"v\": # vertical\n",
|
||
" return (\n",
|
||
" np.array([[1.0, -1.0], [1.0, -1.0]])\n",
|
||
" if variant == 1\n",
|
||
" else np.array([[-1.0, 1.0], [-1.0, 1.0]])\n",
|
||
" )\n",
|
||
" elif description == \"d\": # diagonal\n",
|
||
" return (\n",
|
||
" np.array([[1.0, -1.0], [-1.0, 1.0]])\n",
|
||
" if variant == 1\n",
|
||
" else np.array([[-1.0, 1.0], [1.0, -1.0]])\n",
|
||
" )\n",
|
||
" elif description == \"h\": # horizontal\n",
|
||
" return (\n",
|
||
" np.array([[1.0, 1.0], [-1.0, -1.0]])\n",
|
||
" if variant == 1\n",
|
||
" else np.array([[-1.0, -1.0], [1.0, 1.0]])\n",
|
||
" )\n",
|
||
" else:\n",
|
||
" return np.array(\n",
|
||
" [\n",
|
||
" [random.uniform(-1, 1), random.uniform(-1, 1)],\n",
|
||
" [random.uniform(-1, 1), random.uniform(-1, 1)],\n",
|
||
" ]\n",
|
||
" )"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 22,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"import numpy as np\n",
|
||
"import random\n",
|
||
"\n",
|
||
"num_classes = 4\n",
|
||
"\n",
|
||
"trainset_size = 4000\n",
|
||
"testset_size = 1000\n",
|
||
"\n",
|
||
"y4_train = np.array([random.choice([\"s\", \"v\", \"d\", \"h\"]) for i in range(trainset_size)])\n",
|
||
"x4_train = np.array([generate_example(desc) for desc in y4_train])\n",
|
||
"\n",
|
||
"y4_test = np.array([random.choice([\"s\", \"v\", \"d\", \"h\"]) for i in range(testset_size)])\n",
|
||
"x4_test = np.array([generate_example(desc) for desc in y4_test])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 23,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABRcAAADICAYAAAByFdYYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbGklEQVR4nO3df2xddf0/8Fe30c7FtHNja2nY4DMRjEM2GHapJmzEmolkiQbjRANz0QGGf3REZAScEOMi8gcJmZFg5lQw/IgMjBoQO36EOQYMGwi/ksniBq6d2+SWDe10Pd8/+LassM17D7295777eCSvhHt6zr3vnvvci3tfub2nIcuyLAAAAAAAKjSh1gsAAAAAAOqT4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJBL1YaL+/fvj69+9avR3NwcU6dOja9//etx4MCB4x6zePHiaGhoGFFXXHFFtZYIFZNrUiPTpEiuSY1MkyK5JjUyzXjWkGVZVo07vuCCC2L37t1x2223xX/+859YsWJFfOITn4hf//rXxzxm8eLFcfrpp8eNN944vG3KlCnR3NxcjSVCxeSa1Mg0KZJrUiPTpEiuSY1MM55NqsadvvTSS/Hggw/G008/Heeee25ERNx6663xuc99Lm6++eZob28/5rFTpkyJtra2aiwL3he5JjUyTYrkmtTINCmSa1Ij04x3VRkubtmyJaZOnTr8jyoioqurKyZMmBBbt26NL3zhC8c89s4774w77rgj2traYunSpXH99dfHlClTjrn/wMBADAwMDN8eHByM/fv3x/Tp06OhoWF0fiGIiO7u7mhpaYlzzjlneJtcU882bdoULS0t0dbWFoODgzFhwgSZpu7p1aRGryZFejWp0atJUZZl8eabb0Z7e3tMmHD8b1WsynCxt7c3Zs6cOfKBJk2KadOmRW9v7zGP+8pXvhKnnHJKtLe3x3PPPRff/e5345VXXon77rvvmMesXbs2brjhhlFbO/wvf//73+Pkk0+OCLkmDbNmzYpdu3bFySefLNMkQ68mNXo1KdKrSY1eTYqGMn08FX3n4jXXXBM/+tGPjrvPSy+9FPfdd1/84he/iFdeeWXEz2bOnBk33HBDfPOb3yzr8TZt2hSf/vSnY/v27fHhD3/4qPu8e2pfKpVi9uzZsWvXLt9TUKaWlpZaL6GuvPHGGyPO2VjmGqrlyFzr1RTRmjVr4pZbbjnuPk8//XTce++9cdNNN+nVdaJUKtV6CTVTbqZ/+9vfxp133hmvvvpqzXo1VIteXR/06luOu49eTere3auPpqJPLl511VXxta997bj7zJkzJ9ra2mLPnj0jtv/3v/+N/fv3V/RdAgsXLoyIOO4/rKampmhqanrP9ubmZm9YqYojP2o+1rmGahnKtV5NUV177bVx+eWXH3efOXPmxNatWyNCr64X4/nff7mZfu6552Lfvn0RUbteDdWiV9cHvVqvZnwr58/tKxouzpgxI2bMmPE/9+vs7Iw33ngjtm3bFgsWLIiItyfwg4ODw/9YytHT0xMRESeddFIly4Sq+stf/hKLFi2KCLkmPTJNUZX7GqSjoyMi9GqKr5LX1e/+1JBMkwq9mqLTq6FMWZV89rOfzc4+++xs69at2RNPPJF95CMfyS6++OLhn7/22mvZGWeckW3dujXLsizbvn17duONN2bPPPNMtmPHjuyBBx7I5syZk5133nkVPW6pVMoiIiuVSqP6+6QsIlQFddZZZ9Us10pVqzZt2qRXk4ShbOnV9VGUp6urq6a9WqlqlV5dH0V59GqVapXznq1qnWLfvn3ZxRdfnH3wgx/MmpubsxUrVmRvvvnm8M937NiRRUT2yCOPZFmWZTt37szOO++8bNq0aVlTU1N22mmnZd/5zncqfuPpDWvlah3UeqsvfvGLNcu1UtUqvZpUDGVLr66PojxDmZVplVrp1fVRlEevVqlWOZms6IIu9aC/vz9aWlqiVCqN6++GqITL1VemFtkayjVUy1jnWq+mWmqZLb26com9DK2aWuVapqk2vbo+6NXl0atJVTmZnjBGawEAAAAAEmO4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAORiuAgAAAAA5GK4CAAAAADkYrgIAAAAAOQyJsPFdevWxamnnhqTJ0+OhQsXxlNPPXXc/e+999746Ec/GpMnT46Pf/zj8Yc//GEslgllk2lSc/vtt8s0ydGrSY1eTYr0alKjVzMuZVV21113ZY2Njdn69euzF154IVu5cmU2derUrK+v76j7b968OZs4cWJ20003ZS+++GJ23XXXZSeccEL2/PPPl/V4pVIpi4isVCqN5q+RtIhQFdT69evHNNNZ9k6ulapW1SrTejWjbShbenV9FOUZypZMq9RKr66Pojx6tUq1ynnPVvVO0dHRkV155ZXDtw8fPpy1t7dna9euPer+X/rSl7ILL7xwxLaFCxdml19+eVmP5w1r5Wod1HqrBQsWjGmms8z/MFT1a+XKlTXJtF7NaBvKll5dH0V5hrJVq16tVLVKr66Pojx6tUq1ynnPVtU/iz506FBs27Yturq6hrdNmDAhurq6YsuWLUc9ZsuWLSP2j4hYsmTJMfcfGBiI/v7+EQXV1NPTU9VMR8g1Y2/x4sXD/y3TpECvJiWHDh2KCL2a9OjVpESvZjyr6nBx7969cfjw4WhtbR2xvbW1NXp7e496TG9vb0X7r127NlpaWoZr1qxZo7N4OIZqZzpCrhl7M2fOHHFbpql3ejUp2bdvX0To1aRHryYlejXjWd1fLXr16tVRKpWGa9euXbVeErxvck1qZJoUyTWpkWlSJNekRqYpoknVvPMTTzwxJk6cGH19fSO29/X1RVtb21GPaWtrq2j/pqamaGpqGp0FQxmqnekIuWbs7dmzZ8Rtmabe6dWkZPr06RGhV5MevZqU6NWMZ1X95GJjY2MsWLAguru7h7cNDg5Gd3d3dHZ2HvWYzs7OEftHRDz88MPH3B/G2vz582Wa5Dz22GPD/y3TpECvJiWNjY0RoVeTHr2alOjVjGtlX4Iop7vuuitramrKNmzYkL344ovZZZddlk2dOjXr7e3NsizLLrnkkuyaa64Z3n/z5s3ZpEmTsptvvjl76aWXsjVr1lR0KXZXIK1cFODqQ/VU69evH9NMZ5krgKnqV60yrVcz2oaypVfXR1GeoWzJtEqt9Or6KMqjV6tUq5z3bGPSKW699dZs9uzZWWNjY9bR0ZE9+eSTwz9btGhRtnz58hH733PPPdnpp5+eNTY2ZnPnzs1+//vfl/1Y3rBWrtZBrbcqlUpjmuks8z8MVf368Y9/XJNM69WMtiOzpVcXvyjPULZq1auVqlbp1fVRlEevVqlWOe/ZGrIsyyIh/f390dLSEqVSKZqbm2u9nLrQ0NBQ6yXUlVpkayjXUC1jnWu9mmqpZbb06sol9jK0amqVa5mm2vTq+qBXl0evJlXlZLrurxYNAAAAANSG4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQy5gMF9etWxennnpqTJ48ORYuXBhPPfXUMffdsGFDNDQ0jKjJkyePxTKhbDJNam6//XaZJjl6NanRq0mRXk1q9GrGo6oPF+++++5YtWpVrFmzJp599tmYN29eLFmyJPbs2XPMY5qbm2P37t3D9be//a3ay4Sy/eY3v5FpknPttdfKNEnRq0mRXk1q9GpSpFczLmVV1tHRkV155ZXDtw8fPpy1t7dna9euPer+P//5z7OWlpbcj1cqlbKIyEqlUu77GG8iQlVQCxYsGNNMZ9k7uVaqWrVy5cqaZFqvZrQNZUuvro+iPEPZqlWvVqpapVfXR1EevVqlWuW8Z5sUVXTo0KHYtm1brF69enjbhAkToqurK7Zs2XLM4w4cOBCnnHJKDA4OxjnnnBM//OEPY+7cuUfdd2BgIAYGBoZv9/f3R0RES0vLKP0W6cuyrNZLqAv9/f3R0tISPT09cd111w1vH+1MRxw716VSKZqbm0fht4G37d27N2bMmBGLFy8e3jaWmdarqRa9uj40NDTUegl1pVa9WqYZbV5X1xe9ujJ6NakY6tXlqOqfRe/duzcOHz4cra2tI7a3trZGb2/vUY8544wzYv369fHAAw/EHXfcEYODg/HJT34yXnvttaPuv3bt2mhpaRmuWbNmjfrvAUeqdqYj5Jqxs2/fvoiImDlz5ojtMk2906tJkV5NavRqUqRXMx4V7mrRnZ2dcemll8b8+fNj0aJFcd9998WMGTPitttuO+r+q1evjlKpNFy7du0a4xXD8VWa6Qi5pthkmhTJNamRaVIk16RGpklFVf8s+sQTT4yJEydGX1/fiO19fX3R1tZW1n2ccMIJcfbZZ8f27duP+vOmpqZoamp632uFclU70xFyzdiZPn16RMR7vmRapql3ejUp0qtJjV5NivRqxqOqfnKxsbExFixYEN3d3cPbBgcHo7u7Ozo7O8u6j8OHD8fzzz8fJ510UrWWCRWZP3++TJOMxsbGiIh47LHHhrfJNCnQq0mRXk1q9GpSpFczLr2vyxKV4a677sqampqyDRs2ZC+++GJ22WWXZVOnTs16e3uzLMuySy65JLvmmmuG97/hhhuyhx56KPvrX/+abdu2Lfvyl7+cTZ48OXvhhRfKejxXSqq8KM9QttavXz+mmT7ysV1Zl9E2lK1aZVqpapVeXR9qnZN6K5kmFV5X15da9756K5kmFZVkq6p/Fh0RsWzZsvjHP/4R3/ve96K3tzfmz58fDz744PAX9+7cuTMmTHjnA5T//Oc/Y+XKldHb2xsf+tCHYsGCBfHnP/85Pvaxj1V7qVCWiy66KA4ePCjTJOUHP/iBTJMUvZoU6dWkRq8mRXo141FDlmVZrRcxmiq5VDZvSywCVTOUrVKpFM3NzePmsUlbrbKlV1NtenV9aGhoqPUS6kqterVMM9q8rq4venVl9GpSUUm2Cne1aAAAAACgPhguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5GC4CAAAAALkYLgIAAAAAuRguAgAAAAC5VHW4+Pjjj8fSpUujvb09Ghoa4v777/+fxzz66KNxzjnnRFNTU5x22mmxYcOGai4RKrZ582a5JjnLli2TaZIi06RIrkmNTJMiuWY8qupw8eDBgzFv3rxYt25dWfvv2LEjLrzwwjj//POjp6cnvvWtb8U3vvGNeOihh6q5TKjIW2+9Jdck58wzz5RpkiLTpEiuSY1MkyK5ZlzKxkhEZBs3bjzuPldffXU2d+7cEduWLVuWLVmypOzHKZVKWUSoCoryDGWrVCoNbxvrXB/52DAa3p0tvVqlUmOd6SNzrVeXr9Y5qbeqVa+WaUZbrV5/HO2x+d9q3fvqrfRqUlFJtgr1nYtbtmyJrq6uEduWLFkSW7ZsOeYxAwMD0d/fP6KgSOSa1Mg0qcmT6Qi5ptj0alKjV5MivZpUFGq42NvbG62trSO2tba2Rn9/f/zrX/866jFr166NlpaW4Zo1a9ZYLBXKJtekRqZJTZ5MR8g1xaZXkxq9mhTp1aSiUMPFPFavXh2lUmm4du3aVeslwfsm16RGpkmRXJMamSZFck1qZJoimlTrBRypra0t+vr6Rmzr6+uL5ubm+MAHPnDUY5qamqKpqWkslge5yDWpkWlSkyfTEXJNsenVpEavJkV6Nako1CcXOzs7o7u7e8S2hx9+ODo7O2u0Inj/5JrUyDSpkWlSJNekRqZJkVyTiqoOFw8cOBA9PT3R09MTEW9fZr2npyd27twZEW9/nPfSSy8d3v+KK66IV199Na6++up4+eWX4yc/+Uncc8898e1vf7uay4SKyDUpeu6552SapMg0KZJrUiPTpEiuGZeqednqRx555KiXZl++fHmWZVm2fPnybNGiRe85Zv78+VljY2M2Z86c7Oc//3lFjzl0qWxVflGeoWz97ne/q1muy7kEPFTieD1Tr1apVTUzfWSu9ery1ToTKZTXH9SjWr3+OPKx5bp8te5zKZReTT2qJFsNWZZlkZD+/v5oaWmp9TLqSmIRqJqhbJVKpWhubh43j03aapUtvZpq06vrQ0NDQ62XUFdq1atlmtHmdXV90asro1eTikqyVajvXAQAAAAA6ofhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJBLVYeLjz/+eCxdujTa29ujoaEh7r///uPu/+ijj0ZDQ8N7qre3t5rLhIps3rxZrknOsmXLZJqkyDQpkmtSI9OkSK4Zj6o6XDx48GDMmzcv1q1bV9Fxr7zySuzevXu4Zs6cWaUVQuXeeustuSY5Z555pkyTFJkmRXJNamSaFMk149Gkat75BRdcEBdccEHFx82cOTOmTp06+guCUfCZz3wmLrroooqPk2uK7Prrr4/m5uaKjpFpikymSZFckxqZJkVyzXhU1eFiXvPnz4+BgYE488wz4/vf/3586lOfOua+AwMDMTAwMHy7VCqNxRKT0t/fX+sl1IWh85RlWa7jRyPXnitG2/vJtV5NkVU70xF6NWOvVr1aphltY/X6I0KuGXt6NamoqFdnYyQiso0bNx53n5dffjn76U9/mj3zzDPZ5s2bsxUrVmSTJk3Ktm3bdsxj1qxZk0WEUmNWu3btkmuVXA3lOkKmVRpV7UzLtapF6dUqtdKrVYqlV6vU6sgZyLE0/P/QV11DQ0Ns3LgxPv/5z1d03KJFi2L27Nnxq1/96qg/f/fUfnBwMPbv3x/Tp0+PhoaG97PkUdXf3x+zZs2KXbt2VfwR6fGoiOcry7J48803o729PSZMePvrSsdzrov4HBVZUc/Xu3M9njMdUdznqaiKeL7GKtMR9ZHrIj5HRVbU86VXj1TU56mIinqu9OqRivo8FVVRz5dePVJRn6ciKuq5OtoM5FgK+WfRR+ro6IgnnnjimD9vamqKpqamEduK/F0Fzc3NhQpL0RXtfLW0tIzK/aSU66I9R0VXxPM1GrlOKdMRxXyeiqxo52ssMh1RX7ku2nNUdEU8X3r1exXxeSqqIp4rvfq9ivg8FVkRz5de/V5FfJ6KqojnqtxMV/Vq0aOhp6cnTjrppFovA0aVXJMamSY1Mk2K5JrUyDQpkmvqUVU/uXjgwIHYvn378O0dO3ZET09PTJs2LWbPnh2rV6+O119/PX75y19GRMQtt9wS//d//xdz586Nf//73/Gzn/0sNm3aFH/84x+ruUyoiFyTGpkmNTJNiuSa1Mg0KZJrxquqDhefeeaZOP/884dvr1q1KiIili9fHhs2bIjdu3fHzp07h39+6NChuOqqq+L111+PKVOmxFlnnRV/+tOfRtxHvWpqaoo1a9a85+PLHF2Rz5dcv63Iz1ERFfl8yfQ7ivw8FVFRz5dMv6Ooz1FRFfl8yfU7ivw8FU2Rz5VMv6PIz1MRFfl8yfU7ivw8FU0K52rMLugCAAAAAKSl8N+5CAAAAAAUk+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+HiGFm3bl2ceuqpMXny5Fi4cGE89dRTtV5SIT3++OOxdOnSaG9vj4aGhrj//vtrvSSOQabLI9P1Q6bLJ9f1Q67LI9P1Q6bLJ9f1Q67LI9P1Q6bLl0quDRfHwN133x2rVq2KNWvWxLPPPhvz5s2LJUuWxJ49e2q9tMI5ePBgzJs3L9atW1frpXAcMl0+ma4PMl0Zua4Pcl0+ma4PMl0Zua4Pcl0+ma4PMl2ZZHKdUXUdHR3ZlVdeOXz78OHDWXt7e7Z27doarqr4IiLbuHFjrZfBUch0PjJdXDKdn1wXl1znI9PFJdP5yXVxyXU+Ml1cMp1fPefaJxer7NChQ7Ft27bo6uoa3jZhwoTo6uqKLVu21HBlkI9MkxqZJkVyTWpkmhTJNamR6fHLcLHK9u7dG4cPH47W1tYR21tbW6O3t7dGq4L8ZJrUyDQpkmtSI9OkSK5JjUyPX4aLAAAAAEAuhotVduKJJ8bEiROjr69vxPa+vr5oa2ur0aogP5kmNTJNiuSa1Mg0KZJrUiPT45fhYpU1NjbGggULoru7e3jb4OBgdHd3R2dnZw1XBvnINKmRaVIk16RGpkmRXJMamR6/JtV6AePBqlWrYvny5XHuuedGR0dH3HLLLXHw4MFYsWJFrZdWOAcOHIjt27cP396xY0f09PTEtGnTYvbs2TVcGUeS6fLJdH2Q6crIdX2Q6/LJdH2Q6crIdX2Q6/LJdH2Q6cokk+taX656vLj11luz2bNnZ42NjVlHR0f25JNP1npJhfTII49kEfGeWr58ea2XxrvIdHlkun7IdPnkun7IdXlkun7IdPnkun7IdXlkun7IdPlSyXVDlmVZleeXAAAAAECCfOciAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQi+EiAAAAAJCL4SIAAAAAkIvhIgAAAACQy/8DtjV1ROlkKSAAAAAASUVORK5CYII=",
|
||
"text/plain": [
|
||
"<Figure size 1600x400 with 7 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
" d h h d h d h\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"draw_examples(x4_train[:7], captions=y4_train)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 24,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"x4_train = x4_train.reshape(trainset_size, 4)\n",
|
||
"x4_test = x4_test.reshape(testset_size, 4)\n",
|
||
"x4_train = x4_train.astype(\"float32\")\n",
|
||
"x4_test = x4_test.astype(\"float32\")\n",
|
||
"\n",
|
||
"y4_train = np.array([{\"s\": 0, \"v\": 1, \"d\": 2, \"h\": 3}[desc] for desc in y4_train])\n",
|
||
"y4_test = np.array([{\"s\": 0, \"v\": 1, \"d\": 2, \"h\": 3}[desc] for desc in y4_test])\n",
|
||
"\n",
|
||
"y4_train = keras.utils.to_categorical(y4_train, num_classes)\n",
|
||
"y4_test = keras.utils.to_categorical(y4_test, num_classes)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 25,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Model: \"sequential_4\"\n",
|
||
"_________________________________________________________________\n",
|
||
" Layer (type) Output Shape Param # \n",
|
||
"=================================================================\n",
|
||
" dense_12 (Dense) (None, 4) 20 \n",
|
||
" \n",
|
||
" dense_13 (Dense) (None, 4) 20 \n",
|
||
" \n",
|
||
" dense_14 (Dense) (None, 8) 40 \n",
|
||
" \n",
|
||
" dense_15 (Dense) (None, 4) 36 \n",
|
||
" \n",
|
||
"=================================================================\n",
|
||
"Total params: 116\n",
|
||
"Trainable params: 116\n",
|
||
"Non-trainable params: 0\n",
|
||
"_________________________________________________________________\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"model4 = keras.Sequential()\n",
|
||
"model4.add(Dense(4, activation=\"tanh\", input_shape=(4,)))\n",
|
||
"model4.add(Dense(4, activation=\"tanh\"))\n",
|
||
"model4.add(Dense(8, activation=\"relu\"))\n",
|
||
"model4.add(Dense(num_classes, activation=\"softmax\"))\n",
|
||
"model4.summary()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 26,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"model4.layers[0].set_weights(\n",
|
||
" [\n",
|
||
" np.array(\n",
|
||
" [\n",
|
||
" [1.0, 0.0, 1.0, 0.0],\n",
|
||
" [0.0, 1.0, 0.0, 1.0],\n",
|
||
" [1.0, 0.0, -1.0, 0.0],\n",
|
||
" [0.0, 1.0, 0.0, -1.0],\n",
|
||
" ],\n",
|
||
" dtype=np.float32,\n",
|
||
" ),\n",
|
||
" np.array([0.0, 0.0, 0.0, 0.0], dtype=np.float32),\n",
|
||
" ]\n",
|
||
")\n",
|
||
"model4.layers[1].set_weights(\n",
|
||
" [\n",
|
||
" np.array(\n",
|
||
" [\n",
|
||
" [1.0, -1.0, 0.0, 0.0],\n",
|
||
" [1.0, 1.0, 0.0, 0.0],\n",
|
||
" [0.0, 0.0, 1.0, -1.0],\n",
|
||
" [0.0, 0.0, -1.0, -1.0],\n",
|
||
" ],\n",
|
||
" dtype=np.float32,\n",
|
||
" ),\n",
|
||
" np.array([0.0, 0.0, 0.0, 0.0], dtype=np.float32),\n",
|
||
" ]\n",
|
||
")\n",
|
||
"model4.layers[2].set_weights(\n",
|
||
" [\n",
|
||
" np.array(\n",
|
||
" [\n",
|
||
" [1.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],\n",
|
||
" [0.0, 0.0, 1.0, -1.0, 0.0, 0.0, 0.0, 0.0],\n",
|
||
" [0.0, 0.0, 0.0, 0.0, 1.0, -1.0, 0.0, 0.0],\n",
|
||
" [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, -1.0],\n",
|
||
" ],\n",
|
||
" dtype=np.float32,\n",
|
||
" ),\n",
|
||
" np.array([0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], dtype=np.float32),\n",
|
||
" ]\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 27,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"model4.layers[3].set_weights(\n",
|
||
" [\n",
|
||
" np.array(\n",
|
||
" [\n",
|
||
" [1.0, 0.0, 0.0, 0.0],\n",
|
||
" [1.0, 0.0, 0.0, 0.0],\n",
|
||
" [0.0, 1.0, 0.0, 0.0],\n",
|
||
" [0.0, 1.0, 0.0, 0.0],\n",
|
||
" [0.0, 0.0, 1.0, 0.0],\n",
|
||
" [0.0, 0.0, 1.0, 0.0],\n",
|
||
" [0.0, 0.0, 0.0, 1.0],\n",
|
||
" [0.0, 0.0, 0.0, 1.0],\n",
|
||
" ],\n",
|
||
" dtype=np.float32,\n",
|
||
" ),\n",
|
||
" np.array([0.0, 0.0, 0.0, 0.0], dtype=np.float32),\n",
|
||
" ]\n",
|
||
")\n",
|
||
"\n",
|
||
"model4.compile(\n",
|
||
" loss=\"categorical_crossentropy\",\n",
|
||
" optimizer=keras.optimizers.Adagrad(),\n",
|
||
" metrics=[\"accuracy\"],\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 28,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[array([[ 1., 0., 1., 0.],\n",
|
||
" [ 0., 1., 0., 1.],\n",
|
||
" [ 1., 0., -1., 0.],\n",
|
||
" [ 0., 1., 0., -1.]], dtype=float32), array([0., 0., 0., 0.], dtype=float32)]\n",
|
||
"[array([[ 1., -1., 0., 0.],\n",
|
||
" [ 1., 1., 0., 0.],\n",
|
||
" [ 0., 0., 1., -1.],\n",
|
||
" [ 0., 0., -1., -1.]], dtype=float32), array([0., 0., 0., 0.], dtype=float32)]\n",
|
||
"[array([[ 1., -1., 0., 0., 0., 0., 0., 0.],\n",
|
||
" [ 0., 0., 1., -1., 0., 0., 0., 0.],\n",
|
||
" [ 0., 0., 0., 0., 1., -1., 0., 0.],\n",
|
||
" [ 0., 0., 0., 0., 0., 0., 1., -1.]], dtype=float32), array([0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)]\n",
|
||
"[array([[1., 0., 0., 0.],\n",
|
||
" [1., 0., 0., 0.],\n",
|
||
" [0., 1., 0., 0.],\n",
|
||
" [0., 1., 0., 0.],\n",
|
||
" [0., 0., 1., 0.],\n",
|
||
" [0., 0., 1., 0.],\n",
|
||
" [0., 0., 0., 1.],\n",
|
||
" [0., 0., 0., 1.]], dtype=float32), array([0., 0., 0., 0.], dtype=float32)]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"for layer in model4.layers:\n",
|
||
" print(layer.get_weights())"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 29,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"1/1 [==============================] - 1s 872ms/step\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"array([[0.17831734, 0.17831734, 0.17831734, 0.465048 ]], dtype=float32)"
|
||
]
|
||
},
|
||
"execution_count": 29,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"model4.predict([np.array([[1.0, 1.0], [-1.0, -1.0]]).reshape(1, 4)])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 30,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Test loss: 0.7656148672103882\n",
|
||
"Test accuracy: 1.0\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"score = model4.evaluate(x4_test, y4_test, verbose=0)\n",
|
||
"\n",
|
||
"print(\"Test loss: {}\".format(score[0]))\n",
|
||
"print(\"Test accuracy: {}\".format(score[1]))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 32,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Model: \"sequential_5\"\n",
|
||
"_________________________________________________________________\n",
|
||
" Layer (type) Output Shape Param # \n",
|
||
"=================================================================\n",
|
||
" dense_16 (Dense) (None, 4) 20 \n",
|
||
" \n",
|
||
" dense_17 (Dense) (None, 4) 20 \n",
|
||
" \n",
|
||
" dense_18 (Dense) (None, 8) 40 \n",
|
||
" \n",
|
||
" dense_19 (Dense) (None, 4) 36 \n",
|
||
" \n",
|
||
"=================================================================\n",
|
||
"Total params: 116\n",
|
||
"Trainable params: 116\n",
|
||
"Non-trainable params: 0\n",
|
||
"_________________________________________________________________\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"model5 = keras.Sequential()\n",
|
||
"model5.add(Dense(4, activation=\"tanh\", input_shape=(4,)))\n",
|
||
"model5.add(Dense(4, activation=\"tanh\"))\n",
|
||
"model5.add(Dense(8, activation=\"relu\"))\n",
|
||
"model5.add(Dense(num_classes, activation=\"softmax\"))\n",
|
||
"model5.compile(\n",
|
||
" loss=\"categorical_crossentropy\",\n",
|
||
" optimizer=keras.optimizers.RMSprop(),\n",
|
||
" metrics=[\"accuracy\"],\n",
|
||
")\n",
|
||
"model5.summary()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 33,
|
||
"metadata": {
|
||
"scrolled": true,
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Epoch 1/8\n",
|
||
"125/125 [==============================] - 3s 8ms/step - loss: 1.3014 - accuracy: 0.4947 - val_loss: 1.1876 - val_accuracy: 0.6040\n",
|
||
"Epoch 2/8\n",
|
||
"125/125 [==============================] - 1s 6ms/step - loss: 1.0779 - accuracy: 0.7395 - val_loss: 0.9865 - val_accuracy: 0.8730\n",
|
||
"Epoch 3/8\n",
|
||
"125/125 [==============================] - 1s 4ms/step - loss: 0.8925 - accuracy: 0.8382 - val_loss: 0.8114 - val_accuracy: 0.7460\n",
|
||
"Epoch 4/8\n",
|
||
"125/125 [==============================] - 0s 4ms/step - loss: 0.7266 - accuracy: 0.8060 - val_loss: 0.6622 - val_accuracy: 0.8730\n",
|
||
"Epoch 5/8\n",
|
||
"125/125 [==============================] - 0s 4ms/step - loss: 0.5890 - accuracy: 0.8765 - val_loss: 0.5392 - val_accuracy: 0.8730\n",
|
||
"Epoch 6/8\n",
|
||
"125/125 [==============================] - 1s 4ms/step - loss: 0.4738 - accuracy: 0.8838 - val_loss: 0.4293 - val_accuracy: 0.8730\n",
|
||
"Epoch 7/8\n",
|
||
"125/125 [==============================] - 1s 5ms/step - loss: 0.3636 - accuracy: 0.9337 - val_loss: 0.3191 - val_accuracy: 1.0000\n",
|
||
"Epoch 8/8\n",
|
||
"125/125 [==============================] - 1s 5ms/step - loss: 0.2606 - accuracy: 1.0000 - val_loss: 0.2202 - val_accuracy: 1.0000\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"<keras.callbacks.History at 0x7f860a6a9870>"
|
||
]
|
||
},
|
||
"execution_count": 33,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"model5.fit(x4_train, y4_train, epochs=8, validation_data=(x4_test, y4_test))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 34,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"1/1 [==============================] - 0s 106ms/step\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"array([[1.5366691e-01, 4.4674356e-04, 4.7448810e-02, 7.9843748e-01]],\n",
|
||
" dtype=float32)"
|
||
]
|
||
},
|
||
"execution_count": 34,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"model5.predict([np.array([[1.0, 1.0], [-1.0, -1.0]]).reshape(1, 4)])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 35,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Test loss: 0.22015966475009918\n",
|
||
"Test accuracy: 1.0\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"score = model5.evaluate(x4_test, y4_test, verbose=0)\n",
|
||
"\n",
|
||
"print(\"Test loss: {}\".format(score[0]))\n",
|
||
"print(\"Test accuracy: {}\".format(score[1]))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 36,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "notes"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"import contextlib\n",
|
||
"\n",
|
||
"\n",
|
||
"@contextlib.contextmanager\n",
|
||
"def printoptions(*args, **kwargs):\n",
|
||
" original = np.get_printoptions()\n",
|
||
" np.set_printoptions(*args, **kwargs)\n",
|
||
" try:\n",
|
||
" yield\n",
|
||
" finally:\n",
|
||
" np.set_printoptions(**original)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 37,
|
||
"metadata": {
|
||
"slideshow": {
|
||
"slide_type": "subslide"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"[array([[-0.8, 0.1, -0.6, 0.1],\n",
|
||
" [-0.9, -0.7, -1. , 0.6],\n",
|
||
" [-0.3, 0.5, 0.5, 0.3],\n",
|
||
" [ 0.4, 0.3, -0.9, -0.8]], dtype=float32), array([ 0., -0., 0., 0.], dtype=float32)]\n",
|
||
"[array([[-1.1, 1.2, -0.6, -0.6],\n",
|
||
" [-1.1, -0.2, -0.7, -1.3],\n",
|
||
" [ 0.6, 0.9, 0.3, -1.3],\n",
|
||
" [ 0.8, 0.3, 0.7, 0.4]], dtype=float32), array([ 0.3, 0.5, -0.4, 0.5], dtype=float32)]\n",
|
||
"[array([[ 0.5, 0.4, -0.4, 0.3, 0.8, -1.4, -1.1, 0.8],\n",
|
||
" [ 0.5, -1.3, 0.3, 0.4, -1.3, 0.2, 0.9, 0.7],\n",
|
||
" [-0.2, -0.1, -0.5, -0.2, 1.2, -0.4, -0.4, 1.1],\n",
|
||
" [-1.1, 0.4, 1.3, -1.1, 1. , -1.1, -0.8, 0.3]], dtype=float32), array([ 0.2, 0.2, 0.1, 0.1, 0.2, 0.1, -0.2, 0. ], dtype=float32)]\n",
|
||
"[array([[ 0.7, 0.8, -1.5, -0.2],\n",
|
||
" [ 0.7, -0.9, -1.2, 0.2],\n",
|
||
" [-0.4, 1.1, -0.1, -1.6],\n",
|
||
" [ 0.3, 0.8, -1.4, 0.4],\n",
|
||
" [ 0.2, -1.4, -0.3, 0.5],\n",
|
||
" [-0.2, -1.2, 0.6, 0.7],\n",
|
||
" [-0.1, -1.5, 0.3, -0.1],\n",
|
||
" [-1.4, 0.1, 1.2, -0. ]], dtype=float32), array([-0.2, 0.5, 0.5, -0.5], dtype=float32)]\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"with printoptions(precision=1, suppress=True):\n",
|
||
" for layer in model5.layers:\n",
|
||
" print(layer.get_weights())"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"author": "Paweł Skórzewski",
|
||
"celltoolbar": "Slideshow",
|
||
"email": "pawel.skorzewski@amu.edu.pl",
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"lang": "pl",
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.10.12"
|
||
},
|
||
"livereveal": {
|
||
"start_slideshow_at": "selected",
|
||
"theme": "white"
|
||
},
|
||
"subtitle": "10.Sieci neuronowe – propagacja wsteczna[wykład]",
|
||
"title": "Uczenie maszynowe",
|
||
"vscode": {
|
||
"interpreter": {
|
||
"hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
|
||
}
|
||
},
|
||
"year": "2021"
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 4
|
||
}
|