2022-10-14 11:34:46 +02:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Uczenie maszynowe\n",
"# 3. Regresja liniowa –
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 3.1. Regresja liniowa wielu zmiennych"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"Do przewidywania wartości $y$ możemy użyć więcej niż jednej cechy $x$:"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Przykład –
]
},
{
"cell_type": "code",
2022-10-18 13:52:38 +02:00
"execution_count": 5,
"metadata": {},
2022-10-14 11:34:46 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2022-10-18 13:52:38 +02:00
" y:price x1:isNew x2:rooms x3:floor x4:location x5:sqrMetres\n",
"1 476118.0 False 3 1 Centrum 78\n",
"2 459531.0 False 3 2 Sołacz 62\n",
"3 411557.0 False 3 0 Sołacz 15\n",
"4 496416.0 False 4 0 Sołacz 14\n",
"5 406032.0 False 3 0 Sołacz 15\n",
"... ... ... ... ... ... ...\n",
"1335 349000.0 False 4 0 Szczepankowo 29\n",
"1336 399000.0 False 5 0 Szczepankowo 68\n",
"1337 234000.0 True 2 7 Wilda 50\n",
"1338 210000.0 True 2 1 Wilda 65\n",
"1339 279000.0 True 2 2 Łazarz 36\n",
"\n",
"[1339 rows x 6 columns]\n"
2022-10-14 11:34:46 +02:00
]
}
],
"source": [
2022-10-18 13:52:38 +02:00
"import numpy as np\n",
"import pandas as pd\n",
2022-10-14 11:34:46 +02:00
"\n",
2022-10-18 13:52:38 +02:00
"data = pd.read_csv(\"data02_train.tsv\", sep=\"\\t\")\n",
"data.rename(columns={col: f\"x{i}:{col}\" if i > 0 else f\"y:{col}\" for i, col in enumerate(data.columns)}, inplace=True)\n",
"data.index = np.arange(1, len(data) + 1)\n",
"print(data)"
2022-10-14 11:34:46 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"$$ x^{(2)} = ({\\rm \"False\"}, 3, 2, {\\rm \"Sołacz\"}, 62), \\quad x_3^{(2)} = 2 $$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Hipoteza"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"W naszym przypadku (wybraliśmy 5 cech):\n",
"\n",
"$$ h_\\theta(x) = \\theta_0 + \\theta_1 x_1 + \\theta_2 x_2 + \\theta_3 x_3 + \\theta_4 x_4 + \\theta_5 x_5 $$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"W ogólności ($n$ cech):\n",
"\n",
"$$ h_\\theta(x) = \\theta_0 + \\theta_1 x_1 + \\theta_2 x_2 + \\ldots + \\theta_n x_n $$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Jeżeli zdefiniujemy $x_0 = 1$, będziemy mogli powyższy wzór zapisać w bardziej kompaktowy sposób:\n",
"\n",
"$$\n",
"\\begin{array}{rcl}\n",
"h_\\theta(x)\n",
" & = & \\theta_0 x_0 + \\theta_1 x_1 + \\theta_2 x_2 + \\ldots + \\theta_n x_n \\\\\n",
" & = & \\displaystyle\\sum_{i=0}^{n} \\theta_i x_i \\\\\n",
" & = & \\theta^T \\, x \\\\\n",
" & = & x^T \\, \\theta \\\\\n",
"\\end{array}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Metoda gradientu prostego –
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"Metoda gradientu prostego przyjmie bardzo elegancką formę, jeżeli do jej zapisu użyjemy wektorów i macierzy."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"$$\n",
"X=\\left[\\begin{array}{cc}\n",
"1 & \\left( \\vec x^{(1)} \\right)^T \\\\\n",
"1 & \\left( \\vec x^{(2)} \\right)^T \\\\\n",
"\\vdots & \\vdots\\\\\n",
"1 & \\left( \\vec x^{(m)} \\right)^T \\\\\n",
"\\end{array}\\right] \n",
"= \\left[\\begin{array}{cccc}\n",
"1 & x_1^{(1)} & \\cdots & x_n^{(1)} \\\\\n",
"1 & x_1^{(2)} & \\cdots & x_n^{(2)} \\\\\n",
"\\vdots & \\vdots & \\ddots & \\vdots\\\\\n",
"1 & x_1^{(m)} & \\cdots & x_n^{(m)} \\\\\n",
"\\end{array}\\right]\n",
"\\quad\n",
"\\vec{y} = \n",
"\\left[\\begin{array}{c}\n",
"y^{(1)}\\\\\n",
"y^{(2)}\\\\\n",
"\\vdots\\\\\n",
"y^{(m)}\\\\\n",
"\\end{array}\\right]\n",
"\\quad\n",
"\\theta = \\left[\\begin{array}{c}\n",
"\\theta_0\\\\\n",
"\\theta_1\\\\\n",
"\\vdots\\\\\n",
"\\theta_n\\\\\n",
"\\end{array}\\right]\n",
"$$"
]
},
{
"cell_type": "code",
2022-10-18 13:52:38 +02:00
"execution_count": 6,
2022-10-14 11:34:46 +02:00
"metadata": {
"slideshow": {
"slide_type": "skip"
}
},
"outputs": [],
"source": [
"# Wersje macierzowe funkcji rysowania wykresów punktowych oraz krzywej regresyjnej\n",
"\n",
2022-10-18 13:52:38 +02:00
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"\n",
2022-10-14 11:34:46 +02:00
"\n",
2022-10-18 13:52:38 +02:00
"def h(theta, x):\n",
" return x * theta\n",
2022-10-14 11:34:46 +02:00
"\n",
"\n",
2022-10-18 13:52:38 +02:00
"def regdots(x, y, xlabel=\"populacja\", ylabel=\"zysk\"):\n",
2022-10-14 11:34:46 +02:00
" fig = plt.figure(figsize=(16 * 0.6, 9 * 0.6))\n",
" ax = fig.add_subplot(111)\n",
" fig.subplots_adjust(left=0.1, right=0.9, bottom=0.1, top=0.9)\n",
2022-10-18 13:52:38 +02:00
" ax.scatter([x[:, 1]], [y], c=\"r\", s=50, label=\"Dane\")\n",
2022-10-14 11:34:46 +02:00
"\n",
2022-10-18 13:52:38 +02:00
" ax.set_xlabel(xlabel)\n",
" ax.set_ylabel(ylabel)\n",
2022-10-14 11:34:46 +02:00
" ax.margins(0.05, 0.05)\n",
" plt.ylim(y.min() - 1, y.max() + 1)\n",
2022-10-18 13:52:38 +02:00
" plt.xlim(np.min(x[:, 1]) - 1, np.max(x[:, 1]) + 1)\n",
2022-10-14 11:34:46 +02:00
" return fig\n",
"\n",
"\n",
2022-10-18 13:52:38 +02:00
"def regline(fig, fun, theta, x):\n",
2022-10-14 11:34:46 +02:00
" ax = fig.axes[0]\n",
2022-10-18 13:52:38 +02:00
" x_min = np.min(x[:, 1])\n",
" x_max = np.max(x[:, 1])\n",
" x_range = [x_min, x_max]\n",
" x_matrix = np.matrix([1, x_min, 1, x_max]).reshape(2, 2)\n",
2022-10-14 11:34:46 +02:00
" ax.plot(\n",
2022-10-18 13:52:38 +02:00
" x_range,\n",
" fun(theta, x_matrix),\n",
2022-10-14 11:34:46 +02:00
" linewidth=\"2\",\n",
" label=(\n",
" r\"$y={theta0:.2}{op}{theta1:.2}x$\".format(\n",
" theta0=float(theta[0][0]),\n",
" theta1=(\n",
" float(theta[1][0]) if theta[1][0] >= 0 else float(-theta[1][0])\n",
" ),\n",
" op=\"+\" if theta[1][0] >= 0 else \"-\",\n",
" )\n",
" ),\n",
" )\n"
]
},
{
"cell_type": "code",
2022-10-18 13:52:38 +02:00
"execution_count": 9,
2022-10-14 11:34:46 +02:00
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2022-10-18 13:52:38 +02:00
"x[:5]=matrix([[ 1., 3., 1., 78.],\n",
" [ 1., 3., 2., 62.],\n",
" [ 1., 3., 0., 15.],\n",
" [ 1., 4., 0., 14.],\n",
" [ 1., 3., 0., 15.]])\n",
"x.shape=(1339, 4)\n",
"y[:5]=matrix([[476118.],\n",
" [459531.],\n",
" [411557.],\n",
" [496416.],\n",
" [406032.]])\n",
"y.shape=(1339, 1)\n"
2022-10-14 11:34:46 +02:00
]
}
],
"source": [
2022-10-18 13:52:38 +02:00
"# Wczytwanie danych z pliku – –
2022-10-14 11:34:46 +02:00
"\n",
2022-10-18 13:52:38 +02:00
"import pandas as pd\n",
2022-10-14 11:34:46 +02:00
"\n",
2022-10-18 13:52:38 +02:00
"data = pd.read_csv(\n",
2022-10-14 11:34:46 +02:00
" \"data02_train.tsv\", delimiter=\"\\t\", usecols=[\"price\", \"rooms\", \"floor\", \"sqrMetres\"]\n",
")\n",
"m, n_plus_1 = data.values.shape\n",
"n = n_plus_1 - 1\n",
2022-10-18 13:52:38 +02:00
"xn = data.values[:, 1:].reshape(m, n)\n",
2022-10-14 11:34:46 +02:00
"\n",
"# Dodaj kolumnę jedynek do macierzy\n",
2022-10-18 13:52:38 +02:00
"x = np.matrix(np.concatenate((np.ones((m, 1)), xn), axis=1)).reshape(m, n_plus_1)\n",
"y = np.matrix(data.values[:, 0]).reshape(m, 1)\n",
2022-10-14 11:34:46 +02:00
"\n",
2022-10-18 13:52:38 +02:00
"print(f\"{x[:5]=}\")\n",
"print(f\"{x.shape=}\")\n",
"print(f\"{y[:5]=}\")\n",
"print(f\"{y.shape=}\")\n"
2022-10-14 11:34:46 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Funkcja kosztu –
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"$$J(\\theta)=\\dfrac{1}{2|\\vec y|}\\left(X\\theta-\\vec{y}\\right)^T\\left(X\\theta-\\vec{y}\\right)$$ \n"
]
},
{
"cell_type": "code",
2022-10-18 13:52:38 +02:00
"execution_count": 10,
2022-10-14 11:34:46 +02:00
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [
{
"data": {
"text/latex": [
"$\\displaystyle \\Large J(\\theta) = 85104141370.9717$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from IPython.display import display, Math, Latex\n",
"\n",
"\n",
2022-10-18 13:52:38 +02:00
"def J(theta, X, y):\n",
2022-10-14 11:34:46 +02:00
" \"\"\"Wersja macierzowa funkcji kosztu\"\"\"\n",
" m = len(y)\n",
2022-10-18 13:52:38 +02:00
" cost = 1.0 / (2.0 * m) * ((X * theta - y).T * (X * theta - y))\n",
" return cost.item()\n",
2022-10-14 11:34:46 +02:00
"\n",
"\n",
2022-10-18 13:52:38 +02:00
"theta = np.matrix([10, 90, -1, 2.5]).reshape(4, 1)\n",
2022-10-14 11:34:46 +02:00
"\n",
2022-10-18 13:52:38 +02:00
"cost = J(theta, x, y)\n",
2022-10-14 11:34:46 +02:00
"display(Math(r\"\\Large J(\\theta) = %.4f\" % cost))\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Gradient –
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"$$\\nabla J(\\theta) = \\frac{1}{|\\vec y|} X^T\\left(X\\theta-\\vec y\\right)$$"
]
},
{
"cell_type": "code",
2022-10-18 13:52:38 +02:00
"execution_count": 13,
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"# Wyświetlanie macierzy w LaTeX-u\n",
"\n",
"\n",
"def latex_matrix(matrix):\n",
" ltx = r\"\\left[\\begin{array}\"\n",
" m, n = matrix.shape\n",
" ltx += \"{\" + (\"r\" * n) + \"}\"\n",
" for i in range(m):\n",
" ltx += r\" & \".join([(\"%.4f\" % j.item()) for j in matrix[i]]) + r\" \\\\ \"\n",
" ltx += r\"\\end{array}\\right]\"\n",
" return ltx\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
2022-10-14 11:34:46 +02:00
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [
{
"data": {
"text/latex": [
"$\\displaystyle \\large \\theta = \\left[\\begin{array}{r}10.0000 \\\\ 90.0000 \\\\ -1.0000 \\\\ 2.5000 \\\\ \\end{array}\\right]\\quad\\large \\nabla J(\\theta) = \\left[\\begin{array}{r}-373492.7442 \\\\ -1075656.5086 \\\\ -989554.4921 \\\\ -23806475.6561 \\\\ \\end{array}\\right]$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from IPython.display import display, Math, Latex\n",
"\n",
"\n",
2022-10-18 13:52:38 +02:00
"def dJ(theta, X, y):\n",
2022-10-14 11:34:46 +02:00
" \"\"\"Wersja macierzowa gradientu funckji kosztu\"\"\"\n",
" return 1.0 / len(y) * (X.T * (X * theta - y))\n",
"\n",
"\n",
2022-10-18 13:52:38 +02:00
"theta = np.matrix([10, 90, -1, 2.5]).reshape(4, 1)\n",
2022-10-14 11:34:46 +02:00
"\n",
"display(\n",
" Math(\n",
" r\"\\large \\theta = \"\n",
2022-10-18 13:52:38 +02:00
" + latex_matrix(theta)\n",
2022-10-14 11:34:46 +02:00
" + r\"\\quad\"\n",
" + r\"\\large \\nabla J(\\theta) = \"\n",
2022-10-18 13:52:38 +02:00
" + latex_matrix(dJ(theta, x, y))\n",
2022-10-14 11:34:46 +02:00
" )\n",
")\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Algorytm gradientu prostego –
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"$$ \\theta := \\theta - \\alpha \\, \\nabla J(\\theta) $$"
]
},
{
"cell_type": "code",
2022-10-18 13:52:38 +02:00
"execution_count": 16,
2022-10-14 11:34:46 +02:00
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [
{
"data": {
"text/latex": [
"$\\displaystyle \\large\\textrm{Wynik:}\\quad \\theta = \\left[\\begin{array}{r}17446.2135 \\\\ 86476.7960 \\\\ -1374.8950 \\\\ 2165.0689 \\\\ \\end{array}\\right] \\quad J(\\theta) = 10324864803.1591 \\quad \\textrm{po 374575 iteracjach}$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Implementacja algorytmu gradientu prostego za pomocą numpy i macierzy\n",
"\n",
"\n",
2022-10-18 13:52:38 +02:00
"def gradient_descent(fJ, fdJ, theta, X, y, alpha, eps):\n",
2022-10-14 11:34:46 +02:00
" current_cost = fJ(theta, X, y)\n",
" history = [[current_cost, theta]]\n",
" while True:\n",
" theta = theta - alpha * fdJ(theta, X, y) # implementacja wzoru\n",
" current_cost, prev_cost = fJ(theta, X, y), current_cost\n",
" if abs(prev_cost - current_cost) <= eps:\n",
" break\n",
" if current_cost > prev_cost:\n",
" print(\"Długość kroku (alpha) jest zbyt duża!\")\n",
" break\n",
" history.append([current_cost, theta])\n",
" return theta, history\n",
"\n",
"\n",
2022-10-18 13:52:38 +02:00
"theta_start = np.zeros((n + 1, 1))\n",
2022-10-14 11:34:46 +02:00
"\n",
"# Zmieniamy wartości alpha (rozmiar kroku) oraz eps (kryterium stopu)\n",
2022-10-18 13:52:38 +02:00
"theta_best, history = gradient_descent(J, dJ, theta_start, x, y, alpha=0.0001, eps=0.1)\n",
2022-10-14 11:34:46 +02:00
"\n",
"display(\n",
" Math(\n",
" r\"\\large\\textrm{Wynik:}\\quad \\theta = \"\n",
2022-10-18 13:52:38 +02:00
" + latex_matrix(theta_best)\n",
2022-10-14 11:34:46 +02:00
" + (r\" \\quad J(\\theta) = %.4f\" % history[-1][0])\n",
" + r\" \\quad \\textrm{po %d iteracjach}\" % len(history)\n",
" )\n",
")\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 3.2. Metoda gradientu prostego w praktyce"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"### Kryterium stopu"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"Algorytm gradientu prostego polega na wykonywaniu określonych kroków w pętli. Pytanie brzmi: kiedy należy zatrzymać wykonywanie tej pętli?\n",
"\n",
"W każdej kolejnej iteracji wartość funkcji kosztu maleje o coraz mniejszą wartość.\n",
"Parametr `eps` określa, jaka wartość graniczna tej różnicy jest dla nas wystarczająca:\n",
"\n",
" * Im mniejsza wartość
" * Im większa wartość `eps`, tym krótszy czas działania algorytmu, ale mniej dokładny wynik."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"Na wykresie zobaczymy porównanie regresji dla różnych wartości `eps`"
]
},
{
"cell_type": "code",
2022-10-18 13:52:38 +02:00
"execution_count": 18,
2022-10-14 11:34:46 +02:00
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
2022-10-18 13:52:38 +02:00
{
"ename": "ValueError",
"evalue": "x and y must be the same size",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn [18], line 3\u001b[0m\n\u001b[1;32m 1\u001b[0m theta_start \u001b[39m=\u001b[39m np\u001b[39m.\u001b[39mzeros((\u001b[39m2\u001b[39m, \u001b[39m1\u001b[39m))\n\u001b[0;32m----> 3\u001b[0m fig \u001b[39m=\u001b[39m regdots(x[\u001b[39m1\u001b[39m], y)\n",
"Cell \u001b[0;32mIn [6], line 15\u001b[0m, in \u001b[0;36mregdots\u001b[0;34m(x, y, xlabel, ylabel)\u001b[0m\n\u001b[1;32m 13\u001b[0m ax \u001b[39m=\u001b[39m fig\u001b[39m.\u001b[39madd_subplot(\u001b[39m111\u001b[39m)\n\u001b[1;32m 14\u001b[0m fig\u001b[39m.\u001b[39msubplots_adjust(left\u001b[39m=\u001b[39m\u001b[39m0.1\u001b[39m, right\u001b[39m=\u001b[39m\u001b[39m0.9\u001b[39m, bottom\u001b[39m=\u001b[39m\u001b[39m0.1\u001b[39m, top\u001b[39m=\u001b[39m\u001b[39m0.9\u001b[39m)\n\u001b[0;32m---> 15\u001b[0m ax\u001b[39m.\u001b[39;49mscatter([x[:, \u001b[39m1\u001b[39;49m]], [y], c\u001b[39m=\u001b[39;49m\u001b[39m\"\u001b[39;49m\u001b[39mr\u001b[39;49m\u001b[39m\"\u001b[39;49m, s\u001b[39m=\u001b[39;49m\u001b[39m50\u001b[39;49m, label\u001b[39m=\u001b[39;49m\u001b[39m\"\u001b[39;49m\u001b[39mDane\u001b[39;49m\u001b[39m\"\u001b[39;49m)\n\u001b[1;32m 17\u001b[0m ax\u001b[39m.\u001b[39mset_xlabel(xlabel)\n\u001b[1;32m 18\u001b[0m ax\u001b[39m.\u001b[39mset_ylabel(ylabel)\n",
"File \u001b[0;32m~/.local/lib/python3.10/site-packages/matplotlib/__init__.py:1423\u001b[0m, in \u001b[0;36m_preprocess_data.<locals>.inner\u001b[0;34m(ax, data, *args, **kwargs)\u001b[0m\n\u001b[1;32m 1420\u001b[0m \u001b[39m@functools\u001b[39m\u001b[39m.\u001b[39mwraps(func)\n\u001b[1;32m 1421\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39minner\u001b[39m(ax, \u001b[39m*\u001b[39margs, data\u001b[39m=\u001b[39m\u001b[39mNone\u001b[39;00m, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[1;32m 1422\u001b[0m \u001b[39mif\u001b[39;00m data \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[0;32m-> 1423\u001b[0m \u001b[39mreturn\u001b[39;00m func(ax, \u001b[39m*\u001b[39;49m\u001b[39mmap\u001b[39;49m(sanitize_sequence, args), \u001b[39m*\u001b[39;49m\u001b[39m*\u001b[39;49mkwargs)\n\u001b[1;32m 1425\u001b[0m bound \u001b[39m=\u001b[39m new_sig\u001b[39m.\u001b[39mbind(ax, \u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs)\n\u001b[1;32m 1426\u001b[0m auto_label \u001b[39m=\u001b[39m (bound\u001b[39m.\u001b[39marguments\u001b[39m.\u001b[39mget(label_namer)\n\u001b[1;32m 1427\u001b[0m \u001b[39mor\u001b[39;00m bound\u001b[39m.\u001b[39mkwargs\u001b[39m.\u001b[39mget(label_namer))\n",
"File \u001b[0;32m~/.local/lib/python3.10/site-packages/matplotlib/axes/_axes.py:4512\u001b[0m, in \u001b[0;36mAxes.scatter\u001b[0;34m(self, x, y, s, c, marker, cmap, norm, vmin, vmax, alpha, linewidths, edgecolors, plotnonfinite, **kwargs)\u001b[0m\n\u001b[1;32m 4510\u001b[0m y \u001b[39m=\u001b[39m np\u001b[39m.\u001b[39mma\u001b[39m.\u001b[39mravel(y)\n\u001b[1;32m 4511\u001b[0m \u001b[39mif\u001b[39;00m x\u001b[39m.\u001b[39msize \u001b[39m!=\u001b[39m y\u001b[39m.\u001b[39msize:\n\u001b[0;32m-> 4512\u001b[0m \u001b[39mraise\u001b[39;00m \u001b[39mValueError\u001b[39;00m(\u001b[39m\"\u001b[39m\u001b[39mx and y must be the same size\u001b[39m\u001b[39m\"\u001b[39m)\n\u001b[1;32m 4514\u001b[0m \u001b[39mif\u001b[39;00m s \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m 4515\u001b[0m s \u001b[39m=\u001b[39m (\u001b[39m20\u001b[39m \u001b[39mif\u001b[39;00m mpl\u001b[39m.\u001b[39mrcParams[\u001b[39m'\u001b[39m\u001b[39m_internal.classic_mode\u001b[39m\u001b[39m'\u001b[39m] \u001b[39melse\u001b[39;00m\n\u001b[1;32m 4516\u001b[0m mpl\u001b[39m.\u001b[39mrcParams[\u001b[39m'\u001b[39m\u001b[39mlines.markersize\u001b[39m\u001b[39m'\u001b[39m] \u001b[39m*\u001b[39m\u001b[39m*\u001b[39m \u001b[39m2.0\u001b[39m)\n",
"\u001b[0;31mValueError\u001b[0m: x and y must be the same size"
]
},
2022-10-14 11:34:46 +02:00
{
"data": {
2022-10-18 13:52:38 +02:00
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAz4AAAHhCAYAAACvPGsLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAgDElEQVR4nO3df2zdZb3A8U/b0VOItAznuh/34C54ARXYcGO14EK4qTSRTPfHDXWYbVlARCcBGq9s/FhFdN1VIEukuDDxYnKDmy6ChC1FrCxGqVnc1gTiNoJzbiG22/SunUVb1n7vH4bjrevGTmk7+vh6JeePPTzPOc8hD/O8/Z4fJVmWZQEAAJCw0jO9AQAAgLEmfAAAgOQJHwAAIHnCBwAASJ7wAQAAkid8AACA5AkfAAAgecIHAABInvABAACSJ3wAAIDkFR0+P//5z2PhwoUxY8aMKCkpiWeeeeZt12zbti0+8pGPRC6Xiw984APx5JNPjmCrAAAAI1N0+PT29sbs2bOjpaXltOb/7ne/ixtuuCGuu+666OjoiDvvvDNuueWWeP7554veLAAAwEiUZFmWjXhxSUk8/fTTsWjRopPOufvuu2PLli3xyiuvFMY+/elPx9GjR6O1tXWkDw0AAHDaJo31A7S3t0ddXd2Qsfr6+rjzzjtPuqavry/6+voKfx4cHIw//elP8d73vjdKSkrGaqsAAMC7QJZlcezYsZgxY0aUlo7O1xKMefh0dnZGdXX1kLHq6uro6emJv/zlL3H22WefsKa5uTkeeOCBsd4aAADwLnbw4MH4l3/5l1G5rzEPn5FYtWpVNDY2Fv7c3d0dF1xwQRw8eDAqKyvP4M4AAICx1tPTE/l8Ps4999xRu88xD59p06ZFV1fXkLGurq6orKwc9mpPREQul4tcLnfCeGVlpfABAIB/EqP5MZcx/x2f2traaGtrGzL2wgsvRG1t7Vg/NAAAQESMIHz+/Oc/R0dHR3R0dETE376uuqOjIw4cOBARf3ub2tKlSwvzb7vttti3b198+ctfjj179sRjjz0WP/jBD+Kuu+4anWcAAADwNooOn1//+tdx5ZVXxpVXXhkREY2NjXHllVfG6tWrIyLiD3/4QyGCIiL+9V//NbZs2RIvvPBCzJ49Ox5++OH4zne+E/X19aP0FAAAAE7tHf2Oz3jp6emJqqqq6O7u9hkfAABI3Fi8/h/zz/gAAACcacIHAABInvABAACSJ3wAAIDkCR8AACB5wgcAAEie8AEAAJInfAAAgOQJHwAAIHnCBwAASJ7wAQAAkid8AACA5AkfAAAgecIHAABInvABAACSJ3wAAIDkCR8AACB5wgcAAEie8AEAAJInfAAAgOQJHwAAIHnCBwAASJ7wAQAAkid8AACA5AkfAAAgecIHAABInvABAACSJ3wAAIDkCR8AACB5wgcAAEie8AEAAJInfAAAgOQJHwAAIHnCBwAASJ7wAQAAkid8AACA5AkfAAAgecIHAABInvABAACSJ3wAAIDkCR8AACB5wgcAAEie8AEAAJInfAAAgOQJHwAAIHnCBwAASJ7wAQAAkid8AACA5AkfAAAgecIHAABInvABAACSJ3wAAIDkCR8AACB5wgcAAEie8AEAAJInfAAAgOQJHwAAIHnCBwAASJ7wAQAAkid8AACA5AkfAAAgecIHAABInvABAACSJ3wAAIDkCR8AACB5wgcAAEie8AEAAJInfAAAgOQJHwAAIHnCBwAASJ7wAQAAkid8AACA5AkfAAAgecIHAABInvABAACSN6LwaWlpiVmzZkVFRUXU1NTE9u3bTzl/3bp1cckll8TZZ58d+Xw+7rrrrvjrX/86og0DAAAUq+jw2bRpUzQ2NkZTU1Ps3LkzZs+eHfX19XHo0KFh5z/11FOxcuXKaGpqit27d8cTTzwRmzZtinvuuecdbx4AAOB0FB0+jzzySHz2s5+N5cuXx4c+9KFYv359nHPOOfHd73532PkvvfRSXHPNNXHTTTfFrFmz4vrrr4/Fixe/7VUiAACA0VJU+PT398eOHTuirq7u73dQWhp1dXXR3t4+7Jqrr746duzYUQidffv2xdatW+MTn/jEO9g2AADA6ZtUzOQjR47EwMBAVFdXDxmvrq6OPXv2DLvmpptuiiNHjsTHPvaxyLIsjh8/Hrfddtsp3+rW19cXfX19hT/39PQUs00AAIAhxvxb3bZt2xZr1qyJxx57LHbu3Bk/+tGPYsuWLfHggw+edE1zc3NUVVUVbvl8fqy3CQAAJKwky7LsdCf39/fHOeecE5s3b45FixYVxpctWxZHjx6NH//4xyesWbBgQXz0ox+Nb37zm4Wx//mf/4lbb701/vznP0dp6YntNdwVn3w+H93d3VFZWXm62wUAACagnp6eqKqqGtXX/0Vd8SkvL4+5c+dGW1tbYWxwcDDa2tqitrZ22DVvvPHGCXFTVlYWEREna65cLheVlZVDbgAAACNV1Gd8IiIaGxtj2bJlMW/evJg/f36sW7cuent7Y/ny5RERsXTp0pg5c2Y0NzdHRMTChQvjkUceiSuvvDJqamritddei/vvvz8WLlxYCCAAAICxVHT4NDQ0xOHDh2P16tXR2dkZc+bMidbW1sIXHhw4cGDIFZ777rsvSkpK4r777ovXX3893ve+98XChQvj61//+ug9CwAAgFMo6jM+Z8pYvMcPAAB4dzrjn/EBAACYiIQPAACQPOEDAAAkT/gAAADJEz4AAEDyhA8AAJA84QMAACRP+AAAAMkTPgAAQPKEDwAAkDzhAwAAJE/4AAAAyRM+AABA8oQPAACQPOEDAAAkT/gAAADJEz4AAEDyhA8AAJA84QMAACRP+AAAAMkTPgAAQPKEDwAAkDzhAwAAJE/4AAAAyRM+AABA8oQPAACQPOEDAAAkT/gAAADJEz4AAEDyhA8AAJA84QMAACRP+AAAAMkTPgAAQPKEDwAAkDzhAwAAJE/4AAAAyRM+AABA8oQPAACQPOEDAAAkT/gAAADJEz4AAEDyhA8AAJA84QMAACRP+AAAAMkTPgAAQPKEDwAAkDzhAwAAJE/4AAAAyRM+AABA8oQPAACQPOEDAAAkT/gAAADJEz4AAEDyhA8AAJA84QMAACRP+AAAAMkTPgAAQPKEDwAAkDzhAwAAJE/4AAAAyRM+AABA8oQPAACQPOEDAAAkT/gAAADJEz4AAEDyhA8AAJA84QMAACRP+AAAAMkTPgAAQPKEDwAAkDzhAwAAJE/4AAAAyRM+AABA8oQPAACQPOEDAAAkT/gAAADJG1H4tLS0xKxZs6KioiJqampi+/btp5x/9OjRWLFiRUyfPj1yuVxcfPHFsXXr1hFtGAAAoFiTil2wadOmaGxsjPXr10dNTU2sW7cu6uvrY+/evTF16tQT5vf398fHP/7xmDp1amzevDlmzpwZv//97+O8884bjf0DAAC8rZIsy7JiFtTU1MRVV10Vjz76aEREDA4ORj6fj9tvvz1Wrlx5wvz169fHN7/5zdizZ0+cddZZI9pkT09PVFVVRXd3d1RWVo7oPgAAgIlhLF7/F/VWt/7+/tixY0fU1dX9/Q5KS6Ouri7a29uHXfPss89GbW1trFixIqqrq+Oyyy6LNWvWxMDAwEkfp6+vL3p6eobcAAAARqqo8Dly5EgMDAxEdXX1kPHq6uro7Owcds2+ffti8+bNMTAwEFu3bo37778/Hn744fja17520sdpbm6Oqqqqwi2fzxezTQAAgCHG/FvdBgcHY+rUqfH444/H3Llzo6GhIe69995Yv379SdesWrUquru7C7eDBw+O9TYBAICEFfXlBlOmTImysrLo6uoaMt7V1RXTpk0bds306dPjrLPOirKyssLYBz/4wejs7Iz+/v4oLy8/YU0ul4tcLlfM1gAAAE6qqCs+5eXlMXfu3GhrayuMDQ4ORltbW9TW1g675pprronXXnstBgcHC2OvvvpqTJ8+fdjoAQAAGG1Fv9WtsbExNmzYEN/73vdi9+7d8fnPfz56e3tj+fLlERGxdOnSWLVqVWH+5z//+fjTn/4Ud9xxR7z66quxZcu
2022-10-14 11:34:46 +02:00
"text/plain": [
"<Figure size 960x540 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
2022-10-18 13:52:38 +02:00
"theta_start = np.zeros((2, 1))\n",
"\n",
"fig = regdots(x[1], y)\n",
"# theta_e1, history1 = GDMx(\n",
"# JMx, dJMx, thetaStartMx, XMx, yMx, alpha=0.01, eps=0.01\n",
"# ) # niebieska linia\n",
"# reglineMx(fig, hMx, theta_e1, XMx)\n",
"# theta_e2, history2 = GDMx(\n",
"# JMx, dJMx, thetaStartMx, XMx, yMx, alpha=0.01, eps=0.000001\n",
"# ) # pomarańczowa linia\n",
"# reglineMx(fig, hMx, theta_e2, XMx)\n",
"# legend(fig)\n"
2022-10-14 11:34:46 +02:00
]
},
{
"cell_type": "code",
"execution_count": 77,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/latex": [
"$\\displaystyle \\theta_{10^{-2}} = \\left[\\begin{array}{r}0.0531 \\\\ 0.8365 \\\\ \\end{array}\\right]\\quad\\theta_{10^{-6}} = \\left[\\begin{array}{r}-3.4895 \\\\ 1.1786 \\\\ \\end{array}\\right]$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display(\n",
" Math(\n",
" r\"\\theta_{10^{-2}} = \"\n",
2022-10-18 13:52:38 +02:00
" + latex_matrix(theta_e1)\n",
2022-10-14 11:34:46 +02:00
" + r\"\\quad\\theta_{10^{-6}} = \"\n",
2022-10-18 13:52:38 +02:00
" + latex_matrix(theta_e2)\n",
2022-10-14 11:34:46 +02:00
" )\n",
")\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Długość kroku ($\\alpha$)"
]
},
{
"cell_type": "code",
"execution_count": 78,
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"# Jak zmienia się koszt w kolejnych krokach w zależności od alfa\n",
"\n",
"\n",
"def costchangeplot(history):\n",
" fig = plt.figure(figsize=(16 * 0.6, 9 * 0.6))\n",
" ax = fig.add_subplot(111)\n",
" fig.subplots_adjust(left=0.1, right=0.9, bottom=0.1, top=0.9)\n",
" ax.set_xlabel(\"krok\")\n",
" ax.set_ylabel(r\"$J(\\theta)$\")\n",
"\n",
" X = np.arange(0, 500, 1)\n",
" Y = [history[step][0] for step in X]\n",
" ax.plot(X, Y, linewidth=\"2\", label=(r\"$J(\\theta)$\"))\n",
" return fig\n",
"\n",
"\n",
"def slide7(alpha):\n",
" best_theta, history = gradient_descent(\n",
" h, J, [0.0, 0.0], x, y, alpha=alpha, eps=0.0001\n",
" )\n",
" fig = costchangeplot(history)\n",
" legend(fig)\n",
"\n",
"\n",
"sliderAlpha1 = widgets.FloatSlider(\n",
" min=0.01, max=0.03, step=0.001, value=0.02, description=r\"$\\alpha$\", width=300\n",
")\n"
]
},
{
"cell_type": "code",
"execution_count": 79,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "52b0d91e39104f4facbb7f57819aae0c",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"interactive(children=(FloatSlider(value=0.02, description='$\\\\alpha$', max=0.03, min=0.01, step=0.001), Button…"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"<function __main__.slide7(alpha)>"
]
},
"execution_count": 79,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"widgets.interact_manual(slide7, alpha=sliderAlpha1)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## 3.3. Normalizacja danych"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"Normalizacja danych to proces, który polega na dostosowaniu danych wejściowych w taki sposób, żeby ułatwić działanie algorytmowi gradientu prostego.\n",
"\n",
"Wyjaśnię to na przykladzie."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Użyjemy danych z „Gratka flats challenge 2017”.\n",
"\n",
"Rozważmy model $h(x) = \\theta_0 + \\theta_1 x_1 + \\theta_2 x_2$, w którym cena mieszkania prognozowana jest na podstawie liczby pokoi $x_1$ i metrażu $x_2$:"
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>price</th>\n",
" <th>rooms</th>\n",
" <th>sqrMetres</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>476118.00</td>\n",
" <td>3</td>\n",
" <td>78</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>459531.00</td>\n",
" <td>3</td>\n",
" <td>62</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>411557.00</td>\n",
" <td>3</td>\n",
" <td>15</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>496416.00</td>\n",
" <td>4</td>\n",
" <td>14</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>406032.00</td>\n",
" <td>3</td>\n",
" <td>15</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>450026.00</td>\n",
" <td>3</td>\n",
" <td>80</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>571229.15</td>\n",
" <td>2</td>\n",
" <td>39</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>325000.00</td>\n",
" <td>3</td>\n",
" <td>54</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>268229.00</td>\n",
" <td>2</td>\n",
" <td>90</td>\n",
" </tr>\n",
" <tr>\n",
" <th>9</th>\n",
" <td>604836.00</td>\n",
" <td>4</td>\n",
" <td>40</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" price rooms sqrMetres\n",
"0 476118.00 3 78\n",
"1 459531.00 3 62\n",
"2 411557.00 3 15\n",
"3 496416.00 4 14\n",
"4 406032.00 3 15\n",
"5 450026.00 3 80\n",
"6 571229.15 2 39\n",
"7 325000.00 3 54\n",
"8 268229.00 2 90\n",
"9 604836.00 4 40"
]
},
"execution_count": 81,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Wczytanie danych przy pomocy biblioteki pandas\n",
"import pandas\n",
"\n",
"alldata = pandas.read_csv(\n",
" \"data_flats.tsv\", header=0, sep=\"\\t\", usecols=[\"price\", \"rooms\", \"sqrMetres\"]\n",
")\n",
"alldata[:10]\n"
]
},
{
"cell_type": "code",
"execution_count": 82,
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"# Funkcja, która pokazuje wartości minimalne i maksymalne w macierzy X\n",
"\n",
"\n",
"def show_mins_and_maxs(XMx):\n",
" mins = np.amin(XMx, axis=0).tolist()[0] # wartości minimalne\n",
" maxs = np.amax(XMx, axis=0).tolist()[0] # wartości maksymalne\n",
" for i, (xmin, xmax) in enumerate(zip(mins, maxs)):\n",
" display(Math(r\"${:.2F} \\leq x_{} \\leq {:.2F}$\".format(xmin, i, xmax)))\n"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"# Przygotowanie danych\n",
"\n",
"import numpy as np\n",
"\n",
"%matplotlib inline\n",
"\n",
"data2 = np.matrix(alldata[['rooms', 'sqrMetres', 'price']])\n",
"\n",
"m, n_plus_1 = data2.shape\n",
"n = n_plus_1 - 1\n",
"Xn = data2[:, 0:n]\n",
"\n",
"XMx2 = np.matrix(np.concatenate((np.ones((m, 1)), Xn), axis=1)).reshape(m, n_plus_1)\n",
"yMx2 = np.matrix(data2[:, -1]).reshape(m, 1) / 1000.0"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Cechy w danych treningowych przyjmują wartości z zakresu:"
]
},
{
"cell_type": "code",
"execution_count": 84,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
"text/latex": [
"$\\displaystyle 1.00 \\leq x_0 \\leq 1.00$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/latex": [
"$\\displaystyle 2.00 \\leq x_1 \\leq 7.00$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/latex": [
"$\\displaystyle 12.00 \\leq x_2 \\leq 196.00$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_mins_and_maxs(XMx2)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"Jak widzimy, $x_2$ przyjmuje wartości dużo większe niż $x_1$.\n",
"Powoduje to, że wykres funkcji kosztu jest bardzo „spłaszczony” wzdłuż jednej z osi:"
]
},
{
"cell_type": "code",
"execution_count": 85,
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"outputs": [],
"source": [
"def contour_plot(X, y, rescale=10**8):\n",
" theta0_vals = np.linspace(-100000, 100000, 100)\n",
" theta1_vals = np.linspace(-100000, 100000, 100)\n",
"\n",
" J_vals = np.zeros(shape=(theta0_vals.size, theta1_vals.size))\n",
" for t1, element in enumerate(theta0_vals):\n",
" for t2, element2 in enumerate(theta1_vals):\n",
" thetaT = np.matrix([1.0, element, element2]).reshape(3, 1)\n",
" J_vals[t1, t2] = JMx(thetaT, X, y) / rescale\n",
"\n",
" plt.figure()\n",
" plt.contour(theta0_vals, theta1_vals, J_vals.T, np.logspace(-2, 3, 20))\n",
" plt.xlabel(r\"$\\theta_1$\")\n",
" plt.ylabel(r\"$\\theta_2$\")\n"
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"outputs": [
{
"data": {
2022-10-18 13:52:38 +02:00
"image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"457.840312pt\" height=\"314.667469pt\" viewBox=\"0 0 457.840312 314.667469\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n <cc:Work>\n <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n <dc:date>2022-10-14T11:22:55.380282</dc:date>\n <dc:format>image/svg+xml</dc:format>\n <dc:creator>\n <cc:Agent>\n <dc:title>Matplotlib v3.6.1, https://matplotlib.org/</dc:title>\n </cc:Agent>\n </dc:creator>\n </cc:Work>\n </rdf:RDF>\n </metadata>\n <defs>\n <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n <g id=\"patch_1\">\n <path d=\"M 0 314.667469 \nL 457.840312 314.667469 \nL 457.840312 0 \nL 0 0 \nz\n\" style=\"fill: #ffffff\"/>\n </g>\n <g id=\"axes_1\">\n <g id=\"patch_2\">\n <path d=\"M 74.432812 277.111219 \nL 431.552813 277.111219 \nL 431.552813 10.999219 \nL 74.432812 10.999219 \nz\n\" style=\"fill: #ffffff\"/>\n </g>\n <g id=\"matplotlib.axis_1\">\n <g id=\"xtick_1\">\n <g id=\"line2d_1\">\n <defs>\n <path id=\"mf31e2b1581\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n </defs>\n <g>\n <use xlink:href=\"#mf31e2b1581\" x=\"74.432812\" y=\"277.111219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n </g>\n </g>\n <g id=\"text_1\">\n <!-- − −
2022-10-14 11:34:46 +02:00
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"contour_plot(XMx2, yMx2, rescale=10**10)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"Jeżeli funkcja kosztu ma kształt taki, jak na powyższym wykresie, to łatwo sobie wyobrazić, że znalezienie minimum lokalnego przy użyciu metody gradientu prostego musi stanowć nie lada wyzwanie: algorytm szybko znajdzie „rynnę”, ale „zjazd” wzdłuż „rynny” w poszukiwaniu minimum będzie odbywał się bardzo powoli.\n",
"\n",
"Jak temu zaradzić?\n",
"\n",
"Spróbujemy przekształcić dane tak, żeby funkcja kosztu miała „ładny”, regularny kształt."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Skalowanie"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"Będziemy dążyć do tego, żeby każda z cech przyjmowała wartości w podobnym zakresie.\n",
"\n",
"W tym celu przeskalujemy wartości każdej z cech, dzieląc je przez wartość maksymalną:\n",
"\n",
"$$ \\hat{x_i}^{(j)} := \\frac{x_i^{(j)}}{\\max_j x_i^{(j)}} $$"
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/latex": [
"$\\displaystyle 1.00 \\leq x_0 \\leq 1.00$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/latex": [
"$\\displaystyle 0.29 \\leq x_1 \\leq 1.00$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/latex": [
"$\\displaystyle 0.06 \\leq x_2 \\leq 1.00$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"XMx2_scaled = XMx2 / np.amax(XMx2, axis=0)\n",
"\n",
"show_mins_and_maxs(XMx2_scaled)\n"
]
},
{
"cell_type": "code",
"execution_count": 88,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
2022-10-18 13:52:38 +02:00
"image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"457.840312pt\" height=\"314.667469pt\" viewBox=\"0 0 457.840312 314.667469\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n <cc:Work>\n <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n <dc:date>2022-10-14T11:23:02.698988</dc:date>\n <dc:format>image/svg+xml</dc:format>\n <dc:creator>\n <cc:Agent>\n <dc:title>Matplotlib v3.6.1, https://matplotlib.org/</dc:title>\n </cc:Agent>\n </dc:creator>\n </cc:Work>\n </rdf:RDF>\n </metadata>\n <defs>\n <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n <g id=\"patch_1\">\n <path d=\"M 0 314.667469 \nL 457.840312 314.667469 \nL 457.840312 0 \nL 0 0 \nz\n\" style=\"fill: #ffffff\"/>\n </g>\n <g id=\"axes_1\">\n <g id=\"patch_2\">\n <path d=\"M 74.432812 277.111219 \nL 431.552813 277.111219 \nL 431.552813 10.999219 \nL 74.432812 10.999219 \nz\n\" style=\"fill: #ffffff\"/>\n </g>\n <g id=\"matplotlib.axis_1\">\n <g id=\"xtick_1\">\n <g id=\"line2d_1\">\n <defs>\n <path id=\"m68d512d3be\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n </defs>\n <g>\n <use xlink:href=\"#m68d512d3be\" x=\"74.432812\" y=\"277.111219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n </g>\n </g>\n <g id=\"text_1\">\n <!-- − −
2022-10-14 11:34:46 +02:00
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"contour_plot(XMx2_scaled, yMx2)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"### Normalizacja średniej"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"Będziemy dążyć do tego, żeby dodatkowo średnia wartość każdej z cech była w okolicach $0$.\n",
"\n",
"W tym celu oprócz przeskalowania odejmiemy wartość średniej od wartości każdej z cech:\n",
"\n",
"$$ \\hat{x_i}^{(j)} := \\frac{x_i^{(j)} - \\mu_i}{\\max_j x_i^{(j)}} $$"
]
},
{
"cell_type": "code",
"execution_count": 89,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
"text/latex": [
"$\\displaystyle 0.00 \\leq x_0 \\leq 0.00$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/latex": [
"$\\displaystyle -0.10 \\leq x_1 \\leq 0.62$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/latex": [
"$\\displaystyle -0.23 \\leq x_2 \\leq 0.70$"
],
"text/plain": [
"<IPython.core.display.Math object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"XMx2_norm = (XMx2 - np.mean(XMx2, axis=0)) / np.amax(XMx2, axis=0)\n",
"\n",
"show_mins_and_maxs(XMx2_norm)\n"
]
},
{
"cell_type": "code",
"execution_count": 90,
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"outputs": [
{
"data": {
2022-10-18 13:52:38 +02:00
"image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"457.840312pt\" height=\"314.667469pt\" viewBox=\"0 0 457.840312 314.667469\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n <cc:Work>\n <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n <dc:date>2022-10-14T11:23:08.721094</dc:date>\n <dc:format>image/svg+xml</dc:format>\n <dc:creator>\n <cc:Agent>\n <dc:title>Matplotlib v3.6.1, https://matplotlib.org/</dc:title>\n </cc:Agent>\n </dc:creator>\n </cc:Work>\n </rdf:RDF>\n </metadata>\n <defs>\n <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n <g id=\"patch_1\">\n <path d=\"M 0 314.667469 \nL 457.840312 314.667469 \nL 457.840312 0 \nL 0 0 \nz\n\" style=\"fill: #ffffff\"/>\n </g>\n <g id=\"axes_1\">\n <g id=\"patch_2\">\n <path d=\"M 74.432812 277.111219 \nL 431.552813 277.111219 \nL 431.552813 10.999219 \nL 74.432812 10.999219 \nz\n\" style=\"fill: #ffffff\"/>\n </g>\n <g id=\"matplotlib.axis_1\">\n <g id=\"xtick_1\">\n <g id=\"line2d_1\">\n <defs>\n <path id=\"m495bae0224\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n </defs>\n <g>\n <use xlink:href=\"#m495bae0224\" x=\"74.432812\" y=\"277.111219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n </g>\n </g>\n <g id=\"text_1\">\n <!-- − −
2022-10-14 11:34:46 +02:00
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"contour_plot(XMx2_norm, yMx2)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"Teraz funkcja kosztu ma wykres o bardzo regularnym kształcie –
]
}
],
"metadata": {
"celltoolbar": "Slideshow",
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
},
"livereveal": {
"start_slideshow_at": "selected",
"theme": "white"
},
"vscode": {
"interpreter": {
"hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
