387 lines
76 KiB
Plaintext
387 lines
76 KiB
Plaintext
|
{
|
||
|
|
"cells": [
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"# Algorytm najszybszego spadku dla regresji wielomianowej\n",
|
||
|
|
"\n",
|
||
|
|
"Algorytm przyjmuje zbiór danych - x oraz y i próbuje wyzaczyć funkcję wilomianową, która najlepiej przewiduje wartości y na podstawie x. Wynikiem jest wyznaczenie współczynników wielomianu.\n",
|
||
|
|
"\n",
|
||
|
|
"### Importy"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 98,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"from matplotlib import pyplot as plt\n",
|
||
|
|
"import numpy as np\n",
|
||
|
|
"import random"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Początkowe współczynniki\n",
|
||
|
|
"Tworzymy losowe początkowe współczynniki wielomianu - od nich algorytm rozpocznie dopasowanie. Oraz oryginalne współczynniki na podstawie których zostanie wyznaczony zbiór danych. Aby zmienić generowany zbiór danych należy zmienić tablicę coeffs."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 99,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"rand_coeffs = (random.randrange(-10, 10), random.randrange(-10, 10), random.randrange(-10,10))\n",
|
||
|
|
"coeffs = [2, -5, 4] # a, b, c"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Wyznaczenie wartości wielomianu\n",
|
||
|
|
"Funkcja na podstawie współczynników oraz x wyznacza wartość y wielomianu."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 100,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"def eval_2nd_degree(coeffs, x):\n",
|
||
|
|
" a = (coeffs[0]*(x*x))\n",
|
||
|
|
" b = coeffs[1]*x\n",
|
||
|
|
" c = coeffs[2]\n",
|
||
|
|
" y = a+b+c\n",
|
||
|
|
" return y\n",
|
||
|
|
" "
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Wartości wielomianu z szumem\n",
|
||
|
|
"Funkcja jest analogiczna do poprzedniej - wyznacza wartość wielomianu na podstawie wpółczynników oraz x, ale dodatkowo dodaje szum do wyjściowych wartości - funkcja zostanie użyta przy generowaniu danych."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 101,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"def eval_2nd_degree_jitter(coeffs, x, j):\n",
|
||
|
|
" y = eval_2nd_degree(coeffs, x)\n",
|
||
|
|
" \n",
|
||
|
|
" interval_min = y-j\n",
|
||
|
|
" interval_max = y+j\n",
|
||
|
|
" \n",
|
||
|
|
" return random.uniform(interval_min, interval_max)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Wygenerowanie danych\n",
|
||
|
|
"Kod generuje zbiór danych. Na podstawie wartości x od -10 do 10 i losowych współczynników wielomianu generuje wartości y z szumem. Parametr j określa jak moco dane będą zaszumione"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 102,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"name": "stdout",
|
||
|
|
"output_type": "stream",
|
||
|
|
"text": [
|
||
|
|
"[-2.75898766 -5.65016775 -2.64800795 -0.87530755 -8.01730191 -0.07202148\n",
|
||
|
|
" 5.28994963 -0.73685206 -8.69865485 7.11491828 3.53780935 5.6983085\n",
|
||
|
|
" -3.07830759 1.28764607 9.45509424 6.95589075 -9.90311757 -2.30741362\n",
|
||
|
|
" 0.85246282 -4.39966538 0.79658004 3.29939139 -1.77080971 -8.61248702\n",
|
||
|
|
" -8.08537204 -8.46573256 5.29314956 8.89274736 -5.489419 -9.54545965\n",
|
||
|
|
" 0.57426227 1.98898479 0.28600353 -3.3779621 -6.74739724 -9.74245438\n",
|
||
|
|
" 6.70940724 -8.87576881 2.22682743 0.5484296 -5.13845562 -6.39800992\n",
|
||
|
|
" 3.76662938 0.84736151 1.02810373 6.79544869 2.20334512 3.65087577\n",
|
||
|
|
" -9.40928261 -0.51270946 9.95897626 4.22555829 3.24859186 7.78895395\n",
|
||
|
|
" 2.92230082 2.48586306 -7.51038001 4.53569191 1.27276562 0.75734371\n",
|
||
|
|
" -3.47811458 -2.76641217 0.3687908 5.73108176 -3.10855301 2.15342573\n",
|
||
|
|
" 9.14193125 9.51722955 -0.73921274 9.39733669 -8.08203669 -1.68282233\n",
|
||
|
|
" 0.63287602 2.9995979 -0.17714689 1.99519815 3.651222 3.90420391\n",
|
||
|
|
" 4.96344081 9.64202473 8.79805941 -5.23101012 -9.18668126 -9.79448374\n",
|
||
|
|
" -5.26154655 6.52710444 -7.58126901 -2.83931574 -9.23639541 4.23583082\n",
|
||
|
|
" -3.42590506 -3.22933046 -5.75833418 8.25154698 -8.96797588 -0.79300038\n",
|
||
|
|
" 0.30593884 -6.6657409 -6.58569143 -9.75881443]\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABIYAAAJOCAYAAADChAzjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAzFElEQVR4nO3dfbRs+VkX+O9jGlABSSBtbDqRzmBAAzMGrUnAt3vHBBIyYoNLmcYZDA6uoBKWuHAkwdHaBaIBQRbqGA2SZSIvISJIjPElRE6zmEXAc2MT8gKTBjqTbpvkQt4HJ07CM3+cuqcPt8+997xUnV1V+/NZ66yza+9dVc85u3a9fOv3Ut0dAAAAAKbnN41dAAAAAADjEAwBAAAATJRgCAAAAGCiBEMAAAAAEyUYAgAAAJgowRAAAADARAmGAICdVFXfUFX/ZNX7nuC2uqp+1wn3Harqe1ZxvwAAZ3Hb2AUAANxKVX1Fkq9L8ulJPpDkh5O8qLvfd6PrdPffOuntn2bfsVTVP03yYHf/72PXAgDsDi2GAICNVlVfl+RbkvxvST4pyecm+bQkr6uqj73BdXz5BQBwAoIhAGBjVdVvS7JI8jXd/W+7+//r7geSfGmSu5L8L8v9hqr6war6nqr6QJKvuL6bVlX9map6R1X9alX99ap6oKqedeT637NcvmvZHex5VfV/V9WvVNVfO3I7T6+qn6yq91XVw1X1D24UUB3z9zy5qu6tqg9W1euSPP667f+8qn65qt5fVT9eVZ+1XP/8JP9zkr9aVR+qqn+1XP/CqvqF5e29taq+5Ez/aABgsgRDAMAm+wNJfnOSHzq6srs/lOS1ST7/yOq7k/xgkscm+d6j+1fVU5P8wxyEK3fkoOXRnbe47z+U5DOTPDPJ36iq37Nc/9EkfzkHoc7nLbf/xRP+Pd+X5Mryut+U5HnXbf83SZ6S5LcneeO1v6O7X7pc/tbu/oTu/qLl/r+Q5A8v/55Fku+pqjtOWAsAgGAIANhoj0/yK939kWO2PZzf2OLmJ7v7X3b3r3f3f7lu3z+Z5F919090939N8jeS9C3ue9Hd/6W7fybJzyT5vUnS3Ve6+w3d/ZFl66V/nOTSrf6QqvqdSf77JH+9uz/c3T+e5F8d3ae7X9bdH+zuDycZkvzeqvqkG91md//z7v7Py7/5B5K8PcnTb1ULAMA1giEAYJP9SpLH32DMoDuW2695501u51OPbu/uX0vyq7e4718+svxrST4hSarqM6rqNcsuXx9I8rdyXZewm9Tw3u7+f46se8e1hap6TFW9eNk17ANJHlhuuuFtL7vH3bfs1va+JJ99wloAAJIIhgCAzfaTST6c5E8cXVlVn5DkC5O8/sjqm7UAejjJE49c/7ck+ZQz1vSSJD+X5Cnd/duSfEOSOsH1Hk7yuKr6+CPrfueR5T+dg+5wz8pB17C7rpW7/P0b/r6q+rQk35XkBUk+pbsfm+TNJ6wFACCJYAgA2GDd/f4cjJ3z96vqOVX1MVV1V5JXJXkwyT874U39YJIvqqo/sBwoesjZA5RPTPKBJB+qqt+d5C+c5Erd/Y4k+0kWVfWxVfWHknzRkV0+MQch2K8m+a05aIl01LuS/DdHLn98DsKiq0lSVX82By2GAABOTDAEAGy07v7WHLTK+bYcBDI/lYNuYc9cjsVzktt4S5KvSfLKHLTc+VCSd+cgiDmtv5KD1j0fzEGLnR84xXX/dJJnJHlPknmSVxzZ9oocdC17KMlbk7zhuut+d5KnLruN/cvufmuSb89Bq6p3Jflvk/yfp/5rAIBJq+5bjbsIALBbll3R3peD7mC/NHI5AACj0WIIAJiEqvqiqvqtyzF+vi3Jz+aRAZ4BACZJMAQATMXdSf7z8ucpSe5pTacBgInTlQwAAABgorQYAgAAAJio28Yu4KjHP/7xfdddd41dBgAAAMDOuHLlyq909+3HbduoYOiuu+7K/v7+2GUAAAAA7IyqeseNtulKBgAAADBRJw6Gquo3V9VPV9XPVNVbqmqxXP/kqvqpqrq/qn6gqj52uf7jlpfvX26/a01/AwAAAABncJoWQx9O8ke7+/cmeVqS51TV5yb5liTf0d2/K8l7k3zlcv+vTPLe5frvWO4HAAAAwIY4cTDUBz60vPgxy59O8keT/OBy/cuTfPFy+e7l5Sy3P7Oq6rwFAwAAALAapxpjqKoeU1X3JXl3ktcl+YUk7+vujyx3eTDJncvlO5O8M0mW29+f5FOOuc3nV9V+Ve1fvXr1TH8EAAAAAKd3qmCouz/a3U9L8sQkT0/yu89bQHe/tLtn3T27/fZjZ04DAAAAYA3ONCtZd78vyY8l+bwkj62qa9PePzHJQ8vlh5I8KUmW2z8pya+ep1gAAAAAVuc0s5LdXlWPXS7/liSfn+RtOQiI/uRyt+cl+ZHl8quXl7Pc/h+6u1dQMwAAAAArcNutdzl0R5KXV9VjchAovaq7X1NVb03yyqr6m0n+U5LvXu7/3Un+WVXdn+Q9Se5ZYd0AAAAAnNOJg6HuflOSzzlm/S/mYLyh69f/v0n+1LmqAwAAAGBtzjTGEAAAAADbTzAEAAAAMFGCIQAAAICJEgwBAAAATJRgCAAAAGCiBEMAAAAAEyUYAgAAAJgowRAAAADARAmGAAAAACZKMAQAAAAwUYKhEQx7w9glAAAAAAiGxrC4dzF2CQAAAACCIQAAAICpEgxdkGFvSC0qtagkOVzWrQwAAAAYS3X32DUcms1mvb+/P3YZa1eLSs835/8OAAAA7K6qutLds+O2aTEEAAAAMFGCoRHML83HLgEAAABAMDSG4fIwdgkAAAAAgiEAAACAqRIMAQAAAEyUYAgAAABgogRDAAAAABMlGAIAAACYKMEQAAAAwEQJhgAAAAAmSjAEAAAAMFGCIQAAAICJEgwBAAAATJRgCAAAAGCiBEMAAAAAEyUYAgAAAJgowRAAAADARAmGAAAAACZKMAQAAAAwUYIhAAAAgIkSDAEAAABMlGAIAAAAYKIEQwAAAAATJRgCAAAAmCjBEAAAAMBECYYAAAAAJkowBAAAADBRgiEAAACAiRIMAQAAAEyUYAgAAABgogRDAAAAABMlGAIAAACYKMEQAAAAwEQJhgAAAAAmSjA0kmFv2Kn7AQAAALaPYGgki3sXO3U/AAAAwPYRDAEAAABMlGDoAg17Q2pRqUUlyeHyqrt7XdT9AAAAANutunvsGg7NZrPe398fu4wLUYtKz9f/v7+o+wEAAAA2U1Vd6e7Zcdu0GAIAAACYKMHQSOaX5jt1PwAAAMD20ZUMAAAAYIfpSgYAAADAowiGAAAAACZKMAQAAAAwUYIhAAAAgIkSDAEAAABMlGAIAAAAYKIEQwAAAAATJRgCAAAAmCjBEAAAAMBEnTgYqqonVdWPVdVbq+otVfWXluuHqnqoqu5b/jz3yHVeVFX3V9XPV9Wz1/EHAAAAAHA2t51i348k+brufmNVfWKSK1X1uuW27+jubzu6c1U9Nck9ST4ryacm+dGq+ozu/ugqCgcAAADgfE7cYqi7H+7uNy6XP5jkbUnuvMlV7k7yyu7+cHf/UpL7kzz9PMUCAAAAsDpnGmOoqu5K8jlJfmq56gVV9aaqellVPW657s4k7zxytQdzTJBUVc+vqv2q2r969epZygEAAADgDE4dDFXVJyT5F0m+trs/kOQlST49ydOSPJzk209ze9390u6edffs9ttvP205AAAAAJzRqYKhqvqYHIRC39vdP5Qk3f2u7v5od/96ku/KI93FHkrypCNXf+JyHQAAAAAb4DSzklWS707ytu7+u0fW33Fkty9J8ubl8quT3FNVH1dVT07ylCQ/ff6SAQAAAFiF08xK9geTfHmSn62q+5brviHJl1XV05J0kgeSfFWSdPdbqupVSd6agxnNvtqMZAAAAACb48TBUHf/RJI6ZtNrb3Kdb07yzWeoCwAAAIA1O9OsZAAAAABsP8EQAAAAwEQJhgAAAAAmSjAEAAAAMFGCIQAAAICJEgwBAAAATJRgCAAAAGCiBEMAAAAAEyUYAgAAAJgowRAAAADARAmGAAAAACZ
|
||
|
|
"text/plain": [
|
||
|
|
"<Figure size 1440x720 with 1 Axes>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"metadata": {
|
||
|
|
"needs_background": "light"
|
||
|
|
},
|
||
|
|
"output_type": "display_data"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"hundred_xs=np.random.uniform(-10,10,100)\n",
|
||
|
|
"print(hundred_xs)\n",
|
||
|
|
"\n",
|
||
|
|
"j=50\n",
|
||
|
|
"x_y_pairs = []\n",
|
||
|
|
"for x in hundred_xs:\n",
|
||
|
|
" y = eval_2nd_degree_jitter(coeffs, x, j)\n",
|
||
|
|
" x_y_pairs.append((x,y))\n",
|
||
|
|
" \n",
|
||
|
|
"xs = []\n",
|
||
|
|
"ys = []\n",
|
||
|
|
"for a,b in x_y_pairs:\n",
|
||
|
|
" xs.append(a)\n",
|
||
|
|
" ys.append(b)\n",
|
||
|
|
" \n",
|
||
|
|
"plt.figure(figsize=(20,10))\n",
|
||
|
|
"plt.plot(xs, ys, 'g+')\n",
|
||
|
|
"plt.title('Original data')\n",
|
||
|
|
"plt.show()"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Funkcja straty\n",
|
||
|
|
"Do określenia jak mocno przewidziane wartości y są różne zostanie użyta kwadratowa funkcja straty."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 103,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"def loss_mse(ys, y_bar):\n",
|
||
|
|
" return sum((ys - y_bar)*(ys - y_bar)) / len(ys)\n"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Gradient\n",
|
||
|
|
"Funkcja przyjmuje współczynniki wielomianu, x, y oraz parametr prękości uczenia i wylicza za pomocą gradientu nowe wartości współczynników wielomianu."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 104,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"def calc_gradient_2nd_poly_for_GD(coeffs, inputs_x, outputs_y, lr): \n",
|
||
|
|
" a_s = []\n",
|
||
|
|
" b_s = []\n",
|
||
|
|
" c_s = []\n",
|
||
|
|
" \n",
|
||
|
|
" y_bars = eval_2nd_degree(coeffs, inputs_x)\n",
|
||
|
|
"\n",
|
||
|
|
" for x,y,y_bar in list(zip(inputs_x, outputs_y, y_bars)): # take tuple of (x datapoint, actual y label, predicted y label)\n",
|
||
|
|
" x_squared = x**2 \n",
|
||
|
|
" partial_a = x_squared * (y - y_bar)\n",
|
||
|
|
" a_s.append(partial_a)\n",
|
||
|
|
" partial_b = x * (y-y_bar)\n",
|
||
|
|
" b_s.append(partial_b)\n",
|
||
|
|
" partial_c = (y-y_bar)\n",
|
||
|
|
" c_s.append(partial_c)\n",
|
||
|
|
" \n",
|
||
|
|
" num = [i for i in y_bars]\n",
|
||
|
|
" n = len(num)\n",
|
||
|
|
" \n",
|
||
|
|
" gradient_a = (-2 / n) * sum(a_s)\n",
|
||
|
|
" gradient_b = (-2 / n) * sum(b_s)\n",
|
||
|
|
" gradient_c = (-2 / n) * sum(c_s)\n",
|
||
|
|
"\n",
|
||
|
|
"\n",
|
||
|
|
" a_new = coeffs[0] - lr * gradient_a\n",
|
||
|
|
" b_new = coeffs[1] - lr * gradient_b\n",
|
||
|
|
" c_new = coeffs[2] - lr * gradient_c\n",
|
||
|
|
" \n",
|
||
|
|
" new_model_coeffs = (a_new, b_new, c_new)\n",
|
||
|
|
" \n",
|
||
|
|
" #update with these new coeffs:\n",
|
||
|
|
" new_y_bar = eval_2nd_degree(new_model_coeffs, inputs_x)\n",
|
||
|
|
" \n",
|
||
|
|
" updated_model_loss = loss_mse(outputs_y, new_y_bar)\n",
|
||
|
|
" return updated_model_loss, new_model_coeffs, new_y_bar\n",
|
||
|
|
" \n",
|
||
|
|
" "
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Minimalizacja\n",
|
||
|
|
"Funkcja powtarza proces wyznaczenia nowych współczynników wielomianu zadaną ilość razy - epoch."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 105,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [],
|
||
|
|
"source": [
|
||
|
|
"def gradient_descent(epochs, lr):\n",
|
||
|
|
" losses = []\n",
|
||
|
|
" rand_coeffs_to_test = rand_coeffs\n",
|
||
|
|
" for i in range(epochs):\n",
|
||
|
|
" loss = calc_gradient_2nd_poly_for_GD(rand_coeffs_to_test, hundred_xs, ys, lr)\n",
|
||
|
|
" rand_coeffs_to_test = loss[1]\n",
|
||
|
|
" losses.append(loss[0])\n",
|
||
|
|
" #print(losses)\n",
|
||
|
|
" return loss[0], loss[1], loss[2], losses #(updated_model_loss, new_model_coeffs, new_y_bar, saved loss updates)"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Uruchomienie\n",
|
||
|
|
"Wartości wyznaczonego wielomionu zostają wyświetlone na wykresie z porównaniem do zbioru danych."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 106,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABIYAAAJOCAYAAADChAzjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABTgElEQVR4nO39e5jkZ1kn/r/vTDLJKkgwHASCBFxEkEPAJtAopnGQkyggrsKyBnQxIKCLF/tbiX5DV5PVYV0UZVUgKEoUFSWAyKIcZqkI2IAdDGdcgsKSGEMIZzCZZPL8/vhUTzqTnp4+1+n1uq6+6vSpqqfOXe+67+ep1loAAAAAmD7HDXsAAAAAAAyHYAgAAABgSgmGAAAAAKaUYAgAAABgSgmGAAAAAKaUYAgAAABgSgmGAOAoquqXqur3tnvbdVxWq6p/vx2XtcHrHfvbMLjuflU9fbD/KVX1tk1ezl9X1VO3d3Trut4nVNVnq+prVXX/3b7+7VJVpw2eB8cPDg/l/gQA1iYYAmAqVNXTqurDVfWNqvrXqnpZVZ281nlaa7/aWnv6ei5/I9sO2+C+ODQIHpb/fnucbsN6tdZe01p7xLG2q6peVf3xEed9dGvt1Ts3uqN6cZLntNZukeSLK8OV7VJVe6vqBVX1j1X19aq6fBDcHPO+2qztuj8Hz993H2ObflVdU1VfraqvVNXFVfX8qjpxq9e/E6pqrqouG/Y4AJhOgiEAJl5VPS/J/0jy/0tyqyQPTnKXJG+vqr1HOc+2fhEfQYuttVus+HvOsAe0mil4HFZzlyQf3Y4LWuP+e12SxyU5K8mtk9w1yW8l+aENXs4oe05r7ZZJ7pDkeUmelOQtVVXDHRYAjBbBEAATraq+JclCkp9rrf1Na+261tqnk/x4ktOS/KfBdr2qel1V/XFVfSXJ046sIqmqs6rqM1V1dVWdW1WfrqqHrzj/Hw/2L7fQPLWq/l9Vfb6qfnnF5ZxRVYtV9aWquqKqfvtoAdURt+UnqmrpiON+oareNNj/mKr62KBK4vKq+q8bvK92/DYMztuvqv1V9f5BNcdfVtW3HnG9/7mq/l+S/zM4/qer6uNV9cWqemtV3WXF5f1gVX2iqr5cVb+dpFacdpPqkqr67qp6e1V9oaqurK597lFJfinJTwyqpz64YpzLLWnHVdX/N3j8P1dVF1TVrdZzX61y+3+oqv5hcNs/W1W9wfEnVtXXkuxJ8sGq+lSSvx2c7UuDsc2u4/5oVfXsqvpkkk+ucv0PT/KDSR7XWntfa+3g4O9vWmv/ZcV2n66qX6yqDyX5elUdX13VzacGz7GPVdUTVmy/p6pePLj9/5QjQqaV9+c6b8Mzq+qTg+fY71TnnklenmR2cH986Wj387LW2tdba/0kP5Jkdnlcg8d0+fZcXVV/vuJ5eFJ17wVXD67/76vq9oPTvrWq/qCq/mUw9jeuGPdjq+qSwXn+rqrue8T9+V+r6kOD5+prB9fzzUn+Oskd68YKvjse63YBwHYRDAEw6R6S5KQkr195ZGvta0neku4L8rLHpaukODnJa1ZuX1X3SvK7SZ6SrgLhVknudIzr/r4k90iyL8kLBl9qk+RQkl9Icpt0X1T3JXnWOm7LXyW5R1XdfcVx/zHJnwz2/36SZwyqJO6dQaiyRdt9G5adleSn092X1yd56RGnn5nknkkeWVWPSxfc/GiS2yZ5V5I/TZKquk26x/b/G4zlU0m+d7UrrKpbJnlHkr9Jcsck/z7Jgdba3yT51SSvHVRP3W+Vsz9t8PewJHdLcoskv33ENke7r4709cHtPzldSPGzVfX41tq1g/axJLlfa+07knz/4PDJg7EtrnV/rPD4JA9Kcq9Vrv/hSd7XWltP69KTB2M8ubV2fbr796Hpnv8LSf64qu4w2PZnkjw2yf2TzCT5saNd6Dpvw2OTPDDJfdMFuY9srX08yTNzY8Xbyeu4DUmS1tr/S7I0GH+S/Fy6++nMdM+HLyb5ncFpTx3cxjsnOWVwnf82OO2PknxTku9OcrskLxncpvsneVWSZwzO84okb6qbtq/9eJJHpavQum+Sp7XWvp7k0Un+ZUUF37+s93YBwFYJhgCYdLdJ8vnBl9ojXTE4fdlia+2NrbUbWmv/dsS2P5bkr1pr726tHUzygiTtGNe90Fr7t9baB5N8MMn9kqS1dnFr7b2ttesH1UuvSPfldE2ttW8k+ct0X9YzCIi+K8mbBptcl+ReVfUtrbUvttY+sMbFPXhQ1bD89+DduA0r/FFr7SODL8XnJvnxqtqz4vTeoNLj39J9Kd/fWvv44HH81SSnDypMHpPko62117XWrkvym0n+9SjX+dgk/9pa+/XW2jWtta+21t63zvE+JclvtNb+aRAqnpPkSXXTFqtV76sjtdb6rbUPD55nH0oXiGzkvlvr/li2v7X2hVWex0n3nD98Hw0qYL40qGK55ohtX9pa++zy5bTW/qK19i+Dsb82XUXSGYNtfzzJbw62/0KS/Vu8DS9qrX1pEOi8M8npa98t6/IvSb51xRh+ubV2WWvt2iS9JD82eEyvSxfu/PvW2qHB8/0rgxDs0UmeOXiNXddau2hweWcnecWgCuvQYD6la9O1ri576eD++0K6oHc7bhMAbIlgCIBJ9/kkt6nV50i5w+D0ZZ9d43LuuPL0QUhz9TGue2VA8Y10VSapqu+sqjdXNwn2V9J9Kb7Nahewij/JIBhKVy30xsFYkuSJ6YKSz1TVRcttR0fx3tbaySv+3ruLtyG56X39mSQnHHH+laffJclvLYdYSb6Qrl3sTrn549Jy9MfxzukqXjbjjoNxrhzz8Uluv+K4Ve+rI1XVg6rqnVV1VVV9OV1AsZH7bq37Y9laz+Wr0z33kySDAOnkJN+T5MjJmW9yOdW1U16y4rrvvWLsd8zNH9et3IZ13Z8bdKfBdS2P4Q0rxvDxdJVwt09XFfTWJH82aBn7tao6Id1z6AuttS8e5TY9b2XgOth+ZVvYTtwmANgSwRAAk24x3a/2P7ryyKq6Rbpf/g+sOHqtCqArkpy64vz/Ll1FwWa8LMknkty9tfYt6Vpq1jsh7tuT3LaqTk8XEC23kaW19vettcela295Y5I/3+T41mMrtyHpvjAv+/Z0FRorQ7qVj8Vn07XIrQyy/l1r7e/SPS6HL6uq6ojLzhGXc7ejnHas6q9/SffFf+WYr09y5THOt5o/SVfldefW2q3SzZlztPtutXGtdX+sdb5lB5I8sKpOXWObm13OoJrnlUmek+SUQZj0kRVjv8ljke4+Opr13IZjjmkjqurO6cKvd60Yw6OPGMNJrbXLB5VAC621e6VrR31suva/zyb51lp9RcPPJvmVIy7vm1prR7bIbdttAoDtIBgCYKK11r6cbi6U/1VVj6qqE6rqtHShyWXpKgPW43VJfriqHlLdJMu9bCwIWemWSb6S5GtV9V1Jfna9Zxy0S/1Fkv+ZriXm7cnh5cefUlW3GmzzlSQ3bHJ867Hp2zDwn6rqXlX1TUlemOR1rbVDR9n25UnOqarvTpKqulVV/YfBaf87yXdX1Y8OqsJ+Psm3HeVy3pzkDlX13Oomer5lVT1ocNqVSU6rqqP9b/SnSX6hqu46CBWX5yRarUXxWG6Zrurkmqo6I13l19Fcle5xXBlorXV/HFNr7W3pWrPeOKhe2juohjlaO+Gyb04XYFw1uN6fSlcxtOzPk/x8VZ1aVbdO8vw1Lmsrt+HKJKfW+ic7/6aqOjNdG+b7080ttjyGX1luX6uq2w7mPkpVPayq7jNob/xKuuDyhtbaFekmiv7dqrr14P1keR6oVyZ55uA+rar65uomGr/lOm/TKTWY0BwAdpNgCICJ11r7tXQVLS9O9yXvfel+3d83mFtkPZfx0XST1f5ZusqIryX5XLpqpI36r+nCgK+m+zL52g2e/0/STSD8F0cEEz+Z5NOD1q5nppsXZ6ds9Tb8UZI/TNdac1K6QGdVrbU3JPkf6dp6vpKuSuXRg9M+n+Q/JHlRuhapuyd5z1Eu56vpJhv/4cH1fjLdZNJJF7YlydVVtdrcTK8ajPlvk/xzkmvSPR8
|
||
|
|
"text/plain": [
|
||
|
|
"<Figure size 1440x720 with 1 Axes>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"metadata": {
|
||
|
|
"needs_background": "light"
|
||
|
|
},
|
||
|
|
"output_type": "display_data"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"GD = gradient_descent(1500, 0.0001)\n",
|
||
|
|
"\n",
|
||
|
|
"plt.figure(figsize=(20,10))\n",
|
||
|
|
"plt.plot(xs, ys, 'g+', label = 'original')\n",
|
||
|
|
"plt.plot(xs, GD[2], 'b.', label = 'final_prediction')\n",
|
||
|
|
"plt.title('Original vs Final prediction after Gradient Descent')\n",
|
||
|
|
"plt.legend(loc=\"lower right\")\n",
|
||
|
|
"plt.show()"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Wyznaczone współczynniki wielomianu"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 107,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"name": "stdout",
|
||
|
|
"output_type": "stream",
|
||
|
|
"text": [
|
||
|
|
"Początkowe współczynniki (-5, -5, 4)\n",
|
||
|
|
"Wyznaczone współczynniki (1.9655100088406912, -4.317400439992125, 4.040348505930957)\n",
|
||
|
|
"Oryginalne współczynniki [2, -5, 4]\n"
|
||
|
|
]
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"\n",
|
||
|
|
"print(\"Początkowe współczynniki {}\".format(rand_coeffs))\n",
|
||
|
|
"print(\"Wyznaczone współczynniki {}\".format(GD[1]))\n",
|
||
|
|
"print(\"Oryginalne współczynniki {}\".format(coeffs))"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "markdown",
|
||
|
|
"metadata": {},
|
||
|
|
"source": [
|
||
|
|
"### Funkcja straty\n",
|
||
|
|
"Wykres przedstawia jak zmianiała się wartość funkcji straty w kolejnych krokach algorytmu."
|
||
|
|
]
|
||
|
|
},
|
||
|
|
{
|
||
|
|
"cell_type": "code",
|
||
|
|
"execution_count": 108,
|
||
|
|
"metadata": {},
|
||
|
|
"outputs": [
|
||
|
|
{
|
||
|
|
"data": {
|
||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABJ4AAAJcCAYAAAC4425vAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA57ElEQVR4nO3dffyldV0n/tcbGMQCBZVIgV9gUoZuoo2IuZmrG6BW2NYvpTZRSWpTS3PXvGlXM/2t1W627qq7lKzQqujDNFnDG37mrrklMiIgYOYs6jKEMnLjbYrAe/8419hxnBuIOfO9PjPP5+NxHt9zPtd1rvO5Zg6Hmde8Ptep7g4AAAAA7Gr7rPUEAAAAANgzCZ4AAAAAWAnBEwAAAAArIXgCAAAAYCUETwAAAACshOAJAAAAgJUQPAEA7CWq6stVdd81fP0fqapPrNXrAwC7n+AJANitqurTVfVP13oeu1NV/XZVfayqbqmql2y17VFVddsUCm25nba0/R5V9faq+kpVfaaqfm6r5//cNP6VqvrTqrrH9ubR3Qd291XT815fVS/bxaf6Laqqq+p+S6//F939/at8TQBgXgRPAAC7SFXtt51NG5M8L8mfbWf7306h0Jbb2UvbXp3k5iSHJfn5JK+tqgdMr/eAJP8lyS9M27+a5DV3/kx2bgfnCgDwTYInAGAWquouVfUHVfW30+0Pquou07Z7VdU7q+qmqrqhqv6iqvaZtv1GVV1TVV+qqk9U1WO2c/y7V9U5VbV5agj9ZlXtM73uTVX1wKV9D62qv6uq75oe/3hVXTLt95dV9YNL+356msNlSb6yrUCmu8/u7ncl+dId/DX5ziQ/neRfd/eXu/uDSc7LImhKFkHUf+/uD3T3l5P86yT/rKoO2s7xuqruV1VnTM993tSw+u/T9vtU1Z9Mv0afqqpfXXruS6rqrVX136rqi0meUlXHV9VfTb8u11bVf6qq/af9PzA99dLpNZ44tbs2LR3zB6rqf0zPv6KqfnJp2+ur6tVV9WfT7+2FVfW907aqqldW1XVV9cWpTfbN3z8AYD4ETwDAXLwoyQlJjkvyoCTHJ/nNadtzk2xKcmgWzZ4XJumq+v4kz0zy0O4+KMlJST69neP/xyR3T3LfJD+a5MlJntrdX0/ytiSnLu37s0n+Z3dfV1UPTnJWkl9Kcs8sGkbnbQnFJqcmeXySg7v7ln/AuX9XVX1uCnteOQVOSfJ9SW7p7r9Z2vfSJA+Y7j9gepwk6e7/nUU76vt29GLdfWaSNyT53alh9RNTkPffp+MdnuQxSZ5dVSctPfWUJG9NcvD0/FuTPCfJvZI8fHrOr0yv8cjpOQ+aXuPNy3OoqnXT6703yXcleVaSN0y/p1s8KclvJTkki9bYy6fxE5M8cjrPu2fx+3X9js4ZAFgbgicAYC5+PslLu/u67t6cReCwpdnzjST3TvI93f2N6VpBnUXwcZckx1bVuu7+9BS+fIuq2jeLEOMF3f2l7v50kn+/dPw3Ttu3+LlpLEnOSPJfuvvC7r51Wgb39SxCsi1e1d1Xd/ff/QPO+6+zCNvuneTRSX4oye9P2w5M8sWt9v9CkoOWtn9hB9vviIcmObS7X9rdN0/XgvrDfOuvy1919592923d/Xfd/ZHu/lB33zL9mv6XLEK92+OEaf6vmF7vz5O8M98aAL69uz88hXlvyOLXKVm8Hw5Kcv8k1d0f7+5r/wHnDACsmOAJAJiL+yT5zNLjz0xjSfJ7WTRe3ltVV1XV85OkuzcmeXaSlyS5rqrOrar75NvdK8m6bRz/8On++5N8R1U9rKqOyiLgePu07XuSPHdaDnZTVd2U5MiluSXJ1Xf0ZLfo7s9295VTmPOpLK4F9dPT5i8nudtWT7lb/n7J3s623xHfk+Q+W53nC7NomG3xLedZVd83LYH87LT87v/L4tf69rhPkqu7+7alseXfkyT57NL9r2YRVGUKqf5TFte/uq6qzqyqrX8dAIAZEDwBAHPxt1mEH1v8P9NYppbSc7v7vkl+Msmvb7mWU3e/sbv/8fTcTvI72zj257NoyWx9/GumY9ya5C1ZtG1OTfLO7t4S3lyd5OXdffDS7Tu6+01Lx+o7c+Jb6fz9n9H+Jsl+VXXM0vYHJbliun/F9DhJUlX3zaIBtrw0b0evs+zqJJ/a6jwP6u7H7eA5r82isXVMd98ti6CqbsdrJ4vf2yO3XKtr8s3fk51OvvtV3f1DSY7NYsndv7qdrwsA7EaCJwBgLayrqgOWbvsleVOS35wu7H2vJP8myX9Lvnlx7/tVVWWxlOzWJLdV1fdX1aOn6y19LcnfJblt6xdbCpZeXlUHVdX3JPn1LcefvDHJE7NY8vfGpfE/TPLLUxuqquo7q+rx27uA97ZU1bqqOiCLP3vtN53zvtO2f1JV3zMd+8gkr0jyjmneX8ni+lMvnV73EVlcZ+mPp0O/IclPVNWPTNeFemmSty2FZjvyuSyud7XFh5N8abpQ+l2rat+qemBVPXQHxzgoi6WAX66q+yf5Fzt5jWUXZtFiet706/OoJD+R5NydTbyqHjr9fqxL8pUsfu+/7fcdAFh7gicAYC2cn0VItOX2kiQvS7IhyWVJPpbk4mksSY5J8v9nsbTsr5K8prvfn0W75xVZNJo+m8VFql+wndd8VhYhxVVJPphFuHTWlo3dfeG0/T5J3rU0viHJ07NY2nVjFkv+nnIHz/cPp/M8NYuLqP9d/v76Ug9O8pfTa//ldO6/uvTcX0ly1yTXZRHO/YvuvmKa2xVJfjmLAOq6LIKgX7mdc3pdFtfGuqmq/nQK5348i2WGn8ri1/SPsrh49/b8yyyuh/Wl6RzfvNX2lyQ5e3qNn13e0N03ZxE0PXZ6rdckeXJ3//XtmPvdpte7MYvleddnsRwTAJiZWlyXEwAAAAB2LY0nAAAAAFZC8AQAAADASgieAAAAAFgJwRMAAAAAK7HfWk9gd7vXve7VRx111FpPAwAAAGCP8ZGPfOTz3X3o1uN7XfB01FFHZcOGDWs9DQAAAIA9RlV9ZlvjltoBAAAAsBKCJwAAAABWQvAEAAAAwEoIngAAAABYCcETAAAAACsheAIAAABgJQRPAAAAAKyE4AkAAACAlRA8AQAAALASgicAAAAAVkLwBAAAAMBKCJ4AAAAAWImVBU9VdUBVfbiqLq2qK6rqt6bx11fVp6rqkul23DReVfWqqtpYVZdV1UOWjnVaVX1yup22NP5DVfWx6Tmvqqpa1fkAAAAAcMfst8Jjfz3Jo7v7y1W1LskHq+pd07Z/1d1v3Wr/xyY5Zro9LMlrkzysqu6R5MVJ1ifpJB+pqvO6+8Zpn6cnuTDJ+UlOTvKuAAAAALDmVtZ46oUvTw/XTbfewVNOSXLO9LwPJTm4qu6d5KQkF3T3DVPYdEGSk6dtd+vuD3V3JzknyRNWdT4AAAAA3DErvcZTVe1bVZckuS6L8OjCadPLp+V0r6yqu0xjhye5eunpm6axHY1v2sb4tuZxRlVtqKoNmzdvvrOnBQAAAMDtsNLgqbtv7e7jkhyR5PiqemCSFyS5f5KHJrlHkt9Y5RymeZzZ3eu7e/2hhx666pcDAAAAILvpW+26+6Yk709ycndfOy2n+3qS/5rk+Gm3a5IcufS0I6axHY0fsY1xAAAAAGZgld9qd2hVHTzdv2uSH0vy19O1mTJ9A90Tklw+PeW8JE+evt3uhCRf6O5rk7wnyYlVdUhVHZLkxCTvmbZ9sapOmI715CTvWNX5AAAAAHDHrPJb7e6d5Oyq2jeLgOst3f3Oqvrzqjo0SSW5JMkvT/ufn+RxSTYm+WqSpyZJd99QVb+d5KJpv5d29w3T/V9J8vokd83i2+x8ox0AAADATNTiC+H2HuvXr+8NGzas9TQAAAAA9hhV9ZHuXr/1+G65xhMAAAAAex/B04De+97k/vdP/uZv1nomAAAAANsneBrQl76UfOITyde/vtYzAQAAANg+wdPA9rLLcwEAAACDETwNqGqtZwAAAACwc4KngWk8AQAAAHMmeBrQlsaT4AkAAACYM8HTgCy1AwAAAEYgeBqYxhMAAAAwZ4K
|
||
|
|
"text/plain": [
|
||
|
|
"<Figure size 1440x720 with 1 Axes>"
|
||
|
|
]
|
||
|
|
},
|
||
|
|
"metadata": {
|
||
|
|
"needs_background": "light"
|
||
|
|
},
|
||
|
|
"output_type": "display_data"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source": [
|
||
|
|
"plt.figure(figsize=(20,10))\n",
|
||
|
|
"plt.plot(GD[3], 'b-', label = 'loss')\n",
|
||
|
|
"plt.title('Loss over 1500 iterations')\n",
|
||
|
|
"plt.legend(loc=\"lower right\")\n",
|
||
|
|
"plt.xlabel('Iterations')\n",
|
||
|
|
"plt.ylabel('MSE')\n",
|
||
|
|
"plt.show()"
|
||
|
|
]
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"metadata": {
|
||
|
|
"kernelspec": {
|
||
|
|
"display_name": "Python 3.8.9 64-bit",
|
||
|
|
"language": "python",
|
||
|
|
"name": "python3"
|
||
|
|
},
|
||
|
|
"language_info": {
|
||
|
|
"codemirror_mode": {
|
||
|
|
"name": "ipython",
|
||
|
|
"version": 3
|
||
|
|
},
|
||
|
|
"file_extension": ".py",
|
||
|
|
"mimetype": "text/x-python",
|
||
|
|
"name": "python",
|
||
|
|
"nbconvert_exporter": "python",
|
||
|
|
"pygments_lexer": "ipython3",
|
||
|
|
"version": "3.8.9"
|
||
|
|
},
|
||
|
|
"orig_nbformat": 4,
|
||
|
|
"vscode": {
|
||
|
|
"interpreter": {
|
||
|
|
"hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
},
|
||
|
|
"nbformat": 4,
|
||
|
|
"nbformat_minor": 2
|
||
|
|
}
|