forked from pms/uczenie-maszynowe
820 lines
119 KiB
Plaintext
820 lines
119 KiB
Plaintext
|
{
|
|||
|
|
"cells": [
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "slide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Uczenie maszynowe\n",
|
|||
|
|
"# 8. Uczenie nienadzorowane"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"Wyobraźmy sobie, że mamy następujący problem:\n",
|
|||
|
|
"\n",
|
|||
|
|
"Mamy zbiór okazów roślin i dysponujemy pewnymi danymi na ich temat (długość płatków kwiatów, ich szerokość itp.), ale zupełnie **nie wiemy**, do jakich gatunków one należą (nie wiemy nawet, ile jest tych gatunków).\n",
|
|||
|
|
"\n",
|
|||
|
|
"Chcemy automatycznie podzielić zbiór posiadanych okazów na nie więcej niż $k$ grup (klastrów) ($k$ ustalamy z góry), czyli dokonać **grupowania (klastrowania)** zbioru przykładów.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Jest to zagadnienie z kategorii uczenia nienadzorowanego.\n",
|
|||
|
|
"\n",
|
|||
|
|
"W celu jego rozwiązania użyjemy algorytmu $k$ średnich."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "slide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"## 8.1. Algorytm $k$ średnich"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 14,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Przydatne importy\n",
|
|||
|
|
"\n",
|
|||
|
|
"import ipywidgets as widgets\n",
|
|||
|
|
"import matplotlib.pyplot as plt\n",
|
|||
|
|
"import numpy as np\n",
|
|||
|
|
"import pandas\n",
|
|||
|
|
"import random\n",
|
|||
|
|
"import seaborn\n",
|
|||
|
|
"\n",
|
|||
|
|
"%matplotlib inline"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 15,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Wczytanie danych (gatunki kosaćców)\n",
|
|||
|
|
"\n",
|
|||
|
|
"data_iris_raw = pandas.read_csv('iris.csv')\n",
|
|||
|
|
"data_iris = pandas.DataFrame()\n",
|
|||
|
|
"data_iris['x1'] = data_iris_raw['sl']\n",
|
|||
|
|
"data_iris['x2'] = data_iris_raw['sw']\n",
|
|||
|
|
"data_iris['x3'] = data_iris_raw['pl']\n",
|
|||
|
|
"data_iris['x4'] = data_iris_raw['sw']\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Nie używamy w ogóle kolumny ostatniej kolumny (\"Gatunek\"), \n",
|
|||
|
|
"# ponieważ chcemy dokonać uczenia nienadzorowanego.\n",
|
|||
|
|
"# Przyjmujemy, że w ogóle nie dysponujemy danymi na temat gatunku,\n",
|
|||
|
|
"# mamy tylko 150 nieznanych roślin.\n",
|
|||
|
|
"\n",
|
|||
|
|
"X = data_iris.values\n",
|
|||
|
|
"Xs = data_iris.values[:, 2:4]"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 16,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Wykres danych\n",
|
|||
|
|
"def plot_unlabeled_data(X, col1=0, col2=1, x1label=r'$x_1$', x2label=r'$x_2$'): \n",
|
|||
|
|
" fig = plt.figure(figsize=(16*.7, 9*.7))\n",
|
|||
|
|
" ax = fig.add_subplot(111)\n",
|
|||
|
|
" fig.subplots_adjust(left=0.1, right=0.9, bottom=0.1, top=0.9)\n",
|
|||
|
|
" X1 = X[:, col1].tolist()\n",
|
|||
|
|
" X2 = X[:, col2].tolist()\n",
|
|||
|
|
" ax.scatter(X1, X2, c='k', marker='o', s=50, label='Dane')\n",
|
|||
|
|
" ax.set_xlabel(x1label)\n",
|
|||
|
|
" ax.set_ylabel(x2label)\n",
|
|||
|
|
" ax.margins(.05, .05)\n",
|
|||
|
|
" return fig"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 17,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Przygotowanie interaktywnego wykresu\n",
|
|||
|
|
"\n",
|
|||
|
|
"dropdown_arg1 = widgets.Dropdown(options=[0, 1, 2, 3], value=2, description='arg1')\n",
|
|||
|
|
"dropdown_arg2 = widgets.Dropdown(options=[0, 1, 2, 3], value=3, description='arg2')\n",
|
|||
|
|
"\n",
|
|||
|
|
"def interactive_unlabeled_data(arg1, arg2):\n",
|
|||
|
|
" fig = plot_unlabeled_data(\n",
|
|||
|
|
" X, col1=arg1, col2=arg2, x1label='$x_{}$'.format(arg1), x2label='$x_{}$'.format(arg2))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 18,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [
|
|||
|
|
{
|
|||
|
|
"data": {
|
|||
|
|
"application/vnd.jupyter.widget-view+json": {
|
|||
|
|
"model_id": "1f6d6588f31343a8954f9a91a2971507",
|
|||
|
|
"version_major": 2,
|
|||
|
|
"version_minor": 0
|
|||
|
|
},
|
|||
|
|
"text/plain": [
|
|||
|
|
"interactive(children=(Dropdown(description='arg1', index=2, options=(0, 1, 2, 3), value=2), Dropdown(descripti…"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
"metadata": {},
|
|||
|
|
"output_type": "display_data"
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"data": {
|
|||
|
|
"text/plain": [
|
|||
|
|
"<function __main__.interactive_unlabeled_data(arg1, arg2)>"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
"execution_count": 18,
|
|||
|
|
"metadata": {},
|
|||
|
|
"output_type": "execute_result"
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"source": [
|
|||
|
|
"widgets.interact(interactive_unlabeled_data, arg1=dropdown_arg1, arg2=dropdown_arg2)"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"Powyższy wykres przedstawia surowe dane.\n",
|
|||
|
|
"Ponieważ nasze obserwacje mają 4 współrzędne, na płaskim wykresie możemy przedstawić tylko dwie z nich.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Dlatego powyższy wykres umożliwia wybór dwóch współrzędnych, na które chcemy rzutować.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Wszystkie takie „rzuty” przedstawia również wykres poniżej."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 19,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "skip"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [
|
|||
|
|
{
|
|||
|
|
"data": {
|
|||
|
|
"text/plain": [
|
|||
|
|
"<seaborn.axisgrid.PairGrid at 0x23b2908c8b0>"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
"execution_count": 19,
|
|||
|
|
"metadata": {},
|
|||
|
|
"output_type": "execute_result"
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"data": {
|
|||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAu0AAAG8CAYAAABqhJuoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOyde3gV1bn/v2tmX7JzkYSQIBJAoQiNNAhBDVAvlVOqR9SHgmgFUVQS5BT762kVz+nhpz3U5yeixx5rubaKIlYQ9Nji4+14RE8RFAKVaiSlCBhQSYiJ5rKzb7N+f+zMZF9mzd6T7Mvsvd/P8+xHmcuaNWtmvbPyrnd9X8Y5B0EQBEEQBEEQ1kVKdwUIgiAIgiAIgjCGBu0EQRAEQRAEYXFo0E4QBEEQBEEQFocG7QRBEARBEARhcWjQThAEQRAEQRAWhwbtBEEQBEEQBGFxkjpoZ4z9lDH2MWPsI8bYHxhjecm8HkEQBEEQBEFkI0kbtDPGhgO4G8AUzvkEADKAm5J1PYIgCIIgCILIVpIdHmMD4GKM2QDkA/jc6OCrrrqKA6Af/dL1Swr0XtMvzb+kQe82/dL8Swr0XtMvzT8hNqOdA4Fzfoox9giAzwC4AbzBOX/D6JwzZ84kqzoEkTbovc4eFIWjtcsLrz8Ah01GaYEDksTSXa20Qe82EQ+Z1m/ovc4eMu3di0XSBu2MsRIA1wM4D0A7gBcYYws4589GHFcLoBYARo4cmazqEERKofc6+1AUjsbTHVj8zH6cbHOjosSFjQunYNzQooz+CJiF3m3CDJnSb+i9zj4y5d0zQzLDY/4BwDHOeQvn3AfgRQDTIg/inG/gnE/hnE8pKytLYnUIInXQe519tHZ5NeMPACfb3Fj8zH60dnnTXLPUQu82YYZM6Tf0XmcfmfLumSFpnnYEw2JqGGP5CIbHzACwP4nXsxTn3vdKv847/tA1Ca4JQRCJwOsPaMZf5WSbG15/IE01IgjrQ/2GSBfZ+O4lzdPOOX8fwHYABwD8tfdaG5J1PYIgiGTisMmoKHGFbasoccFhk9NUI4KwPtRviHSRje9eMj3t4JzfD+D+ZF6DIIjEYHbBTrKPtxqlBQ48c/vFONHajXyHjG5vAKNK81Fa4Eh31Qgi7Yj6d2mBAxsXTomKKw7tN36/guZOD3wBBXZZQnmhE5LEMtpepAKy2cZko81O6qCdSC0UkkP0F7MLdpJ9vFXx+BWsePmjsHsgiFwnVv8eN7QILy2drjv48/sVHD7dgSXP1mvnblp0EXx+jsWbM9teJBOy2fGRbTY72TrtBEFkAGYX7PTn+MfebMSKWZXYWluDFbMq8dibjRm1ICgbFzURRCKI1TckiaGsyInhJfkoKwp60RWFo6XDg1Nfu9HS4UFZoVM7t+krtzZg1yuPIJsdD9los8nTThCE6QU7Zo9XFAW3TjsPy3cc0jweq+ZUQVGUxNxACsjGRU0EkQjM24NoL+6qOVV45PVGHGxqR75Dpr4WA7LZsclGm02edoIgTC/YMXt8gEMz/kDQcC7fcQgBw9xv1iIbFzURRCIw2zf0PKDLdxzCkivGAAC6vQHqazEgmx2bbLTZNGgnCEJbLKYaOL3FYgM5nnOu6/HgPHO+AGbvmSByBbN9Q+QBLXbZUVHiwojBLmy8hfqaEWSzY5ONNpvCYwiCiLlYbKDHqx6P0I9Apnk8zN4zQeQKibIHw0tc2FY3VVOPob4mhmx2bLLRZpOnnSAIAPqLxRJFtng8ktlGBJEriOzBOYNcOKfYBZtNor4WB2SzY5Nt7xF52gmCMI1ZObBs9HgQBBGE7IH1oWeUHZCnnSAI0xhJaWlSbm3daOnwQFGCMZDZ5vEgCCKIyB60u726tgAge5BqyGZnB+RpJwjCNKKFZIqiZEVCDoIg4kfPHpQVOvFFew/qQpImkS1IH2Szs4OsH7T3N0soQJlCCUKEaJFSgEPXm/PS0ukoLXBYKoV2pqfoJohUotdfgKAHN8A5nq+tAeccEmNod/swyGXXBuxAuC0oK3Km81Zykmyw2am6hpXJ+kE7QRCJR12kFOmdEcmEef0BS6XQzpYU3QSRCvT6yzO3XwyPXwnbtnpuFR569TBaOj3YfMfFWZfYJpPJdJudqmtYHYppJwjCNKGLlHYv/x5eWjod44YWCZNZMMaSmnLbLNmY3pogkoVefznR2h217Z7twQRJJ9vcOH6mO+sS22QymW6zU3UNq0OedoIg+oW6SCkUkTdHZsEY1xWzKlHssqPd7cO6XUcTlnLbLNmY3pogkkVkf5k0ohgjBrt0+1Cxyw4AePytI1i/oDoqpj3TJAOziUy22am6htVJ2qCdMTYOwNaQTaMB/F/O+a+TdU2CINKLSCas3e3FvVeNwz3bD4VNpbscxim3k5XYIxsShxBEqgjtL5NGFOPnPxiHpq/cun2o3e0DALR0ejCsOI8kAy1OptjsVF3D6iQtPIZz3sg5v5BzfiGAagDdAF5K1vUIgrAGejJhfoVrxh/om0r3K/opsZOd2CNbEocQuY1Iqi/RhPaXJVeMwfIdh/D4W0ewak5VWB9aPbcK63Yd1fpTsctBkoEZQCbY7FRdw+qkKjxmBoCjnPMTKboeQRAWwudXdKc1fX5FeI7TJmHl9ROQ75DR7Q3AaUucj4EShxCZTioX5YX2l26vHyfb3DjZ5sYjrzdq4RMVJS64HDKeuHkS9acswGo2GyC7DaRuIepNAP6QomsRBGExRIudHDYZPl8Ap9q6caK1C6fauuHzBdDa5cVDr34CbyD4gfAGFDz06icJXXBEiUOITEa0KO9MlyfK+x7LI2+0X933xdfB67gcfX35YFM76jbX42cvfAiHTcbgAupP2YIVbTZAdjvpnnbGmAPAdQD+RbC/FkAtAIwcOTLZ1SGIlEDvdTiixU6DnDION3firpDFamsXVGN4sRO3TjsPy3f0xVOumlMFRRF7eYjUQO+2NRAtyuv2BLDg9+8bSjOGeuSNPPYAdKUe9fpypoco0HsdDtlsa5KK8JirARzgnJ/W28k53wBgAwBMmTIlOQF5BJFicuG9NpPkQpIYxpYVYlvdVPgCCuyyhPJCJ0539GjGHwgOOu56th5ba2vw9HvHwpQLnn7vGB64bkIqb5HQIRfe7UxAtCjv2JmuKGnGFS9/JExyJPLYb6ubijy7FLVv4ZMfYOey6VF9OdM9nrnwXpPNznxSMWj/ESg0hiCyCrPxtIrCcaSlM+r4fIes6y0MKFzXayNn9riAIBKGnid07fzJ+L8vfxx2nKiPqTJ5Io/95+1uuOwyygqdYfvLCp041dYTJeWYSwluMhGy2dlBUmPaGWP5AL4P4MVkXocgiNRiNsmF6HibxHTjJmWJacZfPX75jkMIZKX/iyDME7oo7917v4eV109Ap8ePlk5P2HHd3oBhkiNR7HJrlxd1z9bj7hljw/bdPWOsNmAHcjPBTSZCNjs7SKqnnXPeDaA0mdcgCMIcZqZIRZhNciE63iYxrF1QHRUfKUtM93jOOfx+Bc2dnrApW1uCVQoIwirE018551i0aR8mjSjGqjlV2uBpZmU5vlVeiGfvuATHznTh8beOoKzIgX+7phJefwAtHR6UuOxRHvtVc6rwyOuNONnmxtihhXjqtotQnG9HUZ4d+Q4p5xPcpBqy2YQKZUQliBwiUTJxZpNc2GVJ93iAobTQjj8sroHCOSTGIMuAxJju8XkOCYdPd2BJyAdj3YJqjB9aRB8BIuuId5HoilmVqChx4WBTuybDWFHiAjjwo417tXPX31INp03Czb97P6w8NXb583Y3Wru8eOT1RhxsasfMynK0d/u0mPiKEhd+e/NkzKwsxxsNzVo9cy3BTSohm02EQi1GEDmE2SlSEUZJLkTycavnRidiATjmrduLSx9+G5ev3oVLH34b89btBedct3yvj2vGX63/kmfr0RwREkAQ2UBrlxePvRkchG+trcGKWZV47M1GtLu9+PKbHnR5/FgxqxJvNZzWEh0dbGrHyp0NKHLaosJY6jbXo+krd1T/b3P7cPZZeShw2rByZwM
|
|||
|
|
"text/plain": [
|
|||
|
|
"<Figure size 756x432 with 20 Axes>"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
"metadata": {
|
|||
|
|
"needs_background": "light"
|
|||
|
|
},
|
|||
|
|
"output_type": "display_data"
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"source": [
|
|||
|
|
"seaborn.pairplot(data_iris, vars=data_iris.columns, size=1.5, aspect=1.75)"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 20,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Odległość euklidesowa\n",
|
|||
|
|
"def euclidean_distance(x1, x2):\n",
|
|||
|
|
" return np.linalg.norm(x1 - x2)"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 21,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Algorytm k średnich\n",
|
|||
|
|
"def k_means(X, k, distance=euclidean_distance):\n",
|
|||
|
|
" history = []\n",
|
|||
|
|
" Y = []\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Wylosuj centroid dla każdej klasy\n",
|
|||
|
|
" centroids = [[random.uniform(X.min(axis=0)[f], X.max(axis=0)[f])\n",
|
|||
|
|
" for f in range(X.shape[1])]\n",
|
|||
|
|
" for c in range(k)]\n",
|
|||
|
|
" history.append((centroids, Y))\n",
|
|||
|
|
"\n",
|
|||
|
|
" # Powtarzaj, dopóki klasy się zmieniają\n",
|
|||
|
|
" while True:\n",
|
|||
|
|
" distances = [[distance(centroids[c], x) for c in range(k)] for x in X]\n",
|
|||
|
|
" Y_new = [d.index(min(d)) for d in distances]\n",
|
|||
|
|
" if Y_new == Y:\n",
|
|||
|
|
" break\n",
|
|||
|
|
" Y = Y_new\n",
|
|||
|
|
" history.append((centroids, Y))\n",
|
|||
|
|
" XY = np.asarray(np.concatenate((X, np.matrix(Y).T), axis=1))\n",
|
|||
|
|
" Xc = [XY[XY[:, 2] == c][:, :-1] for c in range(k)]\n",
|
|||
|
|
" centroids = [[Xc[c].mean(axis=0)[f] for f in range(X.shape[1])]\n",
|
|||
|
|
" for c in range(k)]\n",
|
|||
|
|
" history.append((centroids, Y))\n",
|
|||
|
|
"\n",
|
|||
|
|
" result = history[-1][1]\n",
|
|||
|
|
" return result, history"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 22,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Wykres danych - klastrowanie\n",
|
|||
|
|
"def plot_clusters(X, Y, k, centroids=None):\n",
|
|||
|
|
" color = ['r', 'g', 'b', 'c', 'm', 'y', 'k']\n",
|
|||
|
|
" fig = plt.figure(figsize=(16*.7, 9*.7))\n",
|
|||
|
|
" ax = fig.add_subplot(111)\n",
|
|||
|
|
" fig.subplots_adjust(left=0.1, right=0.9, bottom=0.1, top=0.9)\n",
|
|||
|
|
"\n",
|
|||
|
|
" if not Y:\n",
|
|||
|
|
" ax.scatter(X[:, 0], X[:, 1], c='gray', marker='o', s=25, label='Dane')\n",
|
|||
|
|
"\n",
|
|||
|
|
" X1 = [[x for x, y in zip(X[:, 0].tolist(), Y) if y == c] for c in range(k)]\n",
|
|||
|
|
" X2 = [[x for x, y in zip(X[:, 1].tolist(), Y) if y == c] for c in range(k)]\n",
|
|||
|
|
"\n",
|
|||
|
|
" for c in range(k):\n",
|
|||
|
|
" ax.scatter(X1[c], X2[c], c=color[c], marker='o', s=25, label='Dane')\n",
|
|||
|
|
" if centroids:\n",
|
|||
|
|
" ax.scatter([centroids[c][0]], [centroids[c][1]], c=color[c], marker='+', s=500, label='Centroid')\n",
|
|||
|
|
" \n",
|
|||
|
|
" ax.set_xlabel(r'$x_1$')\n",
|
|||
|
|
" ax.set_ylabel(r'$x_2$')\n",
|
|||
|
|
" ax.margins(.05, .05)\n",
|
|||
|
|
" return fig"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 23,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [
|
|||
|
|
{
|
|||
|
|
"data": {
|
|||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAroAAAGbCAYAAAAr5TMXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df4wk533f+c9XM2yQqhZPm2gvMxguRQhr5w9LdrfQ3LJ3jg4h2Zm0TFD5I4AUYGNkEUBn30WQIQaG7RvIZ3uM5ICTYdgErCjWMXZWZ8EXOY6gsDMRQFNZzUBFzqpbpmQqwFwikxzPRGt7LbLLS7e395s/akaaXvbO9M5Wd1U//X4Bg55++pmqb3X1j8/WPvWUubsAAACA0Lyp6AIAAACAcSDoAgAAIEgEXQAAAASJoAsAAIAgEXQBAAAQJIIuAAAAgjSRoGtmc2bWNrPPD3nsUTP7tpl19n8+NomaAAAAELb5Ca3nI5JelHT/bR6/7O6P3ckC3/a2t/lDDz10t3UBAABgil25cuVP3f30sMfGHnTN7AFJPybplyV9NK/lPvTQQ9ra2sprcQAAAJhCZvbHt3tsEkMXflXST0u6eUSfHzKzr5pZy8y+bwI1AQAAIHBjDbpm9pikb7n7lSO6fUXS2939ByT9uqTfP2J5HzKzLTPbunr1as7VAgAAICTjPqK7LOlxM/umpM9Ieo+ZXTrcwd1fdffu/u9PS7rHzN42bGHu/kl3b7h74/TpoUMxAAAAAEljDrru/rPu/oC7PyTpg5KecfcLh/uY2YKZ2f7v5/Zr+rNx1gUAAIDwTWrWhQFm9hOS5O6fkPQPJP2kmd2QdF3SB93di6gLAAAA4bBpzZSNRsOZdQEAAGC2mdkVd28Me4wrowEAACBIBF0AAAAEiaALAACAIBF0AQAAECSCLgAAAIJE0AUAAECQCplHFwXp96VWS2q3pXpdajalubmiqwIAABgLgu6s6PellRUpSaQ0laJIimNpfZ2wCwAAgsTQhVnRamUht9uV3LPbJMnaAQAAAkTQnRXtdnYk97A0lTqdYuoBAAAYM4LurKjXs+EKh0WRVKsVUw8AAMCYEXRnRbOZjcmtViWz7DaOs3YAAIAAcTLarJiby048a7Wy4Qq1GrMuAACAoBF0Z8ncnPTYY9kPAABA4Bi6AAAAgCARdAEAABAkgi4AAACCRNAFAABAkAi6AAAACBJBFwAAAEEi6AIAACBIBF0AAAAEiaALAACAIBF0AQAAECSCLgAAAIJE0AUAAECQCLoAAAAIEkEXAAAAQSLoAgAAIEgEXQAAAASJoAsAAIAgEXQBAAAQJIIuAAAAgjRfdAHIQb8vtVpSuy3V61KzKc3NFV0VAABAoQi6067fl1ZWpCSR0lSKIimOpfV1wi4AAJhpDF2Ydq1WFnK7Xck9u02SrB0AAGCGEXSnXbudHck9LE2lTqeYegAAAEqCoDvt6vVsuMJhUSTVasXUAwAAUBIE3WnXbGZjcqtVySy7jeOsHQAAYIZxMtq0m5vLTjxrtbLhCrUasy4AAACIoBuGuTnpsceyHwAAAEhi6AIAAAACRdAFAABAkAi6AAAACBJBFwAAAEEi6AIAACBIBF0AAAAEienFZkm/n823225nV1Rjvl0AABAwgu6s6PellRUpSaQ0zS4THMfZxSYIuwAAIEAMXZgVrVYWcrtdyT27TZKsHQAAIEAE3VnRbmdHcg9L0+yywQAAAAEi6M6Kej0brnBYFEm1WjH1AAAAjBlBd1Y0m9mY3GpVMstu4zhrBwAACBAno82KubnsxLNWKxuuUKsx6wIAAAgaQXeWzM1Jjz2W/QAAAASOoQsAAAAIEkEXAAAAQSLoAgAAIEgEXQAAAASJoAsAAIAgTSTomtmcmbXN7PNDHjMz+zUz2zazPzSzd0+iJgAAAIRtUtOLfUTSi5LuH/JYU9L37P/Ekn5j/zZc/X42n227nV2xjPlsAQAAcjf2oGtmD0j6MUm/LOmjQ7q8X9Jvu7tL+rKZvdXMFt19d9y1FaLfl1ZWpCSR0jS7DG8cZxdzIOwCAADkZhJDF35V0k9Lunmbx5ckvXzo/iv7bWFqtbKQ2+1K7tltkmTtAAAAyM1Yg66ZPSbpW+5+5ahuQ9r8Nsv7kJltmdnW1atXc6lx4trt7EjuYWmaXZYXAAAAuRn3Ed1lSY+b2TclfUbSe8zs0i19XpF05tD9ByT9ybCFufsn3b3h7o3Tp0+Po97xq9ez4QqHRZFUqxVTDwAAQKDGGnTd/Wfd/QF3f0jSByU94+4Xbun2OUk/vj/7wg9K+naw43Ol7MSzOJaqVcksu43jrB0AAAC5mdSsCwPM7Cckyd0/IelpSe+TtC3pLyVdLKKmiZmby048a7Wy4Qq1GrMuAAAAjIFlkx1Mn0aj4VtbW0WXAQAAgAKZ2RV3bwx7jCujAQAAIEgEXQAAAASJoAsAAIAgEXQBAAAQJIIuAAAAglTI9GIYUb+fTUPWbmcXmijbNGRlrw8AAMw0gm5Z9fvSyoqUJNklgqMou7DE+no5wmTZ6wMAADOPoQtl1WplIbLbldyz2yTJ2sug7PUBAICZR9Atq3Y7O1J6WJpmV1Mrg7LXBwAAZh5Bt6zq9Ww4wGFRlF0yuAzKXh8AAJh5BN2yajazMa/VqmSW3cZx1l4GZa8PAADMPE5GK6u5uezErlYrGw5Qq5VrVoOy1wcAAGaeuXvRNZxIo9Hwra2tossAAABAgczsirs3hj3G0AUAAAAEiaALAACAIBF0AQAAECSCLgAAAIJE0AUAAECQCLoAAAAIEvPozpJ+P5v3tt3Ormx2u3lv8+6Xd30AAASif7Ov1nZL7d226ot1Nc82Nfem8X/3FbXeSSPozop+X1pZkZJEStPscr1xnF304XCYzLtf3vUBABCI/s2+Vi6tKNlJlPZSRZVI8VKs9QvrYw2dRa23CAxdmBWtVhYiu13JPbtNkqx9nP3yrg8AgEC0tltKdhJ1e125XN1eV8lOotb2eL/7ilpvEQi6s6Ldzo6UHpam2eV7x9kv7/oAAAhEe7ettDf43Zf2UnX2xvvdV9R6i0DQnRX1ejYc4LAokmq18fbLuz4AAAJRX6wrqgx+90WVSLWF8X73FbXeIhB0Z0WzmY15rVYls+w2jrP2cfbLuz4AAALRPNtUvBSrWqnKZKpWqoqXYjXPjve7r6j1FsHcvegaTqTRaPjW1lbRZUyXg1kNOp3sSOlxsynk1S/v+gAACMTB7AedvY5qC7WJz7ow6fWOg5ldcffG0McIugAAAJhWRwVdhi4AAAAgSARdAAAABImgCwAAgCARdAEAABAkgi4AAACCRNAFAABAkAi6s+T6demDH5Te8Y7s9vr1oisCAAAYm/miC8CEXL8u3X+/dONGdv+//Tfps5+VXn1Vuu++YmsDAAAYA47ozoqLF78bcg/cuJG1AwAABIigOyuee254+/PPT7YOAACACSHozopz54a3P/zwZOsAAACYEILurHjqKWn+liHZ8/NZOwAAQIAIurPivvuyE88+8IFs1oUPfIAT0QAAQNCYdWGW3Hef9JnPFF0FAADARHBEFwAAAEEi6AIAACBIBF0AAAAEiaALAACAIBF0AQAAECSCLgAAAILE9GKj6PelVktqt6V6XWo2pbm5oqv6rrLXBwBAyfVv9tXabqm921Z9sa7m2abm3sR36bQj6B6n35dWVqQkkdJUiiIpjqX19XKEybLXBwBAyfVv9rVyaUXJTqK0lyqqRIqXYq1fWCfsTjmGLhyn1cpCZLcruWe3SZK1l0HZ6wMAoORa2y0lO4m6va5crm6vq2QnUWub79JpR9A9TrudHSk9LE2lTqeYem5V9voAACi59m5baW/wuzTtpers8V067Qi6x6nXs+EAh0WRVKsVU8+tyl4fAAAlV1+sK6oMfpdGlUi1Bb5Lpx1B9zjNZjbmtVqVzLLbOM7ay6Ds9QEAUHLNs03FS7GqlapMpmqlqngpVvMs36XTzty96BpOpNFo+NbW1mRWdjCrQaeTHSkt26wGZa8PAICSO5h1obPXUW2
|
|||
|
|
"text/plain": [
|
|||
|
|
"<Figure size 806.4x453.6 with 1 Axes>"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
"metadata": {
|
|||
|
|
"needs_background": "light"
|
|||
|
|
},
|
|||
|
|
"output_type": "display_data"
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"source": [
|
|||
|
|
"Ys, history = k_means(Xs, 2)\n",
|
|||
|
|
"fig = plot_clusters(Xs, Ys, 2, centroids=history[-1][0]),"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 24,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Przygotowanie interaktywnego wykresu\n",
|
|||
|
|
"\n",
|
|||
|
|
"MAXSTEPS = 15\n",
|
|||
|
|
"\n",
|
|||
|
|
"slider_k = widgets.IntSlider(min=1, max=7, step=1, value=2, description=r'$k$', width=300)\n",
|
|||
|
|
"\n",
|
|||
|
|
"def interactive_kmeans_k(steps, history, k):\n",
|
|||
|
|
" if steps >= len(history) or steps == MAXSTEPS:\n",
|
|||
|
|
" steps = len(history) - 1\n",
|
|||
|
|
" fig = plot_clusters(Xs, history[steps][1], k, centroids=history[steps][0])\n",
|
|||
|
|
" \n",
|
|||
|
|
"def interactive_kmeans(k):\n",
|
|||
|
|
" slider_steps = widgets.IntSlider(min=0, max=MAXSTEPS, step=1, value=0, description=r'steps', width=300)\n",
|
|||
|
|
" _, history = k_means(Xs, k)\n",
|
|||
|
|
" widgets.interact(interactive_kmeans_k, steps=slider_steps,\n",
|
|||
|
|
" history=widgets.fixed(history), k=widgets.fixed(k))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 25,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [
|
|||
|
|
{
|
|||
|
|
"data": {
|
|||
|
|
"application/vnd.jupyter.widget-view+json": {
|
|||
|
|
"model_id": "d60eb5f7bce84fabaa43033382b1c9b8",
|
|||
|
|
"version_major": 2,
|
|||
|
|
"version_minor": 0
|
|||
|
|
},
|
|||
|
|
"text/plain": [
|
|||
|
|
"interactive(children=(IntSlider(value=2, description='$k$', max=7, min=1), Button(description='Run Interact', …"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
"metadata": {},
|
|||
|
|
"output_type": "display_data"
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"data": {
|
|||
|
|
"text/plain": [
|
|||
|
|
"<function __main__.interactive_kmeans(k)>"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
"execution_count": 25,
|
|||
|
|
"metadata": {},
|
|||
|
|
"output_type": "execute_result"
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"source": [
|
|||
|
|
"widgets.interact_manual(interactive_kmeans, k=slider_k) "
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Algorytm $k$ średnich – dane wejściowe\n",
|
|||
|
|
"\n",
|
|||
|
|
"* $k$ – liczba klastrów\n",
|
|||
|
|
"* zbiór uczący $X = \\{ x^{(1)}, x^{(2)}, \\ldots, x^{(m)} \\}$, $x^{(i)} \\in \\mathbb{R}^n$"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "fragment"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"Na wejściu nie ma zbioru $Y$, ponieważ jest to uczenie nienadzorowane!"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Algorytm $k$ średnich – pseudokod\n",
|
|||
|
|
"\n",
|
|||
|
|
"1. Zainicjalizuj losowo $k$ centroidów (środków ciężkości klastrów): $\\mu_1, \\ldots, \\mu_k$.\n",
|
|||
|
|
"1. Powtarzaj dopóki przyporządkowania klastrów się zmieniają:\n",
|
|||
|
|
" 1. Dla $i = 1$ do $m$:\n",
|
|||
|
|
" za $y^{(i)}$ przyjmij klasę najbliższego centroidu.\n",
|
|||
|
|
" 1. Dla $c = 1$ do $k$:\n",
|
|||
|
|
" za $\\mu_c$ przyjmij średnią wszystkich punktów $x^{(i)}$ takich, że $y^{(i)} = c$."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 13,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Algorytm k średnich\n",
|
|||
|
|
"def kmeans(X, k, distance=euclidean_distance):\n",
|
|||
|
|
" Y = []\n",
|
|||
|
|
" centroids = [[random.uniform(X.min(axis=0)[f],X.max(axis=0)[f])\n",
|
|||
|
|
" for f in range(X.shape[1])]\n",
|
|||
|
|
" for c in range(k)] # Wylosuj centroidy\n",
|
|||
|
|
" while True:\n",
|
|||
|
|
" distances = [[distance(centroids[c], x) for c in range(k)]\n",
|
|||
|
|
" for x in X] # Oblicz odległości\n",
|
|||
|
|
" Y_new = [d.index(min(d)) for d in distances]\n",
|
|||
|
|
" if Y_new == Y:\n",
|
|||
|
|
" break # Jeśli nic się nie zmienia, przerwij\n",
|
|||
|
|
" Y = Y_new\n",
|
|||
|
|
" XY = np.asarray(np.concatenate((X,np.matrix(Y).T),axis=1))\n",
|
|||
|
|
" Xc = [XY[XY[:, 2] == c][:, :-1] for c in range(k)]\n",
|
|||
|
|
" centroids = [[Xc[c].mean(axis=0)[f]\n",
|
|||
|
|
" for f in range(X.shape[1])]\n",
|
|||
|
|
" for c in range(k)] # Przesuń centroidy\n",
|
|||
|
|
" return Y"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"* Liczba klastrów jest określona z góry i wynosi $k$."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "fragment"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"* Jeżeli w którymś kroku algorytmu jedna z klas nie zostanie przyporządkowana żadnemu z przykładów, pomija się ją – w ten sposób wynikiem działania algorytmu może być mniej niż $k$ klastrów."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Funkcja kosztu dla problemu klastrowania\n",
|
|||
|
|
"\n",
|
|||
|
|
"$$ J \\left( y^{(i)}, \\ldots, y^{(m)}, \\mu_{1}, \\ldots, \\mu_{k} \\right) = \\frac{1}{m} \\sum_{i=1}^{m} || x^{(i)} - \\mu_{y^{(i)}} || ^2 $$"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "fragment"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"* Zauważmy, że z każdym krokiem algorytmu $k$ średnich koszt się zmniejsza (lub ewentualnie pozostaje taki sam)."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Wielokrotna inicjalizacja\n",
|
|||
|
|
"\n",
|
|||
|
|
"* Algorytm $k$ średnich zawsze znajdzie lokalne minimum funkcji kosztu $J$, ale nie zawsze będzie to globalne minimum."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "fragment"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"* Aby temu zaradzić, można uruchomić algorytm $k$ średnich wiele razy, za każdym razem z innym losowym położeniem centroidów (tzw. **wielokrotna losowa inicjalizacja** – *multiple random initialization*). Za każdym razem obliczamy koszt $J$. Wybieramy ten wynik, który ma najniższy koszt."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Wybór liczby klastrów $k$\n",
|
|||
|
|
"\n",
|
|||
|
|
"Ile powinna wynosić liczba grup $k$?\n",
|
|||
|
|
"* Najlepiej wybrać $k$ ręcznie w zależności od kształtu danych i celu, który chcemy osiągnąć."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "slide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"## 8.2. Analiza głównych składowych"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"Analiza głównych składowych to inny przykład zagadnienia z dziedziny uczenia nienadzorowanego.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Polega na próbie zredukowania liczby wymiarów dla danych wielowymiarowych, czyli zmniejszenia liczby cech, gdy rozpatrujemy przykłady o dużej liczbie cech."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Redukcja liczby wymiarów\n",
|
|||
|
|
"\n",
|
|||
|
|
"Z jakich powodów chcemy redukować liczbę wymiarów?"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "fragment"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"* Chcemy pozbyć się nadmiarowych cech, np. „długość w cm” / „długość w calach”, „długość” i „szerokość” / „powierzchnia”."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "fragment"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"* Chcemy znaleźć bardziej optymalną kombinację cech."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "fragment"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"* Chcemy przyspieszyć działanie algorytmów."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "fragment"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"* Chcemy zwizualizować dane."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Błąd rzutowania\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Błąd rzutowania** – błąd średniokwadratowy pomiędzy danymi oryginalnymi a danymi zrzutowanymi."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Sformułowanie problemu\n",
|
|||
|
|
"\n",
|
|||
|
|
"**Analiza głównych składowych** (*Principal Component Analysis*, PCA):\n",
|
|||
|
|
"\n",
|
|||
|
|
"Zredukować liczbę wymiarów z $n$ do $k$, czyli znaleźć $k$ wektorów $u^{(1)}, u^{(2)}, \\ldots, u^{(k)}$ takich, że rzutowanie danych na podprzeztrzeń rozpiętą na tych wektorach minimalizuje błąd rzutowania."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"* **Uwaga:** analiza głównych składowych to (mimo pozornych podobieństw) zupełnie inne zagadnienie niż regresja liniowa!"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"### Algorytm PCA\n",
|
|||
|
|
"\n",
|
|||
|
|
"1. Dany jest zbiór składający się z $x^{(1)}, x^{(2)}, \\ldots, x^{(m)} \\in \\mathbb{R}^n$.\n",
|
|||
|
|
"1. Chcemy zredukować liczbę wymiarów z $n$ do $k$ ($k < n$).\n",
|
|||
|
|
"1. W ramach wstępnego przetwarzania dokonujemy skalowania i normalizacji średniej.\n",
|
|||
|
|
"1. Znajdujemy macierz kowariancji:\n",
|
|||
|
|
" $$ \\Sigma = \\frac{1}{m} \\sum_{i=1}^{n} \\left( x^{(i)} \\right) \\left( x^{(i)} \\right)^T $$\n",
|
|||
|
|
"1. Znajdujemy wektory własne macierzy $\\Sigma$ (rozkład SVD):\n",
|
|||
|
|
" $$ (U, S, V) := \\mathop{\\rm SVD}(\\Sigma) $$\n",
|
|||
|
|
"1. Pierwszych $k$ kolumn macierzy $U$ to szukane wektory."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 14,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"from sklearn.preprocessing import StandardScaler\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Algorytm PCA - implementacja\n",
|
|||
|
|
"def pca(X, k):\n",
|
|||
|
|
" X_std = StandardScaler().fit_transform(X) # normalizacja\n",
|
|||
|
|
" mean_vec = np.mean(X_std, axis=0)\n",
|
|||
|
|
" cov_mat = np.cov(X_std.T) # macierz kowariancji\n",
|
|||
|
|
" n = cov_mat.shape[0]\n",
|
|||
|
|
" eig_vals, eig_vecs = np.linalg.eig(cov_mat) # wektory własne\n",
|
|||
|
|
" eig_pairs = [(np.abs(eig_vals[i]), eig_vecs[:, i])\n",
|
|||
|
|
" for i in range(len(eig_vals))]\n",
|
|||
|
|
" eig_pairs.sort()\n",
|
|||
|
|
" eig_pairs.reverse()\n",
|
|||
|
|
" matrix_w = np.hstack([eig_pairs[i][1].reshape(n, 1)\n",
|
|||
|
|
" for i in range(k)]) # wybór\n",
|
|||
|
|
" return X_std.dot(matrix_w) # transformacja"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": 15,
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"outputs": [
|
|||
|
|
{
|
|||
|
|
"data": {
|
|||
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAArkAAAGaCAYAAAALjluxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df3Dk9X3n+ddH85Pp7rutOsgN0+NZcCSLUJxxijnDnpW6xZ7Ypu2YGFj3xo5PvlUttdawi5B9lihzm90jjuYmVSbioikO96YItcG0A97geEQMVEgyqtrYGW69WQgzli4TCtKijLOVo9Vifkmf+2PUskZ8+5f6+/1+Pt9vPx9VKkb9banfarq//f5+Pu/P+2OstQIAAADSpM91AAAAAEDYSHIBAACQOiS5AAAASB2SXAAAAKQOSS4AAABShyQXAAAAqbPddQBbceWVV9prrrnGdRgAAABw6KWXXvqJtfaqoGOJTHKvueYanTx50nUYAAAAcMgY81qjY5QrAAAAIHW8SXKNMduMMf/JGPNd17EAAAAg2bxJciXdK+lV10EAAAAg+bxIco0x+yV9QlLJdSwAAABIPi+SXEm/JekrklZdBwIAAIDkc57kGmM+KenH1tqXWtzvbmPMSWPMybfeeium6AAAAJBEzpNcSR+S9CljzN9IelLSh40x/37znay1j1prD1prD151VWA7NAAAAECSB0mutfZ+a+1+a+01kv6ppD+21v6q47AAAACQYM6TXAAAACBsXu14Zq39E0l/4jgMAAAAJBwjuQAAAEgdr0ZyAcC1arWqcrms+fl5DQwMqFgsKpfLuQ4LANAhklwAWDM3N6dCoaDV1VXVajVlMhmNj49rdnZWQ0NDrsMDAHSAcgUA0KUR3EKhoGq1qlqtJkmq1Wrrty8tLTmOEADQCZJcAJBULpe1uhq86eLq6qrK5XLMEQEAukGSCwCS5ufn10dwN6vValpYWIg5IgBAN0hyAUDSwMCAMplMw+PW2hijAQB0iyQXACQVi0UZYxoen5mZoS4XABKEJBcAJOVyOY2OjjY8bq2lLhcAEoQkFwDaQF0uACQLSS4ArGlWl5vJZNTf3x9zRACArSLJBYA1xWJRfX3Bp8W+vj4Vi8WYIwIAbBVJLgCsyeVymp2dVS6XWx/RzWQy67dns1nHEQIA2sW2vgCwwdDQkCqVisrlshYWFtTf369isUiCCwAJQ5ILAJtks1mNjIy4DgMA0AXKFQAAAJA6JLkAAABIHZJcAAAApA5JLgAAAFKHJBcAAACpQ5ILAACA1CHJBQAAQOqQ5AIAACB1SHIBAACQOiS5AAAASB2SXAAAAKQOSS4AAABShyQXAAAAqUOSCwAAgNQhyQUAAEDqkOQCAAAgdUhyAQAAkDokuQAAAEgdklwAAACkDkkuAAAAUockFwAAAKlDkgsAAIDUIckFAABA6jhPco0xu40xPzDG/GdjzCvGmH/rOiYAAAAk23bXAUg6J+nD1tolY8wOSXPGmGettX/uOjAAAAAkk/Mk11prJS2tfbtj7cu6iwgAAABJ57xcQZKMMduMMT+U9GNJz1trv+86JgAAACSXF0mutXbFWvsBSfslfdAYc8Pm+xhj7jbGnDTGnHzrrbfiDxIAAACJ4UWSW2et/XtJfyLp4wHHHrXWHrTWHrzqqqtijw0AAADJ4TzJNcZcZYz5B2v/vkLSIUmn3EYFAACAJHO+8EzS1ZJ+1xizTZeS7m9Za7/rOCYAAAAkmPMk11r7l5J+3nUcAAAASA/n5QoAAABA2EhyAQAAkDrOyxUAwIVqtapyuaz5+XkNDAyoWCwql8u5DgsAEBKSXAA9Z25uToVCQaurq6rVaspkMhofH9fs7KyGhoZchwcACAHlCgB6SrVaVaFQULVaVa1WkyTVarX125eWllr8BgBAEpDkAugp5XJZq6urgcdWV1dVLpdjjggAEAWSXAA9ZX5+fn0Ed7NaraaFhYWYIwIARIEkF0BPGRgYUCaTCTyWyWTU398fc0QAgCiQ5ALoKcViUX19wae+vr4+FYvFmCMCAESBJBdAT8nlcpqdnVUul1sf0c1kMuu3Z7NZxxECAMJACzEAPWdoaEiVSkXlclkLCwvq7+9XsVgkwQWAFCHJBdCTstmsRkZGXIcBAIgI5QoAAABIHZJcAAAApA5JLgAAAFKHJBcAAACpQ5ILAACA1CHJBQAAQOqQ5AIAACB1SHIBAACQOiS5AAAASB2SXAAAAKQOSS4AAABShyQXAAAAqUOSCwAAgNQhyQUAAEDqkOQCAAAgdUhyAQAAkDokuQAAAEgdklwAAACkDkkuAAAAUockFwAAAKlDkgsAAIDU2e46AAAAACRXtVpVuVzW/Py8BgYGVCwWlcvlXIdFkgsAAICtmZubU6FQ0Orqqmq1mjKZjMbHxzU7O6uhoSGnsVGuAAAAgI5Vq1UVCgVVq1XVajVJUq1WW799aWnJaXwkuQAAAOhYuVzW6upq4LHV1VWVy+WYI7ocSS4AAAA6Nj8/vz6Cu1mtVtPCwkLMEV3OeZJrjHmPMeZFY8yrxphXjDH3uo4JAAAAzQ0MDCiTyQQey2Qy6u/vjzmiyzlPciVdlPQla+3PSbpF0mFjzPWOYwIAAEATxWJRfX3BqWRfX5+KxWLMEV3OeXcFa+2ipMW1f1eNMa9Kykv6K6eBwSlf25EAAIBLcrmcZmdn39Vdoa+vT7Ozs8pms07jM9ZapwFsZIy5RtKfSbrBWvt2o/sdPHjQnjx5Mq6wELOgdiT1N4zrdiQAAOByS0tLKpfLWlhYUH9/v4rFYmwJrjHmJWvtwcBjviS5xpispD+V9DVr7bcDjt8t6W5JOnDgwE2vvfZazBEiDtVqVfl8XtVq9V3HcrmcKpXK+hunl0d7e/lvBwCgzvsk1xizQ9J3JX3PWvv1VvdnJDe9SqWSxsbGAldrZjIZTU9Pa2RkpKdHe3v5bwcAYKNmSa7zhWfGGCPp30l6tZ0EF+nWTjsS35tPR6mX/3YAADrhPMmV9CFJn5f0YWPMD9e+Cq6DghvttCPxvfl0lHr5bwcAoBPOk1xr7Zy11lhr32+t/cDa16zruOBGO+1IfG8+HaVe/tsBAOiE8yQX2KjejiSXy62P6GYymfXbs9ms982no9TLfzsAAJ3wYuFZp1h4ln7N2pF00oEhbXr5bwcAYLNmC8+cbwYBBMlmsxoZGQk85nvz6Sj18t8OAEAnGMlFYrlsPu1aL//tAADUed8nt1MkuQAAAPC6Ty4AAAAQNpJcAAAApA5JLoDYVSoVDQ8P6+abb9bw8LAqlYrrkAAAKUNNbopVq1WVy2XNz89rYGBAxWJRuVzOdVjocceOHdPhw4ffdfvMzIxGR0cdRAQASCoWnvWgubm5hm2mhoaGXIeHHlWpVJTP5xseX1xc1N69e2OMCACQZCw86zHValWFQkHVanV9C9harbZ++9LSkuMI4ZtqtapSqaSJiQmVSqXAzSbCcP/99zc9Pjk5GcnjAgB6D0luCpXLZa2urgYeW11dVblcjjki+Oy5557TVVddpS9+8Ys6evSo7r33XuXzec3NzYX+WKdOnWp6/PTp06E/JgCgN5HkptD8/Pz6CO5mtVpNCwsLMUcEXz333HP62Mc+pnPnzunixYuSpOXl5chG/a+77rqmxwcHB0N9PABA7yLJTaGBgQFlMpnAY5lMRv39/TFHBB9Vq1XdfvvtDY+vrKyEPuo/NTXV9PiRI0dCfTwAQO8iyU2hYrGovr7g/7V9fX0qFosxRwQflctlraysNDy+vLwc+qj/vn37NDMzE3hsZmbGi0VncdUnA0BYOG8F2+46AIQvl8tpdna2YXeFbDbrOkR4YH5+XhcuXGh4fPv27ZGM+o+OjuqOO+7Q5OSkTp8+rcHBQR05csSLBDeoK8n4+DhdSQB4i/NWY7QQS7GlpSWVy2UtLCyov79fxWKRBBfrSqWS7r33Xi0vLwce37Vrl37yk5/0zGumWq0qn88HjoDkcjlVKpWeeS4AJAPnreYtxBjJTbFsNquRkRHXYXTMx00sfIypW8V
|
|||
|
|
"text/plain": [
|
|||
|
|
"<Figure size 806.4x453.6 with 1 Axes>"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
"metadata": {
|
|||
|
|
"needs_background": "light"
|
|||
|
|
},
|
|||
|
|
"output_type": "display_data"
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"source": [
|
|||
|
|
"# X - dane ze zbioru \"iris\" z poprzedniego przykładu\n",
|
|||
|
|
"\n",
|
|||
|
|
"X_pca = pca(X, 2)\n",
|
|||
|
|
"fig = plot_unlabeled_data(X_pca)"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "notes"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"Analiza głównych składowych umożliwiła stworzenie powyższego wykresu, który wizualizuje 4-wymiarowe dane ze zbioru *iris* na 2-wymiarowej płaszczyźnie.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Współrzędne $x_1$ i $x_2$, stanowiące osi wykresu, zostały uzyskane w wyniku działania algorytmu PCA (nie są to żadne z oryginalnych cech ze zbioru *iris* – długość płatka, szerokość płatka itp.)."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"metadata": {
|
|||
|
|
"slideshow": {
|
|||
|
|
"slide_type": "subslide"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"source": [
|
|||
|
|
"Tutaj można zobaczyć, jak algorytmy redukcji wymiarów (w tym PCA) działają w praktyce:\n",
|
|||
|
|
" * https://projector.tensorflow.org\n",
|
|||
|
|
" * https://biit.cs.ut.ee/clustvis\n",
|
|||
|
|
" * https://labriata.github.io/jsinscience/pca/index.html"
|
|||
|
|
]
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"metadata": {
|
|||
|
|
"celltoolbar": "Slideshow",
|
|||
|
|
"kernelspec": {
|
|||
|
|
"display_name": "Python 3",
|
|||
|
|
"language": "python",
|
|||
|
|
"name": "python3"
|
|||
|
|
},
|
|||
|
|
"language_info": {
|
|||
|
|
"codemirror_mode": {
|
|||
|
|
"name": "ipython",
|
|||
|
|
"version": 3
|
|||
|
|
},
|
|||
|
|
"file_extension": ".py",
|
|||
|
|
"mimetype": "text/x-python",
|
|||
|
|
"name": "python",
|
|||
|
|
"nbconvert_exporter": "python",
|
|||
|
|
"pygments_lexer": "ipython3",
|
|||
|
|
"version": "3.8.3"
|
|||
|
|
},
|
|||
|
|
"livereveal": {
|
|||
|
|
"start_slideshow_at": "selected",
|
|||
|
|
"theme": "white"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"nbformat": 4,
|
|||
|
|
"nbformat_minor": 4
|
|||
|
|
}
|