2021-05-31 11:55:27 +02:00
{
"cells": [
2021-09-28 10:56:21 +02:00
{
"cell_type": "markdown",
"id": "7fe475ae",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
2024-04-09 09:46:32 +02:00
"## Inżynieria uczenia maszynowego\n",
"### 22 maja 2024\n",
"# 10. DVC"
2021-09-28 10:56:21 +02:00
]
},
2021-05-31 11:55:27 +02:00
{
"cell_type": "markdown",
"id": "0c6f27a5",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"<img src=\"img/expcontrol/dvc-logo.png\">"
]
},
{
"cell_type": "markdown",
"id": "560eec71",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## DVC - Data Version Control\n",
"- [dvc.org](https://dvc.org/)\n",
"- \"Version Control System for Machine Learning Projects\" (System kontroli wersji dla projektów uczenia maszynowego)\n",
"- Open Source\n",
"- Umożliwia:\n",
" - wersjonowanie danych i modeli. \"Git dla danych i modeli\"\n",
" - budowanie potoków (\"pipeline\") definiujących jak budować/trenować/ewaluować modele. \"Makefile dla uczenia maszynowego\"\n",
2022-05-23 09:57:46 +02:00
" - śledzenie, porównywanie metryk i parametrów\n",
2021-05-31 11:55:27 +02:00
"- ściśle zintegowany z gitem\n",
"- działa niezależnie od używanego języka/bibliotek i systemu operacyjnego\n",
2024-05-21 14:24:34 +02:00
"- 5-minutowe wprowadzenie: https://www.youtube.com/watch?v=UbL7VUpv1Bs"
2021-05-31 11:55:27 +02:00
]
},
2023-04-26 13:44:55 +02:00
{
"cell_type": "markdown",
"id": "3d4ce1cb",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Śledzenie plików za pomocą DVC\n",
" - dużymi plikami, takimi jak plikami z danymi wejściowymi czy plikami modeli, trudno zarządza się za pomocą gita, ze względu na problemy z:\n",
" - wydajnością\n",
" - przestrzenią w repozytorium\n",
" - ograniczenia ze strony serwisu (np. [limit 100 MB na plik w Github](https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github))\n",
" - Git posiada rozszerzenie [lfs(Large File Storage)](https://git-lfs.github.com/), które stanowi pewne rozwiązanie tego problemu. \n",
" - Same pliki przechowywane są na specjalnym zdalnym serwerze, w repozytorium przechowywane są jedynie odnośniki do tych plików i pewne metadane\n",
" - Github ma zintegrowany LFS z [limitem 1GB dla kont bezpłatnych](https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-storage-and-bandwidth-usage)"
]
},
{
"cell_type": "markdown",
"id": "dd8e529b",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
" - DVC proponuje podobne podejście co LFS, ale:\n",
" - pliki mogą być przechowywane na niemal dowolnym serwerze, również lokalnie\n",
" - brak limitu wielkości plików (w Git-LFS na Github [limit 2GB](https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-git-large-file-storage))\n",
" - DVC zapewnia dodatkowe narzędzie umożliwiające śledzenie plików i ich powiązań z wynikami eksperymentów\n",
" - więcej, patrz [tutaj](https://dvc.org/doc/user-guide/related-technologies)"
]
},
2021-05-31 11:55:27 +02:00
{
"cell_type": "markdown",
"id": "9bfb356e",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Instalacja i inicjalizacja\n",
" - https://dvc.org/doc/install\n",
2024-05-21 14:24:34 +02:00
" - ```pip install dvc```\n",
" - ```pipx install dvc```\n",
2021-05-31 11:55:27 +02:00
" - ```conda install dvc```"
]
},
{
"cell_type": "code",
2024-05-21 14:24:34 +02:00
"execution_count": 1,
2021-05-31 11:55:27 +02:00
"id": "054c7a11",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
2024-05-21 14:24:34 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: dvc in ./venv/lib/python3.10/site-packages (3.50.2)\n",
"Requirement already satisfied: attrs>=22.2.0 in ./venv/lib/python3.10/site-packages (from dvc) (23.2.0)\n",
"Requirement already satisfied: psutil>=5.8 in ./venv/lib/python3.10/site-packages (from dvc) (5.9.8)\n",
"Requirement already satisfied: zc.lockfile>=1.2.1 in ./venv/lib/python3.10/site-packages (from dvc) (3.0.post1)\n",
"Requirement already satisfied: ruamel.yaml>=0.17.11 in ./venv/lib/python3.10/site-packages (from dvc) (0.18.6)\n",
"Requirement already satisfied: dvc-http>=2.29.0 in ./venv/lib/python3.10/site-packages (from dvc) (2.32.0)\n",
"Requirement already satisfied: shortuuid>=0.5 in ./venv/lib/python3.10/site-packages (from dvc) (1.0.13)\n",
"Requirement already satisfied: platformdirs<4,>=3.1.1 in ./venv/lib/python3.10/site-packages (from dvc) (3.11.0)\n",
"Requirement already satisfied: scmrepo<4,>=3.3.2 in ./venv/lib/python3.10/site-packages (from dvc) (3.3.5)\n",
"Requirement already satisfied: pygtrie>=2.3.2 in ./venv/lib/python3.10/site-packages (from dvc) (2.5.0)\n",
"Requirement already satisfied: dvc-data<3.16,>=3.15 in ./venv/lib/python3.10/site-packages (from dvc) (3.15.1)\n",
"Requirement already satisfied: fsspec in ./venv/lib/python3.10/site-packages (from dvc) (2024.5.0)\n",
"Requirement already satisfied: dvc-objects in ./venv/lib/python3.10/site-packages (from dvc) (5.1.0)\n",
"Requirement already satisfied: grandalf<1,>=0.7 in ./venv/lib/python3.10/site-packages (from dvc) (0.8)\n",
"Requirement already satisfied: hydra-core>=1.1 in ./venv/lib/python3.10/site-packages (from dvc) (1.3.2)\n",
"Requirement already satisfied: tabulate>=0.8.7 in ./venv/lib/python3.10/site-packages (from dvc) (0.9.0)\n",
2024-05-21 14:24:34 +02:00
"Requirement already satisfied: colorama>=0.3.9 in ./venv/lib/python3.10/site-packages (from dvc) (0.4.6)\n",
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: tomlkit>=0.11.1 in ./venv/lib/python3.10/site-packages (from dvc) (0.12.5)\n",
"Requirement already satisfied: rich>=12 in ./venv/lib/python3.10/site-packages (from dvc) (13.7.1)\n",
"Requirement already satisfied: dvc-render<2,>=1.0.1 in ./venv/lib/python3.10/site-packages (from dvc) (1.0.2)\n",
"Requirement already satisfied: flufl.lock<8,>=5 in ./venv/lib/python3.10/site-packages (from dvc) (7.1.1)\n",
"Requirement already satisfied: packaging>=19 in ./venv/lib/python3.10/site-packages (from dvc) (24.0)\n",
2024-05-21 14:24:34 +02:00
"Requirement already satisfied: pyparsing>=2.4.7 in ./venv/lib/python3.10/site-packages (from dvc) (3.1.2)\n",
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: flatten-dict<1,>=0.4.1 in ./venv/lib/python3.10/site-packages (from dvc) (0.4.2)\n",
"Requirement already satisfied: dulwich in ./venv/lib/python3.10/site-packages (from dvc) (0.22.1)\n",
"Requirement already satisfied: configobj>=5.0.6 in ./venv/lib/python3.10/site-packages (from dvc) (5.0.8)\n",
"Requirement already satisfied: dvc-studio-client<1,>=0.20 in ./venv/lib/python3.10/site-packages (from dvc) (0.20.0)\n",
"Requirement already satisfied: omegaconf in ./venv/lib/python3.10/site-packages (from dvc) (2.3.0)\n",
"Requirement already satisfied: distro>=1.3 in ./venv/lib/python3.10/site-packages (from dvc) (1.9.0)\n",
"Requirement already satisfied: dpath<3,>=2.1.0 in ./venv/lib/python3.10/site-packages (from dvc) (2.1.6)\n",
"Requirement already satisfied: funcy>=1.14 in ./venv/lib/python3.10/site-packages (from dvc) (2.0)\n",
"Requirement already satisfied: dvc-task<1,>=0.3.0 in ./venv/lib/python3.10/site-packages (from dvc) (0.4.0)\n",
"Requirement already satisfied: voluptuous>=0.11.7 in ./venv/lib/python3.10/site-packages (from dvc) (0.14.2)\n",
"Requirement already satisfied: kombu in ./venv/lib/python3.10/site-packages (from dvc) (5.3.7)\n",
"Requirement already satisfied: shtab<2,>=1.3.4 in ./venv/lib/python3.10/site-packages (from dvc) (1.7.1)\n",
"Requirement already satisfied: iterative-telemetry>=0.0.7 in ./venv/lib/python3.10/site-packages (from dvc) (0.0.8)\n",
"Requirement already satisfied: celery in ./venv/lib/python3.10/site-packages (from dvc) (5.4.0)\n",
"Requirement already satisfied: pathspec>=0.10.3 in ./venv/lib/python3.10/site-packages (from dvc) (0.12.1)\n",
2024-05-21 14:24:34 +02:00
"Requirement already satisfied: requests>=2.22 in ./venv/lib/python3.10/site-packages (from dvc) (2.31.0)\n",
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: tqdm<5,>=4.63.1 in ./venv/lib/python3.10/site-packages (from dvc) (4.66.2)\n",
"Requirement already satisfied: gto<2,>=1.6.0 in ./venv/lib/python3.10/site-packages (from dvc) (1.7.1)\n",
"Requirement already satisfied: networkx>=2.5 in ./venv/lib/python3.10/site-packages (from dvc) (3.3)\n",
"Requirement already satisfied: pydot>=1.2.4 in ./venv/lib/python3.10/site-packages (from dvc) (2.0.0)\n",
2024-05-21 14:24:34 +02:00
"Requirement already satisfied: six in ./venv/lib/python3.10/site-packages (from configobj>=5.0.6->dvc) (1.16.0)\n",
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: sqltrie<1,>=0.11.0 in ./venv/lib/python3.10/site-packages (from dvc-data<3.16,>=3.15->dvc) (0.11.0)\n",
"Requirement already satisfied: dictdiffer>=0.8.1 in ./venv/lib/python3.10/site-packages (from dvc-data<3.16,>=3.15->dvc) (0.9.0)\n",
"Requirement already satisfied: diskcache>=5.2.1 in ./venv/lib/python3.10/site-packages (from dvc-data<3.16,>=3.15->dvc) (5.6.3)\n",
"Requirement already satisfied: aiohttp-retry>=2.5.0 in ./venv/lib/python3.10/site-packages (from dvc-http>=2.29.0->dvc) (2.8.3)\n",
"Requirement already satisfied: billiard<5.0,>=4.2.0 in ./venv/lib/python3.10/site-packages (from celery->dvc) (4.2.0)\n",
2024-05-21 14:24:34 +02:00
"Requirement already satisfied: python-dateutil>=2.8.2 in ./venv/lib/python3.10/site-packages (from celery->dvc) (2.9.0.post0)\n",
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: click-plugins>=1.1.1 in ./venv/lib/python3.10/site-packages (from celery->dvc) (1.1.1)\n",
"Requirement already satisfied: click-repl>=0.2.0 in ./venv/lib/python3.10/site-packages (from celery->dvc) (0.3.0)\n",
"Requirement already satisfied: vine<6.0,>=5.1.0 in ./venv/lib/python3.10/site-packages (from celery->dvc) (5.1.0)\n",
"Requirement already satisfied: tzdata>=2022.7 in ./venv/lib/python3.10/site-packages (from celery->dvc) (2024.1)\n",
"Requirement already satisfied: click<9.0,>=8.1.2 in ./venv/lib/python3.10/site-packages (from celery->dvc) (8.1.7)\n",
"Requirement already satisfied: click-didyoumean>=0.3.0 in ./venv/lib/python3.10/site-packages (from celery->dvc) (0.3.1)\n",
"Requirement already satisfied: atpublic>=2.3 in ./venv/lib/python3.10/site-packages (from flufl.lock<8,>=5->dvc) (4.1.0)\n",
"Requirement already satisfied: pydantic!=2.0.0,<3,>=1.9.0 in ./venv/lib/python3.10/site-packages (from gto<2,>=1.6.0->dvc) (2.7.1)\n",
"Requirement already satisfied: semver>=2.13.0 in ./venv/lib/python3.10/site-packages (from gto<2,>=1.6.0->dvc) (3.0.2)\n",
"Requirement already satisfied: typer>=0.4.1 in ./venv/lib/python3.10/site-packages (from gto<2,>=1.6.0->dvc) (0.12.3)\n",
"Requirement already satisfied: entrypoints in ./venv/lib/python3.10/site-packages (from gto<2,>=1.6.0->dvc) (0.4)\n",
"Requirement already satisfied: antlr4-python3-runtime==4.9.* in ./venv/lib/python3.10/site-packages (from hydra-core>=1.1->dvc) (4.9.3)\n",
"Requirement already satisfied: appdirs in ./venv/lib/python3.10/site-packages (from iterative-telemetry>=0.0.7->dvc) (1.4.4)\n",
"Requirement already satisfied: filelock in ./venv/lib/python3.10/site-packages (from iterative-telemetry>=0.0.7->dvc) (3.14.0)\n",
"Requirement already satisfied: amqp<6.0.0,>=5.1.1 in ./venv/lib/python3.10/site-packages (from kombu->dvc) (5.2.0)\n",
"Requirement already satisfied: PyYAML>=5.1.0 in ./venv/lib/python3.10/site-packages (from omegaconf->dvc) (6.0.1)\n",
2024-05-21 14:24:34 +02:00
"Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.10/site-packages (from requests>=2.22->dvc) (2024.2.2)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv/lib/python3.10/site-packages (from requests>=2.22->dvc) (2.2.1)\n",
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: charset-normalizer<4,>=2 in ./venv/lib/python3.10/site-packages (from requests>=2.22->dvc) (3.3.2)\n",
2024-05-21 14:24:34 +02:00
"Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.10/site-packages (from requests>=2.22->dvc) (3.6)\n",
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in ./venv/lib/python3.10/site-packages (from rich>=12->dvc) (2.17.2)\n",
"Requirement already satisfied: markdown-it-py>=2.2.0 in ./venv/lib/python3.10/site-packages (from rich>=12->dvc) (3.0.0)\n",
"Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in ./venv/lib/python3.10/site-packages (from ruamel.yaml>=0.17.11->dvc) (0.2.8)\n",
"Requirement already satisfied: pygit2>=1.14.0 in ./venv/lib/python3.10/site-packages (from scmrepo<4,>=3.3.2->dvc) (1.15.0)\n",
"Requirement already satisfied: asyncssh<3,>=2.13.1 in ./venv/lib/python3.10/site-packages (from scmrepo<4,>=3.3.2->dvc) (2.14.2)\n",
"Requirement already satisfied: gitpython>3 in ./venv/lib/python3.10/site-packages (from scmrepo<4,>=3.3.2->dvc) (3.1.43)\n",
2024-05-21 14:24:34 +02:00
"Requirement already satisfied: setuptools in ./venv/lib/python3.10/site-packages (from zc.lockfile>=1.2.1->dvc) (59.6.0)\n",
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: aiohttp in ./venv/lib/python3.10/site-packages (from aiohttp-retry>=2.5.0->dvc-http>=2.29.0->dvc) (3.9.5)\n",
"Requirement already satisfied: typing-extensions>=3.6 in ./venv/lib/python3.10/site-packages (from asyncssh<3,>=2.13.1->scmrepo<4,>=3.3.2->dvc) (4.11.0)\n",
"Requirement already satisfied: cryptography>=39.0 in ./venv/lib/python3.10/site-packages (from asyncssh<3,>=2.13.1->scmrepo<4,>=3.3.2->dvc) (42.0.7)\n",
"Requirement already satisfied: prompt-toolkit>=3.0.36 in ./venv/lib/python3.10/site-packages (from click-repl>=0.2.0->celery->dvc) (3.0.43)\n",
2024-05-21 14:24:34 +02:00
"Requirement already satisfied: gitdb<5,>=4.0.1 in ./venv/lib/python3.10/site-packages (from gitpython>3->scmrepo<4,>=3.3.2->dvc) (4.0.11)\n",
2024-05-22 08:17:56 +02:00
"Requirement already satisfied: mdurl~=0.1 in ./venv/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich>=12->dvc) (0.1.2)\n",
"Requirement already satisfied: pydantic-core==2.18.2 in ./venv/lib/python3.10/site-packages (from pydantic!=2.0.0,<3,>=1.9.0->gto<2,>=1.6.0->dvc) (2.18.2)\n",
"Requirement already satisfied: annotated-types>=0.4.0 in ./venv/lib/python3.10/site-packages (from pydantic!=2.0.0,<3,>=1.9.0->gto<2,>=1.6.0->dvc) (0.7.0)\n",
"Requirement already satisfied: cffi>=1.16.0 in ./venv/lib/python3.10/site-packages (from pygit2>=1.14.0->scmrepo<4,>=3.3.2->dvc) (1.16.0)\n",
"Requirement already satisfied: orjson in ./venv/lib/python3.10/site-packages (from sqltrie<1,>=0.11.0->dvc-data<3.16,>=3.15->dvc) (3.10.3)\n",
"Requirement already satisfied: shellingham>=1.3.0 in ./venv/lib/python3.10/site-packages (from typer>=0.4.1->gto<2,>=1.6.0->dvc) (1.5.4)\n",
"Requirement already satisfied: async-timeout<5.0,>=4.0 in ./venv/lib/python3.10/site-packages (from aiohttp->aiohttp-retry>=2.5.0->dvc-http>=2.29.0->dvc) (4.0.3)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in ./venv/lib/python3.10/site-packages (from aiohttp->aiohttp-retry>=2.5.0->dvc-http>=2.29.0->dvc) (1.3.1)\n",
"Requirement already satisfied: yarl<2.0,>=1.0 in ./venv/lib/python3.10/site-packages (from aiohttp->aiohttp-retry>=2.5.0->dvc-http>=2.29.0->dvc) (1.9.4)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in ./venv/lib/python3.10/site-packages (from aiohttp->aiohttp-retry>=2.5.0->dvc-http>=2.29.0->dvc) (6.0.5)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in ./venv/lib/python3.10/site-packages (from aiohttp->aiohttp-retry>=2.5.0->dvc-http>=2.29.0->dvc) (1.4.1)\n",
"Requirement already satisfied: pycparser in ./venv/lib/python3.10/site-packages (from cffi>=1.16.0->pygit2>=1.14.0->scmrepo<4,>=3.3.2->dvc) (2.22)\n",
"Requirement already satisfied: smmap<6,>=3.0.1 in ./venv/lib/python3.10/site-packages (from gitdb<5,>=4.0.1->gitpython>3->scmrepo<4,>=3.3.2->dvc) (5.0.1)\n",
"Requirement already satisfied: wcwidth in ./venv/lib/python3.10/site-packages (from prompt-toolkit>=3.0.36->click-repl>=0.2.0->celery->dvc) (0.2.13)\n"
2024-05-21 14:24:34 +02:00
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
2022-05-23 09:57:46 +02:00
"!pip3 install dvc"
2021-05-31 11:55:27 +02:00
]
},
{
"cell_type": "markdown",
"id": "20975d62",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Stwórzmy katalog, w którym będziemy przechowywać nasz projekt:"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 4,
2023-04-26 13:44:55 +02:00
"id": "4d94e912",
2021-05-31 11:55:27 +02:00
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
"outputs": [],
"source": [
2024-05-22 08:17:56 +02:00
"!rm -r -f IUM_10/sample-ml-project-2024\n",
"!mkdir -p IUM_10/sample-ml-project-2024"
2021-05-31 11:55:27 +02:00
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 5,
2023-04-26 13:44:55 +02:00
"id": "aae59ec2",
2021-05-31 11:55:27 +02:00
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/home/pawel/ium/IUM_10/sample-ml-project-2024\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/pawel/ium/venv/lib/python3.10/site-packages/IPython/core/magics/osm.py:417: UserWarning: This is now an optional IPython functionality, setting dhist requires you to install the `pickleshare` library.\n",
" self.shell.db['dhist'] = compress_dhist(dhist)[-100:]\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"#Jupyter notebook magic https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-cd\n",
2024-05-22 08:17:56 +02:00
"%cd \"IUM_10/sample-ml-project-2024\""
2021-05-31 11:55:27 +02:00
]
},
{
"cell_type": "markdown",
"id": "199c0d92",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Inicjalizujemy puste repozytorium Git (możemy też pominąć ten krok i działać w istniejącym już repozytorium)"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 7,
2021-05-31 11:55:27 +02:00
"id": "c13c525b",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Reinitialized existing Git repository in /home/pawel/ium/IUM_10/sample-ml-project-2024/.git/\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!git init"
]
},
{
"cell_type": "markdown",
"id": "c7155369",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Teraz inicjalizujemy repozytorium DVC:"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 8,
2021-05-31 11:55:27 +02:00
"id": "44f28226",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Initialized DVC repository.\n",
"\n",
"You can now commit the changes to git.\n",
"\n",
"\u001b[31m+---------------------------------------------------------------------+\n",
"\u001b[0m\u001b[31m|\u001b[0m \u001b[31m|\u001b[0m\n",
"\u001b[31m|\u001b[0m DVC has enabled anonymous aggregate usage analytics. \u001b[31m|\u001b[0m\n",
"\u001b[31m|\u001b[0m Read the analytics documentation (and how to opt-out) here: \u001b[31m|\u001b[0m\n",
"\u001b[31m|\u001b[0m <\u001b[36mhttps://dvc.org/doc/user-guide/analytics\u001b[39m> \u001b[31m|\u001b[0m\n",
"\u001b[31m|\u001b[0m \u001b[31m|\u001b[0m\n",
"\u001b[31m+---------------------------------------------------------------------+\n",
"\u001b[0m\n",
"\u001b[33mWhat's next?\u001b[39m\n",
"\u001b[33m------------\u001b[39m\n",
"- Check out the documentation: <\u001b[36mhttps://dvc.org/doc\u001b[39m>\n",
"- Get help and share ideas: <\u001b[36mhttps://dvc.org/chat\u001b[39m>\n",
"- Star us on GitHub: <\u001b[36mhttps://github.com/iterative/dvc\u001b[39m>\n",
"\u001b[0m"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!dvc init"
]
},
{
"cell_type": "markdown",
"id": "00bc72ed",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Zobaczmy jakie pliki dodał (również do repozytorium git) DVC.\n",
"Ich opis znajdziemy tutaj: https://dvc.org/doc/user-guide/project-structure/internal-files"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 9,
2021-05-31 11:55:27 +02:00
"id": "d1aefe16",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"On branch master\n",
"\n",
"No commits yet\n",
"\n",
"Changes to be committed:\n",
" (use \"git rm --cached <file>...\" to unstage)\n",
"\t\u001b[32mnew file: .dvc/.gitignore\u001b[m\n",
"\t\u001b[32mnew file: .dvc/config\u001b[m\n",
"\t\u001b[32mnew file: .dvcignore\u001b[m\n",
"\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!git status"
]
},
2023-04-26 13:44:55 +02:00
{
"cell_type": "markdown",
"id": "b16a62e6",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
2024-05-22 08:17:56 +02:00
"- `.dvc/config` - główny plik konfiguracyjny DVC\n",
"- `.dvc/config.local` - nadpisuje wartości z `config`, do lokalnych zmian niecommitowanych do repozytorium\n",
"- `.dvc/.gitignore` - pliki DVC, które nie mają znaleźć się w repo\n",
"- `.dvcignore` - DVC pomija pliki zdefiniowane w tym pliku (np. aby poprawić wydajność)"
2023-04-26 13:44:55 +02:00
]
},
2021-05-31 11:55:27 +02:00
{
"cell_type": "markdown",
"id": "72e0a272",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Możemy teraz zacommitować zmiany w git:"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 10,
2021-05-31 11:55:27 +02:00
"id": "59780e99",
"metadata": {
2022-05-29 09:51:07 +02:00
"scrolled": true,
2021-05-31 11:55:27 +02:00
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[master (root-commit) a9746ad] Initial commit\n",
" 3 files changed, 6 insertions(+)\n",
" create mode 100644 .dvc/.gitignore\n",
" create mode 100644 .dvc/config\n",
" create mode 100644 .dvcignore\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!git commit -m \"Initial commit\""
]
},
{
"cell_type": "markdown",
"id": "a8861abe",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Przygotujmy przykładowe dane, pobierając je z Kaggle:"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 11,
2021-05-31 11:55:27 +02:00
"id": "f05ece1b",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Downloading iris.zip to /home/pawel/ium/IUM_10/sample-ml-project-2024\n",
" 0%| | 0.00/3.60k [00:00<?, ?B/s]\n",
"100%|██████████████████████████████████████| 3.60k/3.60k [00:00<00:00, 8.23MB/s]\n",
"Archive: iris.zip\n",
" inflating: Iris.csv \n",
" inflating: database.sqlite \n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!kaggle datasets download -d uciml/iris\n",
"!unzip -o iris.zip\n",
"!rm database.sqlite iris.zip\n",
"!mkdir -p data\n",
"!mv Iris.csv data/"
]
},
{
"cell_type": "markdown",
"id": "adb9a522",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Teraz dodamy plik(i) z danymi do DVC:"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 12,
2021-05-31 11:55:27 +02:00
"id": "74d182c7",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[?25l\u001b[32m⠋\u001b[0m Checking graph core\u001b[39m>\n",
"Adding... \n",
"!\u001b[A\n",
"Collecting files and computing hashes in data/Iris.csv |0.00 [00:00, ?file/s\u001b[A\n",
" \u001b[A\n",
"!\u001b[A\n",
" 0% Checking cache in '/home/pawel/ium/IUM_10/sample-ml-project-2024/.dvc/cache\u001b[A\n",
" \u001b[A\n",
"!\u001b[A\n",
" 0%| |Adding data/Iris.csv to cache 0/1 [00:00<?, ?file/s]\u001b[A\n",
" \u001b[A\n",
"!\u001b[A\n",
" 0%| |Checking out /home/pawel/ium/IUM_10/sa0/1 [00:00<?, ?files/s]\u001b[A\n",
"100% Adding...|████████████████████████████████████████|1/1 [00:00, 31.90file/s]\u001b[A\n",
"\n",
"To track the changes with git, run:\n",
"\n",
"\tgit add data/.gitignore data/Iris.csv.dvc\n",
"\n",
"To enable auto staging, run:\n",
"\n",
"\tdvc config core.autostage true\n",
"\u001b[0m"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!dvc add data/Iris.csv"
]
},
{
"cell_type": "markdown",
"id": "72c6b5d0",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
2024-05-22 08:17:56 +02:00
" - DVC utworzył plik `data/Iris.csv.dvc` i dodał oryginalny plik do `.gitignore`\n",
2021-05-31 11:55:27 +02:00
" - W repozytorium będzie obecny tylko plik `*.dvc`, zawierający odnośnik do prawdziwego pliku"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 13,
2021-05-31 11:55:27 +02:00
"id": "74d54652",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"On branch master\n",
"Untracked files:\n",
" (use \"git add <file>...\" to include in what will be committed)\n",
"\t\u001b[31mdata/.gitignore\u001b[m\n",
"\t\u001b[31mdata/Iris.csv.dvc\u001b[m\n",
"\n",
"nothing added to commit but untracked files present (use \"git add\" to track)\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!git status -u"
]
},
{
"cell_type": "markdown",
"id": "8589fecf",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Dodajmy pliki `data/Iris.csv.dvc data/.gitignore` do repozytorium git, zgodnie z sugestią DVC:"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 14,
2021-05-31 11:55:27 +02:00
"id": "460c4a17",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
"outputs": [],
"source": [
"!git add data/Iris.csv.dvc data/.gitignore"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 15,
2021-05-31 11:55:27 +02:00
"id": "80644077",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[master 92b2c9d] Dodano dane IRIS (DVC)\n",
" 2 files changed, 6 insertions(+)\n",
" create mode 100644 data/.gitignore\n",
" create mode 100644 data/Iris.csv.dvc\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!git commit -m \"Dodano dane IRIS (DVC)\""
]
},
{
"cell_type": "markdown",
"id": "03899863",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Plik `*.dvc` zawiera m.in. hash pliku. Więcej o plikach `*.dvc`: [link](https://dvc.org/doc/user-guide/project-structure/dvc-files)"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 16,
2021-05-31 11:55:27 +02:00
"id": "8cb2ba7c",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
"outputs": [],
"source": [
2023-04-26 17:35:34 +02:00
"# %load data/Iris.csv.dvc\n"
2021-05-31 11:55:27 +02:00
]
},
{
"cell_type": "markdown",
"id": "0b421d45",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Oryginalny plik `Iris.csv` został przeniesiony do katalogu ./dvc/cache/{wartość hash pliku) i podlinkowany z powrotem do oryginalnej lokalizacji. Sposób tworzenia linków może być [różny w zależności od systemu plików](https://dvc.org/doc/user-guide/large-dataset-optimization)."
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 18,
2021-05-31 11:55:27 +02:00
"id": "1d471f3a",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"total 8\n",
"-r--r--r-- 1 pawel pawel 5107 Sep 19 2019 7820ef0af287ff346c5cabfb4c612c\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
2024-05-22 08:17:56 +02:00
"!ls -l .dvc/cache/files/md5/71"
2021-05-31 11:55:27 +02:00
]
},
2022-05-23 09:57:46 +02:00
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 19,
2023-04-26 13:44:55 +02:00
"id": "32531aa8",
2022-05-23 09:57:46 +02:00
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2022-05-23 09:57:46 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Id,SepalLengthCm,SepalWidthCm,PetalLengthCm,PetalWidthCm,Species\n",
"1,5.1,3.5,1.4,0.2,Iris-setosa\n",
"2,4.9,3.0,1.4,0.2,Iris-setosa\n"
]
}
],
2023-04-26 13:44:55 +02:00
"source": [
2024-05-22 08:17:56 +02:00
"!head -n 3 .dvc/cache/files/md5/71/7820ef0af287ff346c5cabfb4c612c"
2022-05-23 09:57:46 +02:00
]
},
2021-05-31 11:55:27 +02:00
{
"cell_type": "markdown",
"id": "901e8e90",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## dvc remote\n",
2024-05-22 08:17:56 +02:00
" - Żeby wysłać właściwe pliki śledzone przez DVC do zdalnej lokalizacji (z której będą mogłby być pobrane np. przez system CI albo innych użytkowników), musimy mieć skonfigurowaną taką lokazliację.\n",
" - Służy do tego polecenie [`dvc remote add`](https://dvc.org/doc/command-reference/remote/add).\n",
2024-05-22 08:38:20 +02:00
" - Użyjemy lokalnego \"remote\". Tutaj będzie to po prostu utworzony wcześniej katalog `~/dvcstore`. Taki katalog istnieje też na naszym Jenkinsie, oczywiście należy go podmontować w Dockerze.\n",
2024-05-22 08:17:56 +02:00
" - W rzeczywistych zastosowaniach podalibyśmy tutaj ścieżkę do jakiegoś zasobu dostępnego przez internet jak np. serwer SFTP, ścieżka do AWS S3 itp."
2021-05-31 11:55:27 +02:00
]
},
2022-05-23 09:57:46 +02:00
{
"cell_type": "markdown",
2022-05-29 09:51:07 +02:00
"id": "53429521",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
2022-05-23 09:57:46 +02:00
"source": [
"Obsługiwane typy zdalnych lokalizacji (remotes): https://dvc.org/doc/command-reference/remote/add#supported-storage-types\n",
" - Amazon S3\n",
" - S3-compatible storage\n",
" - Microsoft Azure Blob Storage\n",
" - Google Drive\n",
" - Google Cloud Storage\n",
" - Aliyun OSS\n",
" - SSH\n",
" - HDFS\n",
" - WebHDFS\n",
" - HTTP\n",
" - WebDAV\n",
" - local remote"
]
},
2023-04-26 13:44:55 +02:00
{
"cell_type": "markdown",
"id": "507e3a09",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"#### Dodawanie remote typu local"
]
},
2021-05-31 11:55:27 +02:00
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 30,
2022-05-29 09:51:07 +02:00
"id": "a16f2bfa",
2021-05-31 11:55:27 +02:00
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Setting 'my_local_remote' as a default remote.\n",
"\u001b[0m"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
2024-05-22 08:17:56 +02:00
"!dvc remote add -d my_local_remote ~/dvcstore"
2021-05-31 11:55:27 +02:00
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 31,
2021-05-31 11:55:27 +02:00
"id": "9c3deeaf",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"On branch master\n",
"Changes not staged for commit:\n",
" (use \"git add <file>...\" to update what will be committed)\n",
" (use \"git restore <file>...\" to discard changes in working directory)\n",
"\t\u001b[31mmodified: .dvc/config\u001b[m\n",
"\n",
"no changes added to commit (use \"git add\" and/or \"git commit -a\")\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!git status"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 32,
2021-05-31 11:55:27 +02:00
"id": "899eac7d",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[master 7123494] Added DVC remote\n",
" 1 file changed, 1 insertion(+), 1 deletion(-)\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!git add .dvc/config\n",
"!git commit -m \"Added DVC remote\""
]
},
{
"cell_type": "markdown",
"id": "8c556c96",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## dvc push\n",
"Kiedy mamy już skonfigurowany \"remote\" możemy wypchnąć do niego pliki korzystając z polecenia `dvc push`:"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 33,
2021-06-04 15:52:21 +02:00
"id": "c7f24f75",
2021-09-28 10:56:21 +02:00
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-09-28 10:56:21 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting |1.00 [00:00, 137entry/s]\n",
"Pushing\n",
"!\u001b[A\n",
" 0% Checking cache in '/home/pawel/dvcstore/files/md5'| |0/? [00:00<?, ?file\u001b[A\n",
" \u001b[A\n",
"!\u001b[A\n",
" 0% Checking cache in '/home/pawel/ium/IUM_10/sample-ml-project-2024/.dvc/cache\u001b[A\n",
" \u001b[A\n",
"!\u001b[A\n",
" 0%| |Pushing to local 0/1 [00:00<?, ?file/s]\u001b[A\n",
"Pushing \u001b[A\n",
"1 file pushed\n",
"\u001b[0m"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
"!dvc push"
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 35,
2021-05-31 11:55:27 +02:00
"id": "8a355575",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[01;34m/home/pawel/dvcstore\u001b[0m\n",
"└── \u001b[01;34mfiles\u001b[0m\n",
" └── \u001b[01;34mmd5\u001b[0m\n",
" └── \u001b[01;34m71\u001b[0m\n",
" └── 7820ef0af287ff346c5cabfb4c612c\n",
"\n",
"3 directories, 1 file\n"
]
}
],
2021-05-31 11:55:27 +02:00
"source": [
2024-05-22 08:17:56 +02:00
"!tree ~/dvcstore"
2021-05-31 11:55:27 +02:00
]
},
{
"cell_type": "markdown",
"id": "af59ecb3",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## dvc pull\n",
"Żeby pobrać dane z DVC (np. w innej lokalizacji, przez innego użytkownika), musimy:\n",
" - sklonować repozytorium git (żeby m.in. pobrać pliki `*.dvc`\n",
" - wykonać `dvc pull`"
]
},
{
"cell_type": "markdown",
"id": "9fa914a7",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
2022-05-23 09:57:46 +02:00
"Dodawanie nowych plików i modyfikacja istniejących wygląda podobnie jak przy zwykłych plikach śledzonych przez git, tylko zamiast `git` używamy polecenia `dvc` a dodatkowo pamiętamy o zarządzaniu plikami `*.dvc` za pomocą gita:"
2021-05-31 11:55:27 +02:00
]
},
{
"cell_type": "code",
2024-05-22 08:17:56 +02:00
"execution_count": 37,
2021-05-31 11:55:27 +02:00
"id": "dde39796",
"metadata": {
"slideshow": {
2023-04-26 13:44:55 +02:00
"slide_type": "fragment"
2021-05-31 11:55:27 +02:00
}
},
2024-05-22 08:17:56 +02:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Id,SepalLengthCm,SepalWidthCm,PetalLengthCm,PetalWidthCm,Species\n",
"1,5.1,3.5,1.4,0.2,Iris-setosa\n",
"2,4.9,3.0,1.4,0.2,Iris-setosa\n",
"3,4.7,3.2,1.3,0.2,Iris-setosa\n",
"4,4.6,3.1,1.5,0.2,Iris-setosa\n",
"5,5.0,3.6,1.4,0.2,Iris-setosa\n",
"6,5.4,3.9,1.7,0.4,Iris-setosa\n",
"7,4.6,3.4,1.4,0.3,Iris-setosa\n",
"8,5.0,3.4,1.5,0.2,Iris-setosa\n",
"9,4.4,2.9,1.4,0.2,Iris-setosa\n",
"10,4.9,3.1,1.5,0.1,Iris-setosa\n",
"11,5.4,3.7,1.5,0.2,Iris-setosa\n",
"12,4.8,3.4,1.6,0.2,Iris-setosa\n",
"13,4.8,3.0,1.4,0.1,Iris-setosa\n",
"14,4.3,3.0,1.1,0.1,Iris-setosa\n",
"15,5.8,4.0,1.2,0.2,Iris-setosa\n",
"16,5.7,4.4,1.5,0.4,Iris-setosa\n",
"17,5.4,3.9,1.3,0.4,Iris-setosa\n",
"18,5.1,3.5,1.4,0.3,Iris-setosa\n",
"19,5.7,3.8,1.7,0.3,Iris-setosa\n",
"20,5.1,3.8,1.5,0.3,Iris-setosa\n",
"21,5.4,3.4,1.7,0.2,Iris-setosa\n",
"22,5.1,3.7,1.5,0.4,Iris-setosa\n",
"23,4.6,3.6,1.0,0.2,Iris-setosa\n",
"24,5.1,3.3,1.7,0.5,Iris-setosa\n",
"25,4.8,3.4,1.9,0.2,Iris-setosa\n",
"26,5.0,3.0,1.6,0.2,Iris-setosa\n",
"27,5.0,3.4,1.6,0.4,Iris-setosa\n",
"28,5.2,3.5,1.5,0.2,Iris-setosa\n",
"29,5.2,3.4,1.4,0.2,Iris-setosa\n",
"30,4.7,3.2,1.6,0.2,Iris-setosa\n",
"31,4.8,3.1,1.6,0.2,Iris-setosa\n",
"32,5.4,3.4,1.5,0.4,Iris-setosa\n",
"33,5.2,4.1,1.5,0.1,Iris-setosa\n",
"34,5.5,4.2,1.4,0.2,Iris-setosa\n",
"35,4.9,3.1,1.5,0.1,Iris-setosa\n",
"36,5.0,3.2,1.2,0.2,Iris-setosa\n",
"37,5.5,3.5,1.3,0.2,Iris-setosa\n",
"38,4.9,3.1,1.5,0.1,Iris-setosa\n",
"39,4.4,3.0,1.3,0.2,Iris-setosa\n",
"40,5.1,3.4,1.5,0.2,Iris-setosa\n",
"41,5.0,3.5,1.3,0.3,Iris-setosa\n",
"42,4.5,2.3,1.3,0.3,Iris-setosa\n",
"43,4.4,3.2,1.3,0.2,Iris-setosa\n",
"44,5.0,3.5,1.6,0.6,Iris-setosa\n",
"45,5.1,3.8,1.9,0.4,Iris-setosa\n",
"46,4.8,3.0,1.4,0.3,Iris-setosa\n",
"47,5.1,3.8,1.6,0.2,Iris-setosa\n",
"48,4.6,3.2,1.4,0.2,Iris-setosa\n",
"49,5.3,3.7,1.5,0.2,Iris-setosa\n",
"50,5.0,3.3,1.4,0.2,Iris-setosa\n",
"51,7.0,3.2,4.7,1.4,Iris-versicolor\n",
"52,6.4,3.2,4.5,1.5,Iris-versicolor\n",
"53,6.9,3.1,4.9,1.5,Iris-versicolor\n",
"54,5.5,2.3,4.0,1.3,Iris-versicolor\n",
"55,6.5,2.8,4.6,1.5,Iris-versicolor\n",
"56,5.7,2.8,4.5,1.3,Iris-versicolor\n",
"57,6.3,3.3,4.7,1.6,Iris-versicolor\n",
"58,4.9,2.4,3.3,1.0,Iris-versicolor\n",
"59,6.6,2.9,4.6,1.3,Iris-versicolor\n",
"60,5.2,2.7,3.9,1.4,Iris-versicolor\n",
"61,5.0,2.0,3.5,1.0,Iris-versicolor\n",
"62,5.9,3.0,4.2,1.5,Iris-versicolor\n",
"63,6.0,2.2,4.0,1.0,Iris-versicolor\n",
"64,6.1,2.9,4.7,1.4,Iris-versicolor\n",
"65,5.6,2.9,3.6,1.3,Iris-versicolor\n",
"66,6.7,3.1,4.4,1.4,Iris-versicolor\n",
"67,5.6,3.0,4.5,1.5,Iris-versicolor\n",
"68,5.8,2.7,4.1,1.0,Iris-versicolor\n",
"69,6.2,2.2,4.5,1.5,Iris-versicolor\n",
"70,5.6,2.5,3.9,1.1,Iris-versicolor\n",
"71,5.9,3.2,4.8,1.8,Iris-versicolor\n",
"72,6.1,2.8,4.0,1.3,Iris-versicolor\n",
"73,6.3,2.5,4.9,1.5,Iris-versicolor\n",
"74,6.1,2.8,4.7,1.2,Iris-versicolor\n",
"75,6.4,2.9,4.3,1.3,Iris-versicolor\n",
"76,6.6,3.0,4.4,1.4,Iris-versicolor\n",
