AITech/aitech-ium
112
8
Fork 2
forked from tzietkiewicz/aitech-ium
Code Issues Pull Requests Releases Wiki Activity

Merge pull request 'Dane konfiguracji serwera MLflow registry' (#13) from tzietkiewicz/aitech-ium:master into master

Browse Source 
Reviewed-on: #13
This commit is contained in:
Tomasz Zietkiewicz 2021-05-19 22:57:14 +02:00
commit 3b9dc60bda
1 changed files with 45 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
IUM_08.MLFlow.ipynb
View File

@ -78,7 +78,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 30, "execution_count": 80,
"metadata": { "metadata": {
"slideshow": { "slideshow": {
"slide_type": "slide" "slide_type": "slide"
@ -117,7 +117,8 @@
"logging.basicConfig(level=logging.WARN)\n", "logging.basicConfig(level=logging.WARN)\n",
"logger = logging.getLogger(__name__)\n", "logger = logging.getLogger(__name__)\n",
"\n", "\n",
"#mlflow.set_tracking_uri(\"http://localhost:5001\")\n", "mlflow.set_tracking_uri(\"http://localhost:5000\")\n",
"mlflow.set_experiment(\"s123456\")\n",
"\n", "\n",
"def eval_metrics(actual, pred):\n", "def eval_metrics(actual, pred):\n",
" rmse = np.sqrt(mean_squared_error(actual, pred))\n", " rmse = np.sqrt(mean_squared_error(actual, pred))\n",
@ -156,7 +157,10 @@
" l1_ratio = float(sys.argv[2]) if len(sys.argv) > 2 else 0.5\n", " l1_ratio = float(sys.argv[2]) if len(sys.argv) > 2 else 0.5\n",
" #l1_ratio = 0.5\n", " #l1_ratio = 0.5\n",
"\n", "\n",
" with mlflow.start_run():\n", " with mlflow.start_run() as run:\n",
" print(\"MLflow run experiment_id: {0}\".format(run.info.experiment_id))\n",
" print(\"MLflow run artifact_uri: {0}\".format(run.info.artifact_uri))\n",
"\n",
" lr = ElasticNet(alpha=alpha, l1_ratio=l1_ratio, random_state=42)\n", " lr = ElasticNet(alpha=alpha, l1_ratio=l1_ratio, random_state=42)\n",
" lr.fit(train_x, train_y)\n", " lr.fit(train_x, train_y)\n",
"\n", "\n",
@ -187,14 +191,14 @@
" # There are other ways to use the Model Registry, which depends on the use case,\n", " # There are other ways to use the Model Registry, which depends on the use case,\n",
" # please refer to the doc for more information:\n", " # please refer to the doc for more information:\n",
" # https://mlflow.org/docs/latest/model-registry.html#api-workflow\n", " # https://mlflow.org/docs/latest/model-registry.html#api-workflow\n",
" mlflow.sklearn.log_model(lr, \"model\", registered_model_name=\"ElasticnetWineModel\", signature=signature)\n", " mlflow.sklearn.log_model(lr, \"wines-model\", registered_model_name=\"ElasticnetWineModel\", signature=signature)\n",
" else:\n", " else:\n",
" mlflow.sklearn.log_model(lr, \"model\", signature=signature)" " mlflow.sklearn.log_model(lr, \"model\", signature=signature)"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 31, "execution_count": 81,
"metadata": { "metadata": {
"slideshow": { "slideshow": {
"slide_type": "slide" "slide_type": "slide"
@ -205,14 +209,23 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"Elasticnet model (alpha=0.500000, l1_ratio=0.500000):\r\n", "total 4\n",
" RMSE: 0.7931640229276851\r\n", "drwxrwxr-x 3 tomek tomek 4096 maj 19 21:31 1\n",
" MAE: 0.6271946374319586\r\n", "INFO: 's123456' does not exist. Creating a new experiment\n",
" R2: 0.10862644997792614\r\n" "MLflow run experiment_id: 2\n",
"MLflow run artifact_uri: /tmp/mlruns/2/c15feb5df335490ba990ddd4dd977c1b/artifacts\n",
"Elasticnet model (alpha=0.500000, l1_ratio=0.500000):\n",
" RMSE: 0.7931640229276851\n",
" MAE: 0.6271946374319586\n",
" R2: 0.10862644997792614\n",
"Registered model 'ElasticnetWineModel' already exists. Creating a new version of this model...\n",
"2021/05/19 22:34:48 INFO mlflow.tracking._model_registry.client: Waiting up to 300 seconds for model version to finish creation. Model name: ElasticnetWineModel, version 2\n",
"Created version '2' of model 'ElasticnetWineModel'.\n"
] ]
} }
], ],
"source": [ "source": [
"! ls -l /tmp/mlruns\n",
"### Wtyrenujmy model z domyślnymi wartościami parametrów\n", "### Wtyrenujmy model z domyślnymi wartościami parametrów\n",
"! cd ./IUM_08/examples/; python sklearn_elasticnet_wine/train.py" "! cd ./IUM_08/examples/; python sklearn_elasticnet_wine/train.py"
] ]
@ -1374,6 +1387,29 @@
"\n", "\n",
"4. [6 pkt] Stwórz na Jenkinsie projekt `s123456-predict-s654321-from-registry`, który zrealizuje to samo zadanie co `s123456-predict-s654321`, ale tym razem pobierze model z rejestru MLflow zamiast z artefaktów Jenkinsa" "4. [6 pkt] Stwórz na Jenkinsie projekt `s123456-predict-s654321-from-registry`, który zrealizuje to samo zadanie co `s123456-predict-s654321`, ale tym razem pobierze model z rejestru MLflow zamiast z artefaktów Jenkinsa"
] ]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"Dane do konfiguracji MLflow registry\n",
"\n",
"- Podgląd w przeglądarce: http://tzietkiewicz.vm.wmi.amu.edu.pl/#/\n",
" - user: `student`\n",
" - hasło: Podane na MS Teams\n",
"\n",
"- Tracking URI:\n",
" - Python: `mlflow.set_tracking_uri(\"http://172.17.0.1:5000\")`\n",
" - CLI: `export MLFLOW_TRACKING_URI=http://172.17.0.1:5000`\n",
" \n",
"- Żeby klient MLflow działający w kontenerze docker mógł zapisywać i pdczytywać artefakty, muszą Państwo podmonotwać katalog /tmp/mlrunsMożna to zrobić za pomocą flagi `-v`, którą można przekazać tak, jak pokazano tutaj: https://www.jenkins.io/doc/book/pipeline/docker/#caching-data-for-containers\n",
"- Proszę ustawić nazwę eksperymentu na numer indeksu, dzięki temu każdy z Państwa będzie widział swoje eksperymenty oddzielnie:\n",
"`mlflow.set_experiment(\"s123456\")`"
]
} }
], ],
"metadata": { "metadata": {