forked from tzietkiewicz/aitech-ium
9081 lines
804 KiB
Plaintext
9081 lines
804 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## \n",
|
|
"<div class=\"alert alert-block alert-info\">\n",
|
|
"<h1> Inżynieria uczenia maszynowego </h1>\n",
|
|
"<h2> 8. <i>MLFlow</i> [laboratoria]</h2> \n",
|
|
"<h3> Tomasz Ziętkiewicz (2023)</h3>\n",
|
|
"</div>\n",
|
|
"\n",
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
" ## MLflow\n",
|
|
" </br><img style=\"width: 50%;\" src=\"img/expcontrol/mlflow-logo-d.png\"/>\n",
|
|
"\n",
|
|
" - https://mlflow.org/\n",
|
|
" - Narzędzie podobne do omawianego na poprzednich zajęciach Sacred\n",
|
|
" - Nieco inne podejście: mniej ingerencji w istniejący kod\n",
|
|
" - Bardziej kompleksowe rozwiązanie: 4 komponenty, pierwszy z nich ma funkcjonalność podobną do Sacred\n",
|
|
" - Działa \"z każdym\" językiem. A tak naprawdę: Python, R, Java + CLI API + REST API\n",
|
|
" - Popularna wśród pracodawców - wyniki wyszukiwania ofert pracy: 20 ofert (https://pl.indeed.com/), 36 ofert (linkedin). Sacred: 0\n",
|
|
" - Integracja z licznymi bibliotekami / chmurami\n",
|
|
" - Rozwiązanie OpenSource, stworzone przez firmę Databricks\n",
|
|
" - Dostępna [odpłatna wersja \"Managed\"](https://databricks.com/product/managed-mlflow) (w ordóżnieniu od \"self-hosted\")\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## Komponenty\n",
|
|
"\n",
|
|
"MLflow składa się z czterech niezależnych komponentów:\n",
|
|
" - **MLflow Tracking** - pozwala śledzić zmiany parametrów, kodu, środowiska i ich wpływ na metryki. Jest to funkcjonalność bardzo zbliżona do tej, którą zapewnia Sacred\n",
|
|
" - **MLflow Projects** - umożliwia \"pakowanie\" kodu ekserymentów w taki sposób, żeby mogłby być w łatwy sposób zreprodukowane przez innych\n",
|
|
" - **MLflow Models** - ułatwia \"pakowanie\" modeli uczenia maszynowego\n",
|
|
" - **MLflow Registry** - zapewnia centralne miejsce do przechowywania i współdzielenia modeli. Zapewnia narzędzia do wersjonowania i śledzenia pochodzenia tych modeli.\n",
|
|
" \n",
|
|
"Komponenty te mogą być używane razem bądź oddzielnie."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## MLflow Tracking - przykład\n",
|
|
"(poniższe przykłady kodu trenującego pochodzą z tutoriala MLflow: https://mlflow.org/docs/latest/tutorials-and-examples/tutorial.html)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%capture null\n",
|
|
"!pip install mlflow\n",
|
|
"!pip install sklearn"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!mkdir -p IUM_08/examples/sklearn_elasticnet_wine/"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Overwriting IUM_08/examples/sklearn_elasticnet_wine/train.py\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%writefile IUM_08/examples/sklearn_elasticnet_wine/train.py\n",
|
|
"# The data set used in this example is from http://archive.ics.uci.edu/ml/datasets/Wine+Quality\n",
|
|
"# P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis.\n",
|
|
"# Modeling wine preferences by data mining from physicochemical properties. In Decision Support Systems, Elsevier, 47(4):547-553, 2009.\n",
|
|
"\n",
|
|
"import os\n",
|
|
"import warnings\n",
|
|
"import sys\n",
|
|
"\n",
|
|
"import pandas as pd\n",
|
|
"import numpy as np\n",
|
|
"from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score\n",
|
|
"from sklearn.model_selection import train_test_split\n",
|
|
"from sklearn.linear_model import ElasticNet\n",
|
|
"from urllib.parse import urlparse\n",
|
|
"import mlflow\n",
|
|
"import mlflow.sklearn\n",
|
|
"\n",
|
|
"import logging\n",
|
|
"\n",
|
|
"logging.basicConfig(level=logging.WARN)\n",
|
|
"logger = logging.getLogger(__name__)\n",
|
|
"\n",
|
|
"mlflow.set_tracking_uri(\"http://localhost:5000\")\n",
|
|
"mlflow.set_experiment(\"s123456\")\n",
|
|
"\n",
|
|
"def eval_metrics(actual, pred):\n",
|
|
" rmse = np.sqrt(mean_squared_error(actual, pred))\n",
|
|
" mae = mean_absolute_error(actual, pred)\n",
|
|
" r2 = r2_score(actual, pred)\n",
|
|
" return rmse, mae, r2\n",
|
|
"\n",
|
|
"\n",
|
|
"if __name__ == \"__main__\":\n",
|
|
" warnings.filterwarnings(\"ignore\")\n",
|
|
" np.random.seed(40)\n",
|
|
"\n",
|
|
" # Read the wine-quality csv file from the URL\n",
|
|
" csv_url = (\n",
|
|
" \"http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv\"\n",
|
|
" )\n",
|
|
" try:\n",
|
|
" data = pd.read_csv(csv_url, sep=\";\")\n",
|
|
" except Exception as e:\n",
|
|
" logger.exception(\n",
|
|
" \"Unable to download training & test CSV, check your internet connection. Error: %s\", e\n",
|
|
" )\n",
|
|
"\n",
|
|
" # Split the data into training and test sets. (0.75, 0.25) split.\n",
|
|
" train, test = train_test_split(data)\n",
|
|
"\n",
|
|
" # The predicted column is \"quality\" which is a scalar from [3, 9]\n",
|
|
" train_x = train.drop([\"quality\"], axis=1)\n",
|
|
" test_x = test.drop([\"quality\"], axis=1)\n",
|
|
" train_y = train[[\"quality\"]]\n",
|
|
" test_y = test[[\"quality\"]]\n",
|
|
"\n",
|
|
" \n",
|
|
" alpha = float(sys.argv[1]) if len(sys.argv) > 1 else 0.5\n",
|
|
" #alpha = 0.5\n",
|
|
" l1_ratio = float(sys.argv[2]) if len(sys.argv) > 2 else 0.5\n",
|
|
" #l1_ratio = 0.5\n",
|
|
"\n",
|
|
" with mlflow.start_run() as run:\n",
|
|
" print(\"MLflow run experiment_id: {0}\".format(run.info.experiment_id))\n",
|
|
" print(\"MLflow run artifact_uri: {0}\".format(run.info.artifact_uri))\n",
|
|
"\n",
|
|
" lr = ElasticNet(alpha=alpha, l1_ratio=l1_ratio, random_state=42)\n",
|
|
" lr.fit(train_x, train_y)\n",
|
|
"\n",
|
|
" predicted_qualities = lr.predict(test_x)\n",
|
|
"\n",
|
|
" (rmse, mae, r2) = eval_metrics(test_y, predicted_qualities)\n",
|
|
"\n",
|
|
" print(\"Elasticnet model (alpha=%f, l1_ratio=%f):\" % (alpha, l1_ratio))\n",
|
|
" print(\" RMSE: %s\" % rmse)\n",
|
|
" print(\" MAE: %s\" % mae)\n",
|
|
" print(\" R2: %s\" % r2)\n",
|
|
"\n",
|
|
" mlflow.log_param(\"alpha\", alpha)\n",
|
|
" mlflow.log_param(\"l1_ratio\", l1_ratio)\n",
|
|
" mlflow.log_metric(\"rmse\", rmse)\n",
|
|
" mlflow.log_metric(\"r2\", r2)\n",
|
|
" mlflow.log_metric(\"mae\", mae)\n",
|
|
" \n",
|
|
" # Infer model signature to log it\n",
|
|
" # Więcej o sygnaturach: https://mlflow.org/docs/latest/models.html?highlight=signature#model-signature\n",
|
|
" signature = mlflow.models.signature.infer_signature(train_x, lr.predict(train_x))\n",
|
|
"\n",
|
|
" tracking_url_type_store = urlparse(mlflow.get_tracking_uri()).scheme\n",
|
|
"\n",
|
|
" # Model registry does not work with file store\n",
|
|
" if tracking_url_type_store != \"file\":\n",
|
|
"\n",
|
|
" # Register the model\n",
|
|
" # There are other ways to use the Model Registry, which depends on the use case,\n",
|
|
" # please refer to the doc for more information:\n",
|
|
" # https://mlflow.org/docs/latest/model-registry.html#api-workflow\n",
|
|
" mlflow.sklearn.log_model(lr, \"wines-model\", registered_model_name=\"ElasticnetWineModel\", signature=signature)\n",
|
|
" else:\n",
|
|
" mlflow.sklearn.log_model(lr, \"model\", signature=signature)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 81,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"total 4\n",
|
|
"drwxrwxr-x 3 tomek tomek 4096 maj 19 21:31 1\n",
|
|
"INFO: 's123456' does not exist. Creating a new experiment\n",
|
|
"MLflow run experiment_id: 2\n",
|
|
"MLflow run artifact_uri: /tmp/mlruns/2/c15feb5df335490ba990ddd4dd977c1b/artifacts\n",
|
|
"Elasticnet model (alpha=0.500000, l1_ratio=0.500000):\n",
|
|
" RMSE: 0.7931640229276851\n",
|
|
" MAE: 0.6271946374319586\n",
|
|
" R2: 0.10862644997792614\n",
|
|
"Registered model 'ElasticnetWineModel' already exists. Creating a new version of this model...\n",
|
|
"2021/05/19 22:34:48 INFO mlflow.tracking._model_registry.client: Waiting up to 300 seconds for model version to finish creation. Model name: ElasticnetWineModel, version 2\n",
|
|
"Created version '2' of model 'ElasticnetWineModel'.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"! ls -l /tmp/mlruns\n",
|
|
"### Wtyrenujmy model z domyślnymi wartościami parametrów\n",
|
|
"! cd ./IUM_08/examples/; python sklearn_elasticnet_wine/train.py"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d0c7e520>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d0c7e340>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d0c7ed00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d0c7ecd0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d0ce8640>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f14d0ce8430>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d0ce8430>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d0ce8430>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d0ce8430>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb9a5088280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb9a5088970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb9a5088a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb9a5088d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb9a50ef3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fb9a50ef340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb9a50ef340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb9a50ef340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb9a50ef340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6b4a6b5280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6b4a6b5970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6b4a6b5a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6b4a6b5d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6b4a71c3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6b4a71c340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6b4a71c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6b4a71c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6b4a71c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feff2503280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feff2503970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feff2503a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feff2503d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feff256a3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7feff256a340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feff256a340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feff256a340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feff256a340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f6fec5280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f6fec5970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f6fec5a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f6fec5d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f6ff2c3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6f6ff2c340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f6ff2c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f6ff2c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f6ff2c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870312a280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870312a970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870312aa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870312ad30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f87031913a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f8703191340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8703191340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8703191340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8703191340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f48842ea280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f48842ea970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f48842eaa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f48842ead30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f48843513a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f4884351340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4884351340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4884351340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4884351340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbabcb03280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbabcb03970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbabcb03a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbabcb03d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbabcb6a3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbabcb6a340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbabcb6a340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbabcb6a340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbabcb6a340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa41326d280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa41326d970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa41326da00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa41326dd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4132d43a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa4132d4340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4132d4340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4132d4340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4132d4340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d52df5280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d52df5970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d52df5a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d52df5d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d52e5c3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3d52e5c340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d52e5c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d52e5c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d52e5c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ef444c280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ef444c970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ef444ca00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ef444cd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ef44b33a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6ef44b3340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ef44b3340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ef44b3340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6ef44b3340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f427017f280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f427017f970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f427017fa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f427017fd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f42701e63a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f42701e6340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f42701e6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f42701e6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f42701e6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f575cd57280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f575cd57970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f575cd57a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f575cd57d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f575cdbe3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f575cdbe340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f575cdbe340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f575cdbe340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f575cdbe340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f45572df280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f45572df970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f45572dfa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f45572dfd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f45573463a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f4557346340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4557346340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4557346340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4557346340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e0bdcc280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e0bdcc970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e0bdcca00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e0bdccd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e0be333a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0e0be33340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e0be33340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e0be33340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e0be33340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9c9145f280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9c9145f970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9c9145fa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9c9145fd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9c914c63a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f9c914c6340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9c914c6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9c914c6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9c914c6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f92a909e280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f92a909e970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f92a909ea00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f92a909ed30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f92a91053a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f92a9105340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f92a9105340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f92a9105340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f92a9105340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc3c0166280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc3c0166970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc3c0166a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc3c0166d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc3c01cd3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fc3c01cd340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc3c01cd340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc3c01cd340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc3c01cd340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f49c9766190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f49c97660a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f49c9766a60>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f49c9766fa0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f49c97cd280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f49c97cd2e0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f49c97cd2e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f49c97cd2e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f49c97cd2e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f899b767280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f899b767970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f899b767a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f899b767d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f899b7ce3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f899b7ce340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f899b7ce340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f899b7ce340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f899b7ce340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f797ac3e280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f797ac3e970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f797ac3ea00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f797ac3ed30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f797aca53a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f797aca5340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f797aca5340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f797aca5340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f797aca5340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efeaa7bf280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efeaa7bf970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efeaa7bfa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efeaa7bfd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efeaa8263a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7efeaa826340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efeaa826340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efeaa826340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efeaa826340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1c0a7d1190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1c0a7d10a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1c0a7d1a60>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1c0a7d1fa0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1c0a838280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f1c0a8382e0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1c0a8382e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1c0a8382e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1c0a8382e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cf9d28280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cf9d28970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cf9d28a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cf9d28d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cf9d8f3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f2cf9d8f340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cf9d8f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cf9d8f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2cf9d8f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86cf391190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86cf3910a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86cf391a60>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86cf391fa0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86cf3f8280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f86cf3f82e0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86cf3f82e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86cf3f82e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f86cf3f82e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff43920c280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff43920c970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff43920ca00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff43920cd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff4392733a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff439273340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff439273340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff439273340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff439273340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7af1be190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7af1be0a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7af1bea60>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7af1befa0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7af225280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fb7af2252e0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7af2252e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7af2252e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7af2252e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fabac5fe280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fabac5fe970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fabac5fea00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fabac5fed30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fabac6653a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fabac665340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fabac665340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fabac665340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fabac665340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe42728280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe42728970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe42728a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe42728d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe4278f3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbe4278f340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe4278f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe4278f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe4278f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f88bbe6f280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f88bbe6f970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f88bbe6fa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f88bbe6fd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f88bbed63a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f88bbed6340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f88bbed6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f88bbed6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f88bbed6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d4583f280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d4583f970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d4583fa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d4583fd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d458a63a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f8d458a6340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d458a6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d458a6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d458a6340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f36e189a280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f36e189a970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f36e189aa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f36e189ad30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f36e19013a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f36e1901340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f36e1901340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f36e1901340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f36e1901340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc0a8112280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc0a8112970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc0a8112a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc0a8112d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc0a81793a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fc0a8179340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc0a8179340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc0a8179340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc0a8179340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa66d73a280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa66d73a970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa66d73aa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa66d73ad30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa66d7a13a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa66d7a1340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa66d7a1340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa66d7a1340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa66d7a1340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd15623f190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd15623f0a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd15623fa60>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd15623ffa0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd1562a6280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fd1562a62e0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd1562a62e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd1562a62e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd1562a62e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f41afcfa190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f41afcfa0a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f41afcfaa60>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f41afcfafa0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f41afd61280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f41afd612e0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f41afd612e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f41afd612e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f41afd612e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0c113f5280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0c113f5970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0c113f5a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0c113f5d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0c1145c3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0c1145c340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0c1145c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0c1145c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0c1145c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f332131f280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f332131f970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f332131fa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f332131fd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f33213863a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3321386340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3321386340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3321386340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3321386340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f40dfc5a280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f40dfc5a970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f40dfc5aa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f40dfc5ad30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f40dfcc13a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f40dfcc1340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f40dfcc1340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f40dfcc1340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f40dfcc1340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa5101a7280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa5101a7970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa5101a7a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa5101a7d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa51020e3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa51020e340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa51020e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa51020e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa51020e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcd78c51280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcd78c51970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcd78c51a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcd78c51d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcd78cb83a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fcd78cb8340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcd78cb8340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcd78cb8340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcd78cb8340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f426b375280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f426b375970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f426b375a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f426b375d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f426b3dc3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f426b3dc340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f426b3dc340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f426b3dc340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f426b3dc340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffb3b4ea280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffb3b4ea970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffb3b4eaa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffb3b4ead30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffb3b5513a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ffb3b551340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffb3b551340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffb3b551340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffb3b551340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f756c143280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f756c143970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f756c143a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f756c143d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f756c1aa3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f756c1aa340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f756c1aa340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f756c1aa340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f756c1aa340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff0b08be280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff0b08be970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff0b08bea00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff0b08bed30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff0b09253a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff0b0925340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff0b0925340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff0b0925340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff0b0925340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3f6cb51280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3f6cb51970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3f6cb51a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3f6cb51d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3f6cbb83a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3f6cbb8340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3f6cbb8340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3f6cbb8340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3f6cbb8340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f810732a280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f810732a970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f810732aa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f810732ad30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f81073913a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f8107391340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8107391340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8107391340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8107391340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffaf3dbf280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffaf3dbf970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffaf3dbfa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffaf3dbfd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffaf3e263a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ffaf3e26340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffaf3e26340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffaf3e26340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffaf3e26340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f739cb7e280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f739cb7e970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f739cb7ea00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f739cb7ed30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f739cbe53a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f739cbe5340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f739cbe5340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f739cbe5340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f739cbe5340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f30ba0cc280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f30ba0cc970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f30ba0cca00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f30ba0ccd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f30ba1333a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f30ba133340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f30ba133340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f30ba133340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f30ba133340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0f9d9e7280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0f9d9e7970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0f9d9e7a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0f9d9e7d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0f9da4e3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0f9da4e340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0f9da4e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0f9da4e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0f9da4e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58dcbad280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58dcbad970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58dcbada00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58dcbadd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58dcc143a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f58dcc14340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58dcc14340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58dcc14340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58dcc14340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc52f8fa280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc52f8fa970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc52f8faa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc52f8fad30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc52f9613a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fc52f961340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc52f961340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc52f961340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc52f961340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9185f0c280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9185f0c970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9185f0ca00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9185f0cd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9185f733a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f9185f73340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9185f73340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9185f73340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9185f73340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f59522280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f59522970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f59522a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f59522d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f595893a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6f59589340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f59589340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f59589340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f59589340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f501c76d280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f501c76d970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f501c76da00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f501c76dd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f501c7d43a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f501c7d4340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f501c7d4340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f501c7d4340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f501c7d4340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f532de64280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f532de64970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f532de64a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f532de64d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f532decb3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f532decb340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f532decb340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f532decb340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f532decb340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7500d4c280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7500d4c970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7500d4ca00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7500d4cd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7500db33a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7500db3340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7500db3340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7500db3340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7500db3340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe3590c8280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe3590c8970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe3590c8a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe3590c8d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe35912f3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe35912f340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe35912f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe35912f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe35912f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa00712a280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa00712a970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa00712aa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa00712ad30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa0071913a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa007191340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa007191340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa007191340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa007191340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2690997280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2690997970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2690997a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2690997d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f26909fe3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f26909fe340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f26909fe340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f26909fe340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f26909fe340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1fad147280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1fad147970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1fad147a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1fad147d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1fad1ae3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f1fad1ae340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1fad1ae340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1fad1ae340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1fad1ae340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5125d4c280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5125d4c970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5125d4ca00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5125d4cd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5125db33a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f5125db3340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5125db3340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5125db3340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5125db3340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ea15f5280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ea15f5970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ea15f5a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ea15f5d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ea165c3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7ea165c340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ea165c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ea165c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ea165c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fae93f28280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fae93f28970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fae93f28a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fae93f28d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fae93f8f3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fae93f8f340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fae93f8f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fae93f8f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fae93f8f340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fad2760c280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fad2760c970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fad2760ca00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fad2760cd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fad276733a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fad27673340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fad27673340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fad27673340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fad27673340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd380fad280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd380fad970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd380fada00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd380fadd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd3810143a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fd381014340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd381014340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd381014340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd381014340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0de685e190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0de685e0a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0de685ea60>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0de685efa0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0de68c5280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0de68c52e0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0de68c52e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0de68c52e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0de68c52e0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffada3a7280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffada3a7970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffada3a7a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffada3a7d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffada40e3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ffada40e340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffada40e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffada40e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffada40e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7959330280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7959330970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7959330a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7959330d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f79593973a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7959397340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7959397340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7959397340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7959397340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdad2d03280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdad2d03970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdad2d03a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdad2d03d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdad2d6a3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fdad2d6a340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdad2d6a340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdad2d6a340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdad2d6a340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9b67d9a280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9b67d9a970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9b67d9aa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9b67d9ad30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9b67e013a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f9b67e01340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9b67e01340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9b67e01340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9b67e01340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f566d3cc280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f566d3cc970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f566d3cca00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f566d3ccd30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f566d4333a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f566d433340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f566d433340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f566d433340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f566d433340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5d160d5280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5d160d5970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5d160d5a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5d160d5d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5d1613c3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f5d1613c340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5d1613c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5d1613c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5d1613c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f85eb500280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f85eb500970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f85eb500a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f85eb500d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f85eb5673a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f85eb567340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f85eb567340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f85eb567340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f85eb567340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6a194d9280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6a194d9970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6a194d9a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6a194d9d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6a195403a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6a19540340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6a19540340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6a19540340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6a19540340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe549507280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe549507970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe549507a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe549507d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe54956e3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe54956e340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe54956e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe54956e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe54956e340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f46bb15e280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f46bb15e970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f46bb15ea00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f46bb15ed30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f46bb1c53a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f46bb1c5340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f46bb1c5340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f46bb1c5340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f46bb1c5340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d1c25a280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d1c25a970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d1c25aa00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d1c25ad30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d1c2c13a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f2d1c2c1340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d1c2c1340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d1c2c1340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d1c2c1340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffa7fab5280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffa7fab5970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffa7fab5a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffa7fab5d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffa7fb1c3a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ffa7fb1c340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffa7fb1c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffa7fb1c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffa7fb1c340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e71891280>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e71891970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e71891a00>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e71891d30>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e718f83a0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 174, in _new_conn\n",
|
|
" conn = connection.create_connection(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 95, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 703, in urlopen\n",
|
|
" httplib_response = self._make_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 398, in _make_request\n",
|
|
" conn.request(method, url, **httplib_request_kw)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 239, in request\n",
|
|
" super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1285, in request\n",
|
|
" self._send_request(method, url, body, headers, encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1331, in _send_request\n",
|
|
" self.endheaders(body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1280, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 1040, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/http/client.py\", line 980, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 205, in connect\n",
|
|
" conn = self._new_conn()\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connection.py\", line 186, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f5e718f8340>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 440, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 813, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/connectionpool.py\", line 785, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/urllib3/util/retry.py\", line 592, in increment\n",
|
|
" raise MaxRetryError(_pool, url, error or ResponseError(cause))\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e718f8340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 174, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 105, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 529, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/sessions.py\", line 645, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e718f8340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/tomek/repos/aitech-ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/fluent.py\", line 113, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/client.py\", line 461, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/tracking/_tracking_service/client.py\", line 220, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 304, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/store/tracking/rest_store.py\", line 56, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 283, in call_endpoint\n",
|
|
" response = http_request(\n",
|
|
" File \"/home/tomek/miniconda3/lib/python3.9/site-packages/mlflow/utils/rest_utils.py\", line 192, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e718f8340>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"### I jeszcze raz, tym razem ze zmienionymi wartościami parametrów\n",
|
|
"! cd ./IUM_08/examples/; for l in {1..9}; do for a in {1..9}; do python sklearn_elasticnet_wine/train.py 0.$a 0.$l; done; done"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 32,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"total 16\r\n",
|
|
"drwxrwxr-x 6 tomek tomek 4096 maj 17 08:43 375cde31bdd44a45a91fd7cee92ebcda\r\n",
|
|
"drwxrwxr-x 6 tomek tomek 4096 maj 17 10:38 b395b55b47fc43de876b67f5a4a5dae9\r\n",
|
|
"drwxrwxr-x 6 tomek tomek 4096 maj 17 09:15 b3ead42eca964113b29e7e5f8bcb7bb7\r\n",
|
|
"-rw-rw-r-- 1 tomek tomek 151 maj 17 08:43 meta.yaml\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"### Informacje o przebieagach eksperymentu zostały zapisane w katalogu mlruns\n",
|
|
"! ls -l IUM_08/examples/mlruns/0 | head"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 33,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"total 20\r\n",
|
|
"drwxrwxr-x 3 tomek tomek 4096 maj 17 08:43 artifacts\r\n",
|
|
"-rw-rw-r-- 1 tomek tomek 423 maj 17 08:43 meta.yaml\r\n",
|
|
"drwxrwxr-x 2 tomek tomek 4096 maj 17 08:43 metrics\r\n",
|
|
"drwxrwxr-x 2 tomek tomek 4096 maj 17 08:43 params\r\n",
|
|
"drwxrwxr-x 2 tomek tomek 4096 maj 17 08:43 tags\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"! ls -l IUM_08/examples/mlruns/0/375cde31bdd44a45a91fd7cee92ebcda"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[2021-05-16 17:58:43 +0200] [118029] [INFO] Starting gunicorn 20.1.0\n",
|
|
"[2021-05-16 17:58:43 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:43 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:44 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:44 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:45 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:45 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:46 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:46 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:47 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:47 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:48 +0200] [118029] [ERROR] Can't connect to ('127.0.0.1', 5000)\n",
|
|
"Running the mlflow server failed. Please see the logs above for details.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"### Możemy je obejrzeć w przeglądarce uruchamiając interfejs webowy:\n",
|
|
"### (powinniśmy to wywołać w normalnej konsoli, w jupyter będziemy mieli zablokowany kernel)\n",
|
|
"! cd IUM_08/examples/; mlflow ui"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Instancja na naszym serwerze: http://tzietkiewicz.vm.wmi.amu.edu.pl:5000/#/"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Wygląd interfejsu webowego\n",
|
|
"<img width=\"75%\" src=\"IUM_08/mlflowui.png\"/>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Porównywanie wyników\n",
|
|
"<img width=\"75%\" src=\"IUM_08/compare-metrics.png\"/>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## Logowanie\n",
|
|
" - logowania metryk i parametrów można dokonać m.in. poprzez wywołania Python-owego API: `mlflow.log_param()` i `mlflow.log_metric()`. Więcej dostępnych funkcji: [link](https://mlflow.org/docs/latest/tracking.html#logging-functions)\n",
|
|
" - wywołania te muszą nastąpić po wykonaniu [`mlflow.start_run()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.start_run), najlepiej wewnątrz bloku:\n",
|
|
"```python\n",
|
|
" with mlflow.start_run():\n",
|
|
" \n",
|
|
" #[...]\n",
|
|
"\n",
|
|
" mlflow.log_param(\"alpha\", alpha)\n",
|
|
" mlflow.log_param(\"l1_ratio\", l1_ratio)\n",
|
|
"```\n",
|
|
" - jest też możliwość automatycznego logwania dla wybranych bibliotek: https://mlflow.org/docs/latest/tracking.html#automatic-logging"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"# MLflow Projects\n",
|
|
" - MLflow projects to zestaw konwencji i kilku narzędzi\n",
|
|
" - ułatwiają one uruchamianie eskperymentów"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Konfiguracja projektu\n",
|
|
" - W pliku `MLproject` zapisuje się konfigurację projektu ([specyfikacja](https://mlflow.org/docs/latest/projects.html))\n",
|
|
" - Zawiera ona:\n",
|
|
" - odnośnik do środowiska, w którym ma być wywołany eksperyment [szczegóły](https://mlflow.org/docs/latest/projects.html#specifying-an-environment):\n",
|
|
" - nazwa obrazu Docker\n",
|
|
" - albo ścieżka do pliku conda.yaml definiującego środowisko wykonania Conda\n",
|
|
" - parametry, z którymi można wywołać eksperyment\n",
|
|
" - polecenia służące do wywołania eksperymentu"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Overwriting IUM_08/examples/sklearn_elasticnet_wine/MLproject\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%writefile IUM_08/examples/sklearn_elasticnet_wine/MLproject\n",
|
|
"name: tutorial\n",
|
|
"\n",
|
|
"conda_env: conda.yaml #ścieżka do pliku conda.yaml z definicją środowiska\n",
|
|
" \n",
|
|
"#docker_env:\n",
|
|
"# image: mlflow-docker-example-environment\n",
|
|
"\n",
|
|
"entry_points:\n",
|
|
" main:\n",
|
|
" parameters:\n",
|
|
" alpha: {type: float, default: 0.5}\n",
|
|
" l1_ratio: {type: float, default: 0.1}\n",
|
|
" command: \"python train.py {alpha} {l1_ratio}\"\n",
|
|
" test:\n",
|
|
" parameters:\n",
|
|
" alpha: {type: cutoff, default: 0}\n",
|
|
" command: \"python test.py {cutoff}\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Środowisko Conda\n",
|
|
" </br><img style=\"height: 50px;\" src=\"img/environments/conda.png\"/>\n",
|
|
" - https://docs.conda.io\n",
|
|
" - Składnia plików conda.yaml definiujących środowisko: https://docs.conda.io/projects/conda/en/4.6.1/user-guide/tasks/manage-environments.html#create-env-file-manually\n",
|
|
" - Składnia YAML: [przystępnie](https://learnxinyminutes.com/docs/yaml/), [oficjalnie](https://yaml.org/spec/1.2/spec.html)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Overwriting IUM_08/examples/sklearn_elasticnet_wine/conda.yaml\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%writefile IUM_08/examples/sklearn_elasticnet_wine/conda.yaml\n",
|
|
"name: tutorial\n",
|
|
"channels:\n",
|
|
" - defaults\n",
|
|
"dependencies:\n",
|
|
" - python=3.6 #Te zależności będą zainstalowane za pomocą conda isntall\n",
|
|
" - pip\n",
|
|
" - pip: #Te ząś za pomocą pip install\n",
|
|
" - scikit-learn==0.23.2\n",
|
|
" - mlflow>=1.0"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Środowisko docker\n",
|
|
"- zamiast środowiska Conda możemy również podać nazwę obrazu docker, w którym ma być wywołany eksperyment.\n",
|
|
"- obraz będzie szukany lokalnie a następnie na DockerHub, lub w innym repozytorium dockera\n",
|
|
"- składnia specyfikacji ścieżki jest taka sama jak w przypadki poleceń dockera, np. docker pull [link](https://docs.docker.com/engine/reference/commandline/pull/#pull-from-a-different-registry)\n",
|
|
"- Można również podać katalogi do podmontowania wewnątrz kontenera oraz wartości zmiennych środowiskowych do ustawienia w kontenerze:\n",
|
|
"```yaml\n",
|
|
"docker_env:\n",
|
|
" image: mlflow-docker-example-environment\n",
|
|
" volumes: [\"/local/path:/container/mount/path\"]\n",
|
|
" environment: [[\"NEW_ENV_VAR\", \"new_var_value\"], \"VAR_TO_COPY_FROM_HOST_ENVIRONMENT\"]\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Parametry\n",
|
|
" - Specyfikacja parametrów w pliku MLproject pozwala na ich walidację i używanie wartości domyślnych\n",
|
|
" - Dostępne typy:\n",
|
|
" - String\n",
|
|
" - Float - dowolna liczba (MLflow waliduje, czy podana wartość jest liczbą)\n",
|
|
" - Path - pozwala podawać ścieżki względne (przekształca je na bezwzlędne) do plików lokalnych albo do plików zdalnych (np. do s3://) - zostaną wtedy ściągnięte lokalnie\n",
|
|
" - URI - podobnie jak path, ale do rozproszonych systemów plików\n",
|
|
"\n",
|
|
"- [Składnia](https://mlflow.org/docs/latest/projects.html#specifying-parameters)\n",
|
|
" \n",
|
|
"```yml:\n",
|
|
" parameter_name: {type: data_type, default: value} # Short syntax\n",
|
|
"\n",
|
|
" parameter_name: # Long syntax\n",
|
|
" type: data_type\n",
|
|
" default: value\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Uruchamianie projektu\n",
|
|
" - Projekt możemy uruchomić przy pomocy polecenia `mlflow run` ([dokumentacja](https://mlflow.org/docs/latest/cli.html#mlflow-run))\n",
|
|
" - Spowoduje to przygotowanie środowiska i uruchomienie eksperymentu wewnątrz środowiska\n",
|
|
" - domyślnie zostanie uruchomione polecenie zdefiniowane w \"entry point\" `main`. Żeby uruchomić inny \"entry point\", możemy użyć parametru `-e`, np:\n",
|
|
" ```bash\n",
|
|
" mlflow run sklearn_elasticnet_wine -e test\n",
|
|
" ```\n",
|
|
" - Parametry do naszego polecenia możemy przekazywać przy pomocy flagi `-P`"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"2021/05/16 17:59:10 INFO mlflow.projects.utils: === Created directory /tmp/tmprq4mdosv for downloading remote URIs passed to arguments of type 'path' ===\n",
|
|
"2021/05/16 17:59:10 INFO mlflow.projects.backend.local: === Running command 'source /home/tomek/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-5987e03d4dbaa5faa1a697bb113be9b9bdc39b29 1>&2 && python train.py 0.42 0.1' in run with ID '1860d321ea1545ff8866e4ba199d1712' === \n",
|
|
"Elasticnet model (alpha=0.420000, l1_ratio=0.100000):\n",
|
|
" RMSE: 0.7420620899060748\n",
|
|
" MAE: 0.5722846717246247\n",
|
|
" R2: 0.21978513651550236\n",
|
|
"2021/05/16 17:59:19 INFO mlflow.projects: === Run (ID '1860d321ea1545ff8866e4ba199d1712') succeeded ===\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"!cd IUM_08/examples/; mlflow run sklearn_elasticnet_wine -P alpha=0.42"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"# Zadania [10p pkt]\n",
|
|
"1. Dodaj do swojego projektu logowanie parametrów i metryk za pomocą MLflow (polecenia `mlflow.log_param` i `mlflow.log_metric`\n",
|
|
"2. Dodaj plik MLProject definiujący polecenia do trenowania i testowania, ich parametry wywołania oraz środowisko (Conda albo Docker)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## MLflow Models\n",
|
|
"\n",
|
|
"MLflow Models to konwencja zapisu modeli, która ułatwia potem ich załadowanie i użycie"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"Rodzaje modeli (\"flavors\") wspierane przez MLflow:\n",
|
|
"\n",
|
|
" - Python Function (python_function)\n",
|
|
" - PyTorch (pytorch)\n",
|
|
" - TensorFlow (tensorflow)\n",
|
|
" - Keras (keras)\n",
|
|
" - Scikit-learn (sklearn)\n",
|
|
" - Spacy(spaCy)\n",
|
|
" - ONNX (onnx)\n",
|
|
" - R Function (crate)\n",
|
|
" - H2O (h2o)\n",
|
|
" - MLeap (mleap)\n",
|
|
" - Spark MLlib (spark)\n",
|
|
" - MXNet Gluon (gluon)\n",
|
|
" - XGBoost (xgboost)\n",
|
|
" - LightGBM (lightgbm)\n",
|
|
" - CatBoost (catboost)\n",
|
|
" - Fastai(fastai)\n",
|
|
" - Statsmodels (statsmodels)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Zapisywanie modelu\n",
|
|
"Model ML można zapisać w MLflow przy pomocy jednej z dwóch funkcji z pakietu odpowiadającego używanej przez nas bibliotece:\n",
|
|
" - `save_model()` - zapisuje model na dysku\n",
|
|
" - `log_model()` - zapisuje model razem z innymi informacjami (metrykami, parametrami). W zależności od ustawień [\"tracking_uri\"](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.set_tracking_uri) może być to lokalny folder w `mlruns/ ` lub ścieżka na zdalnym serwerze MLflow\n",
|
|
"\n",
|
|
"```Python\n",
|
|
" mlflow.sklearn.save_model(lr, \"my_model\")\n",
|
|
"```\n",
|
|
"\n",
|
|
"```Python\n",
|
|
" mlflow.keras.save_model(lr, \"my_model\")\n",
|
|
"```\n",
|
|
"\n",
|
|
"Wywołanie tej funkcji spowoduje stworzenie katalogu \"my_model\" zawierającego:\n",
|
|
" - plik *MLmodel* zawierający informacje o sposobach, w jaki model można załadować (\"flavors\") oraz ścieżki do plików związanych z modelem, takich jak:\n",
|
|
" - *conda.yaml* - opis środowiska potrzebnego do załadowania modelu\n",
|
|
" - *model.pkl* - plik z zserializowanym modelem\n",
|
|
"\n",
|
|
"Tylko plik *MLmodel* jest specjalnym plikiem MLflow - reszta zależy od konkrentego \"flavour\"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"conda.yaml MLmodel model.pkl\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"ls IUM_08/examples/my_model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 39,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"total 12\r\n",
|
|
"-rw-rw-r-- 1 tomek tomek 153 maj 17 10:38 conda.yaml\r\n",
|
|
"-rw-rw-r-- 1 tomek tomek 958 maj 17 10:38 MLmodel\r\n",
|
|
"-rw-rw-r-- 1 tomek tomek 641 maj 17 10:38 model.pkl\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"! ls -l IUM_08/examples/mlruns/0/b395b55b47fc43de876b67f5a4a5dae9/artifacts/model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# %load IUM_08/examples/mlruns/0/b395b55b47fc43de876b67f5a4a5dae9/artifacts/model/MLmodel\n",
|
|
"artifact_path: model\n",
|
|
"flavors:\n",
|
|
" python_function:\n",
|
|
" env: conda.yaml\n",
|
|
" loader_module: mlflow.sklearn\n",
|
|
" model_path: model.pkl\n",
|
|
" python_version: 3.9.1\n",
|
|
" sklearn:\n",
|
|
" pickled_model: model.pkl\n",
|
|
" serialization_format: cloudpickle\n",
|
|
" sklearn_version: 0.24.2\n",
|
|
"run_id: b395b55b47fc43de876b67f5a4a5dae9\n",
|
|
"signature:\n",
|
|
" inputs: '[{\"name\": \"fixed acidity\", \"type\": \"double\"}, {\"name\": \"volatile acidity\",\n",
|
|
" \"type\": \"double\"}, {\"name\": \"citric acid\", \"type\": \"double\"}, {\"name\": \"residual\n",
|
|
" sugar\", \"type\": \"double\"}, {\"name\": \"chlorides\", \"type\": \"double\"}, {\"name\": \"free\n",
|
|
" sulfur dioxide\", \"type\": \"double\"}, {\"name\": \"total sulfur dioxide\", \"type\": \"double\"},\n",
|
|
" {\"name\": \"density\", \"type\": \"double\"}, {\"name\": \"pH\", \"type\": \"double\"}, {\"name\":\n",
|
|
" \"sulphates\", \"type\": \"double\"}, {\"name\": \"alcohol\", \"type\": \"double\"}]'\n",
|
|
" outputs: '[{\"type\": \"tensor\", \"tensor-spec\": {\"dtype\": \"float64\", \"shape\": [-1]}}]'\n",
|
|
"utc_time_created: '2021-05-17 08:38:41.749670'\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# %load IUM_08/examples/my_model/conda.yaml\n",
|
|
"channels:\n",
|
|
"- defaults\n",
|
|
"- conda-forge\n",
|
|
"dependencies:\n",
|
|
"- python=3.9.1\n",
|
|
"- pip\n",
|
|
"- pip:\n",
|
|
" - mlflow\n",
|
|
" - scikit-learn==0.24.2\n",
|
|
" - cloudpickle==1.6.0\n",
|
|
"name: mlflow-env"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Dodatkowe pola w MLmodel\n",
|
|
"\n",
|
|
"\n",
|
|
"- *utc_time_created* - timestamp z czasem stworzenia modelu\n",
|
|
"- *run_id* - ID uruchomienia (\"run\"), które stworzyło ten model, jeśli model był zapisany za pomocą MLflow Tracking.\n",
|
|
"- *signature* - opisa danych wejściowych i wyjściowych w formacie JSON\n",
|
|
"- *input_example* przykładowe wejście przyjmowane przez model. Można je podać poprzez parametr `input_example` funkcji [log_model](https://mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.log_model)\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 50,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"array([5.57688397, 5.50664777, 5.52550482, 5.50431125, 5.57688397])"
|
|
]
|
|
},
|
|
"execution_count": 50,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import mlflow\n",
|
|
"import pandas as pd\n",
|
|
"model = mlflow.sklearn.load_model(\"IUM_08/examples/mlruns/0/b395b55b47fc43de876b67f5a4a5dae9/artifacts/model\")\n",
|
|
"csv_url = \"http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv\"\n",
|
|
"data = pd.read_csv(csv_url, sep=\";\")\n",
|
|
"model.predict(data.drop([\"quality\"], axis=1).head())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Serwowanie modeli"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Usage: mlflow models [OPTIONS] COMMAND [ARGS]...\r\n",
|
|
"\r\n",
|
|
" Deploy MLflow models locally.\r\n",
|
|
"\r\n",
|
|
" To deploy a model associated with a run on a tracking server, set the\r\n",
|
|
" MLFLOW_TRACKING_URI environment variable to the URL of the desired server.\r\n",
|
|
"\r\n",
|
|
"Options:\r\n",
|
|
" --help Show this message and exit.\r\n",
|
|
"\r\n",
|
|
"Commands:\r\n",
|
|
" build-docker **EXPERIMENTAL**: Builds a Docker image whose default...\r\n",
|
|
" predict Generate predictions in json format using a saved MLflow...\r\n",
|
|
" prepare-env **EXPERIMENTAL**: Performs any preparation necessary to...\r\n",
|
|
" serve Serve a model saved with MLflow by launching a webserver on...\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"!cd IUM_08/examples/; mlflow models --help"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Usage: mlflow models serve [OPTIONS]\r\n",
|
|
"\r\n",
|
|
" Serve a model saved with MLflow by launching a webserver on the specified\r\n",
|
|
" host and port. The command supports models with the ``python_function`` or\r\n",
|
|
" ``crate`` (R Function) flavor. For information about the input data\r\n",
|
|
" formats accepted by the webserver, see the following documentation:\r\n",
|
|
" https://www.mlflow.org/docs/latest/models.html#built-in-deployment-tools.\r\n",
|
|
"\r\n",
|
|
" You can make requests to ``POST /invocations`` in pandas split- or record-\r\n",
|
|
" oriented formats.\r\n",
|
|
"\r\n",
|
|
" Example:\r\n",
|
|
"\r\n",
|
|
" .. code-block:: bash\r\n",
|
|
"\r\n",
|
|
" $ mlflow models serve -m runs:/my-run-id/model-path &\r\n",
|
|
"\r\n",
|
|
" $ curl http://127.0.0.1:5000/invocations -H 'Content-Type:\r\n",
|
|
" application/json' -d '{ \"columns\": [\"a\", \"b\", \"c\"],\r\n",
|
|
" \"data\": [[1, 2, 3], [4, 5, 6]] }'\r\n",
|
|
"\r\n",
|
|
"Options:\r\n",
|
|
" -m, --model-uri URI URI to the model. A local path, a 'runs:/' URI, or a\r\n",
|
|
" remote storage URI (e.g., an 's3://' URI). For more\r\n",
|
|
" information about supported remote URIs for model\r\n",
|
|
" artifacts, see\r\n",
|
|
" https://mlflow.org/docs/latest/tracking.html#artifact-\r\n",
|
|
" stores [required]\r\n",
|
|
"\r\n",
|
|
" -p, --port INTEGER The port to listen on (default: 5000).\r\n",
|
|
" -h, --host HOST The network address to listen on (default: 127.0.0.1).\r\n",
|
|
" Use 0.0.0.0 to bind to all addresses if you want to\r\n",
|
|
" access the tracking server from other machines.\r\n",
|
|
"\r\n",
|
|
" -w, --workers TEXT Number of gunicorn worker processes to handle requests\r\n",
|
|
" (default: 4).\r\n",
|
|
"\r\n",
|
|
" --no-conda If specified, will assume that MLmodel/MLproject is\r\n",
|
|
" running within a Conda environment with the necessary\r\n",
|
|
" dependencies for the current project instead of\r\n",
|
|
" attempting to create a new conda environment.\r\n",
|
|
"\r\n",
|
|
" --install-mlflow If specified and there is a conda environment to be\r\n",
|
|
" activated mlflow will be installed into the environment\r\n",
|
|
" after it has been activated. The version of installed\r\n",
|
|
" mlflow will be the same asthe one used to invoke this\r\n",
|
|
" command.\r\n",
|
|
"\r\n",
|
|
" --help Show this message and exit.\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"!cd IUM_08/examples/; mlflow models serve --help"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 54,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{\"columns\":[\"fixed acidity\",\"volatile acidity\",\"citric acid\",\"residual sugar\",\"chlorides\",\"free sulfur dioxide\",\"total sulfur dioxide\",\"density\",\"pH\",\"sulphates\",\"alcohol\"],\"index\":[0],\"data\":[[7.4,0.7,0.0,1.9,0.076,11.0,34.0,0.9978,3.51,0.56,9.4]]}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import pandas as pd\n",
|
|
"csv_url = \"http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv\"\n",
|
|
"data = pd.read_csv(csv_url, sep=\";\").drop([\"quality\"], axis=1).head(1).to_json(orient='split')\n",
|
|
"print(data)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 57,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[5.576883967129615]"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"!curl http://127.0.0.1:5003/invocations -H 'Content-Type: application/json' -d '{\\\n",
|
|
" \"columns\":[\\\n",
|
|
" \"fixed acidity\",\"volatile acidity\",\"citric acid\",\"residual sugar\",\"chlorides\",\"free sulfur dioxide\",\"total sulfur dioxide\",\"density\",\"pH\",\"sulphates\",\"alcohol\"],\\\n",
|
|
" \"index\":[0],\\\n",
|
|
" \"data\":[[7.4,0.7,0.0,1.9,0.076,11.0,34.0,0.9978,3.51,0.56,9.4]]}'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"```\n",
|
|
"$ cd IUM_08/examples/\n",
|
|
"$ mlflow models serve -m my_model\n",
|
|
"2021/05/17 08:52:07 INFO mlflow.models.cli: Selected backend for flavor 'python_function'\n",
|
|
"2021/05/17 08:52:07 INFO mlflow.pyfunc.backend: === Running command 'source /home/tomek/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-503f0c7520a32f054a9d168bd099584a9439de9d 1>&2 && gunicorn --timeout=60 -b 127.0.0.1:5003 -w 1 ${GUNICORN_CMD_ARGS} -- mlflow.pyfunc.scoring_server.wsgi:app'\n",
|
|
"[2021-05-17 08:52:07 +0200] [291217] [INFO] Starting gunicorn 20.1.0\n",
|
|
"[2021-05-17 08:52:07 +0200] [291217] [INFO] Listening at: http://127.0.0.1:5003 (291217)\n",
|
|
"[2021-05-17 08:52:07 +0200] [291217] [INFO] Using worker: sync\n",
|
|
"[2021-05-17 08:52:07 +0200] [291221] [INFO] Booting worker with pid: 291221\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## MLflow Registry\n",
|
|
" - umożliwia [zapisywanie](https://mlflow.org/docs/latest/model-registry.html#adding-an-mlflow-model-to-the-model-registry) i [ładowanie](https://mlflow.org/docs/latest/model-registry.html#fetching-an-mlflow-model-from-the-model-registry) modeli z centralnego rejestru\n",
|
|
" - Modele można też serwować bezpośrednio z rejestru:\n",
|
|
"\n",
|
|
"```bash\n",
|
|
"#!/usr/bin/env sh\n",
|
|
"\n",
|
|
"# Set environment variable for the tracking URL where the Model Registry resides\n",
|
|
"export MLFLOW_TRACKING_URI=http://localhost:5000\n",
|
|
"\n",
|
|
"# Serve the production model from the model registry\n",
|
|
"mlflow models serve -m \"models:/sk-learn-random-forest-reg-model/Production\"\n",
|
|
"```\n",
|
|
"\n",
|
|
"- Żeby było to możliwe, musimy mieć uruchomiony [serwer MLflow](https://mlflow.org/docs/latest/tracking.html#tracking-server)\n",
|
|
"- Umożliwia zarządzanie wersjami modeli i oznaczanie ich różnymi fazami, np. \"Staging\", \"Production\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## Zadania (termin: 14.05 EOD) \n",
|
|
"1. [2 pkt] Dodaj do joba treningowego wywołania MLflow, tak, żeby przy każdym uruchomieniu stworzyć i zarchiwizować katalog z modelem. Plik MLmodel powinien zawierać pola:\n",
|
|
" - signature\n",
|
|
" - input_example\n",
|
|
"\n",
|
|
" Folder powinien również zawierać definicję środowiska - conda lub docker, umożliwiającego uruchomienie projektu.\n",
|
|
"\n",
|
|
"2. [6 pkt] Wybierz jedną osobę z grupy. Załóżmy, że Twoje ID to s123456 a jej s654321. Stwórz na Jenkinsie projekt `s123456-predict-s654321`, w którym:\n",
|
|
" - pobierzesz artefakt z zapisanym modelem z joba osoby s654321\n",
|
|
" - dokonasz na nim predykcji danych wejściowych podanych w formacie json jako parametr zadania Jenkinsowego. Domyślną wartością tego parametru niech będą przykładowe dane wejściowe z `input_example`\n",
|
|
" \n",
|
|
"3. [1 pkt] Zarejestruj swój model w MLflow registry\n",
|
|
"\n",
|
|
"4. [6 pkt] Stwórz na Jenkinsie projekt `s123456-predict-s654321-from-registry`, który zrealizuje to samo zadanie co `s123456-predict-s654321`, ale tym razem pobierze model z rejestru MLflow zamiast z artefaktów Jenkinsa"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"Dane do konfiguracji MLflow registry\n",
|
|
"\n",
|
|
"- Podgląd w przeglądarce: http://tzietkiewicz.vm.wmi.amu.edu.pl/#/\n",
|
|
" - user: `student`\n",
|
|
" - hasło: IUM@2021\n",
|
|
"\n",
|
|
"- Tracking URI:\n",
|
|
" - Python: `mlflow.set_tracking_uri(\"http://172.17.0.1:5000\")`\n",
|
|
" - CLI: `export MLFLOW_TRACKING_URI=http://172.17.0.1:5000`\n",
|
|
" \n",
|
|
"- Żeby klient MLflow działający w kontenerze docker mógł zapisywać i pdczytywać artefakty, muszą Państwo podmonotwać katalog `/tmp/mlruns` i `/mlruns` (ten drugi po to, żeby po restarcie serwera katalog nie został wyczyszczony)\n",
|
|
" - jak podmontować: ```docker.image('my-image').inside('-v /tmp/mlruns:/tmp/mlruns' -v /mlruns:/mlruns')```\n",
|
|
" - Przykład działąjącego joba na Jenkinsie zapisującego wyniki do MLFlow: https://tzietkiewicz.vm.wmi.amu.edu.pl:8081/job/gitea-test/job/ium-helloworld/job/master/\n",
|
|
" - Repo: https://git.wmi.amu.edu.pl/tzietkiewicz/ium-helloworld/src/branch/master\n",
|
|
"\n",
|
|
"- Proszę ustawić nazwę eksperymentu na numer indeksu, dzięki temu każdy z Państwa będzie widział swoje eksperymenty oddzielnie:\n",
|
|
"`mlflow.set_experiment(\"s123456\")`"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"author": "Tomasz Ziętkiewicz",
|
|
"celltoolbar": "Slideshow",
|
|
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"lang": "pl",
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.9.12"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
},
|
|
"subtitle": "8.MLFlow[laboratoria]",
|
|
"title": "Inżynieria uczenia maszynowego",
|
|
"year": "2021"
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|