forked from AITech/aitech-ium
8295 lines
765 KiB
Plaintext
8295 lines
765 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## Inżynieria uczenia maszynowego\n",
|
|
"### 24 kwietnia 2024\n",
|
|
"# 8. MLFlow"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
" ## MLflow\n",
|
|
" </br><img style=\"width: 50%;\" src=\"img/expcontrol/mlflow-logo-d.png\"/>\n",
|
|
"\n",
|
|
" - https://mlflow.org/\n",
|
|
" - Narzędzie podobne do omawianego na poprzednich zajęciach Sacred\n",
|
|
" - Nieco inne podejście: mniej ingerencji w istniejący kod\n",
|
|
" - Bardziej kompleksowe rozwiązanie: 4 komponenty, pierwszy z nich ma funkcjonalność podobną do Sacred\n",
|
|
" - Działa \"z każdym\" językiem. A tak naprawdę: Python, R, Java + CLI API + REST API\n",
|
|
" - Popularna wśród pracodawców - wyniki wyszukiwania ofert pracy: 20 ofert (https://pl.indeed.com/), 36 ofert (linkedin). Sacred: 0\n",
|
|
" - Integracja z licznymi bibliotekami / chmurami\n",
|
|
" - Rozwiązanie OpenSource, stworzone przez firmę Databricks\n",
|
|
" - Dostępne różne wydania / opcje instalacji:\n",
|
|
" - płatne:\n",
|
|
" - Databricks Customers\n",
|
|
" - bezpłatne:\n",
|
|
" - Databricks Community Edition\n",
|
|
" - Self-managed MLflow\n",
|
|
" - Local Tracking Server\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## Komponenty\n",
|
|
"\n",
|
|
"MLflow składa się z czterech niezależnych komponentów:\n",
|
|
" - **MLflow Tracking** - pozwala śledzić zmiany parametrów, kodu, środowiska i ich wpływ na metryki. Jest to funkcjonalność bardzo zbliżona do tej, którą zapewnia Sacred\n",
|
|
" - **MLflow Projects** - umożliwia \"pakowanie\" kodu ekserymentów w taki sposób, żeby mogłby być w łatwy sposób zreprodukowane przez innych\n",
|
|
" - **MLflow Models** - ułatwia \"pakowanie\" modeli uczenia maszynowego\n",
|
|
" - **MLflow Registry** - zapewnia centralne miejsce do przechowywania i współdzielenia modeli. Zapewnia narzędzia do wersjonowania i śledzenia pochodzenia tych modeli.\n",
|
|
" \n",
|
|
"Komponenty te mogą być używane razem bądź oddzielnie."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## MLflow Tracking - przykład\n",
|
|
"(Poniższe przykłady kodu trenującego pochodzą z tutoriala MLflow)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%capture null\n",
|
|
"!pip install mlflow\n",
|
|
"!pip install sklearn"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!mkdir -p IUM_08/examples/sklearn_elasticnet_wine/"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Overwriting IUM_08/examples/sklearn_elasticnet_wine/train.py\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%writefile IUM_08/examples/sklearn_elasticnet_wine/train.py\n",
|
|
"# The data set used in this example is from http://archive.ics.uci.edu/ml/datasets/Wine+Quality\n",
|
|
"# P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis.\n",
|
|
"# Modeling wine preferences by data mining from physicochemical properties. In Decision Support Systems, Elsevier, 47(4):547-553, 2009.\n",
|
|
"\n",
|
|
"import os\n",
|
|
"import warnings\n",
|
|
"import sys\n",
|
|
"\n",
|
|
"import pandas as pd\n",
|
|
"import numpy as np\n",
|
|
"from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score\n",
|
|
"from sklearn.model_selection import train_test_split\n",
|
|
"from sklearn.linear_model import ElasticNet\n",
|
|
"from urllib.parse import urlparse\n",
|
|
"import mlflow\n",
|
|
"import mlflow.sklearn\n",
|
|
"\n",
|
|
"import logging\n",
|
|
"\n",
|
|
"logging.basicConfig(level=logging.WARN)\n",
|
|
"logger = logging.getLogger(__name__)\n",
|
|
"\n",
|
|
"mlflow.set_tracking_uri(\"http://localhost:5000\")\n",
|
|
"mlflow.set_experiment(\"s123456\")\n",
|
|
"\n",
|
|
"def eval_metrics(actual, pred):\n",
|
|
" rmse = np.sqrt(mean_squared_error(actual, pred))\n",
|
|
" mae = mean_absolute_error(actual, pred)\n",
|
|
" r2 = r2_score(actual, pred)\n",
|
|
" return rmse, mae, r2\n",
|
|
"\n",
|
|
"\n",
|
|
"if __name__ == \"__main__\":\n",
|
|
" warnings.filterwarnings(\"ignore\")\n",
|
|
" np.random.seed(40)\n",
|
|
"\n",
|
|
" # Read the wine-quality csv file from the URL\n",
|
|
" csv_url = (\n",
|
|
" \"http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv\"\n",
|
|
" )\n",
|
|
" try:\n",
|
|
" data = pd.read_csv(csv_url, sep=\";\")\n",
|
|
" except Exception as e:\n",
|
|
" logger.exception(\n",
|
|
" \"Unable to download training & test CSV, check your internet connection. Error: %s\", e\n",
|
|
" )\n",
|
|
"\n",
|
|
" # Split the data into training and test sets. (0.75, 0.25) split.\n",
|
|
" train, test = train_test_split(data)\n",
|
|
"\n",
|
|
" # The predicted column is \"quality\" which is a scalar from [3, 9]\n",
|
|
" train_x = train.drop([\"quality\"], axis=1)\n",
|
|
" test_x = test.drop([\"quality\"], axis=1)\n",
|
|
" train_y = train[[\"quality\"]]\n",
|
|
" test_y = test[[\"quality\"]]\n",
|
|
"\n",
|
|
" \n",
|
|
" alpha = float(sys.argv[1]) if len(sys.argv) > 1 else 0.5\n",
|
|
" #alpha = 0.5\n",
|
|
" l1_ratio = float(sys.argv[2]) if len(sys.argv) > 2 else 0.5\n",
|
|
" #l1_ratio = 0.5\n",
|
|
"\n",
|
|
" with mlflow.start_run() as run:\n",
|
|
" print(\"MLflow run experiment_id: {0}\".format(run.info.experiment_id))\n",
|
|
" print(\"MLflow run artifact_uri: {0}\".format(run.info.artifact_uri))\n",
|
|
"\n",
|
|
" lr = ElasticNet(alpha=alpha, l1_ratio=l1_ratio, random_state=42)\n",
|
|
" lr.fit(train_x, train_y)\n",
|
|
"\n",
|
|
" predicted_qualities = lr.predict(test_x)\n",
|
|
"\n",
|
|
" (rmse, mae, r2) = eval_metrics(test_y, predicted_qualities)\n",
|
|
"\n",
|
|
" print(\"Elasticnet model (alpha=%f, l1_ratio=%f):\" % (alpha, l1_ratio))\n",
|
|
" print(\" RMSE: %s\" % rmse)\n",
|
|
" print(\" MAE: %s\" % mae)\n",
|
|
" print(\" R2: %s\" % r2)\n",
|
|
"\n",
|
|
" mlflow.log_param(\"alpha\", alpha)\n",
|
|
" mlflow.log_param(\"l1_ratio\", l1_ratio)\n",
|
|
" mlflow.log_metric(\"rmse\", rmse)\n",
|
|
" mlflow.log_metric(\"r2\", r2)\n",
|
|
" mlflow.log_metric(\"mae\", mae)\n",
|
|
" \n",
|
|
" # Infer model signature to log it\n",
|
|
" # Więcej o sygnaturach: https://mlflow.org/docs/latest/models.html?highlight=signature#model-signature\n",
|
|
" signature = mlflow.models.signature.infer_signature(train_x, lr.predict(train_x))\n",
|
|
"\n",
|
|
" tracking_url_type_store = urlparse(mlflow.get_tracking_uri()).scheme\n",
|
|
"\n",
|
|
" # Model registry does not work with file store\n",
|
|
" if tracking_url_type_store != \"file\":\n",
|
|
"\n",
|
|
" # Register the model\n",
|
|
" # There are other ways to use the Model Registry, which depends on the use case,\n",
|
|
" # please refer to the doc for more information:\n",
|
|
" # https://mlflow.org/docs/latest/model-registry.html#api-workflow\n",
|
|
" mlflow.sklearn.log_model(lr, \"wines-model\", registered_model_name=\"ElasticnetWineModel\", signature=signature)\n",
|
|
" else:\n",
|
|
" mlflow.sklearn.log_model(lr, \"model\", signature=signature)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"ls: cannot access '/tmp/mlruns': No such file or directory\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15f130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15f580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15f730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15f8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15fa90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f09fa15fc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"! ls -l /tmp/mlruns\n",
|
|
"### Wtyrenujmy model z domyślnymi wartościami parametrów\n",
|
|
"! cd ./IUM_08/examples/; python sklearn_elasticnet_wine/train.py"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e5138e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513a90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f447e513c40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0db130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0db580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0db730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0db8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0dba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f000f0dbc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0dbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0dbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0dbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e478e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47a90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f55f8e47c40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2f130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2f580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2f730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2f8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2fa90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa1f5f2fc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcb130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcb580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcb730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcb8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7d4fbcbc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5b130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5b580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5b730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5b8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5ba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fb7f3a5bc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976b130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976b580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976b730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976b8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976ba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa4b976bc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9b130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9b580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9b730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9b8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9ba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f531cc9bc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7f130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7f580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7f730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7f8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7fa90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f911ff7fc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb78e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7a90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff9f0cb7c40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7f160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7187f7fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e75b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f13007e7c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df675b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbe1df67c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c35b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f715f7c3c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607f160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe00607fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a375b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f95a1a37c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc075b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fab0dc07c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c475b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f2440c47c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce5435e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbdce543ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3b190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3b5e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3b790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3b940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3baf0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7efdebe3bca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3bca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3bca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3bca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd8975e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbafd897ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae35e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f28a6ae3ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc7435e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa4cc743ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae75e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fb408ae7ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7eb160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7eb5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7eb760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7eb910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7ebac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f608c7ebc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7ebc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7ebc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7ebc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e191075b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6e19107c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14b160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14b5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14b760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14b910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14bac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe7df14bc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d935b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fcda6d93c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a775b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3e90a77c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece575b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f9fece57c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5f160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7d77d5fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|