forked from AITech/aitech-ium
8295 lines
765 KiB
Plaintext
8295 lines
765 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## Inżynieria uczenia maszynowego\n",
|
|
"### 24 kwietnia 2024\n",
|
|
"# 8. MLFlow"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
" ## MLflow\n",
|
|
" </br><img style=\"width: 50%;\" src=\"img/expcontrol/mlflow-logo-d.png\"/>\n",
|
|
"\n",
|
|
" - https://mlflow.org/\n",
|
|
" - Narzędzie podobne do omawianego na poprzednich zajęciach Sacred\n",
|
|
" - Nieco inne podejście: mniej ingerencji w istniejący kod\n",
|
|
" - Bardziej kompleksowe rozwiązanie: 4 komponenty, pierwszy z nich ma funkcjonalność podobną do Sacred\n",
|
|
" - Działa \"z każdym\" językiem. A tak naprawdę: Python, R, Java + CLI API + REST API\n",
|
|
" - Popularna wśród pracodawców - wyniki wyszukiwania ofert pracy: 20 ofert (https://pl.indeed.com/), 36 ofert (linkedin). Sacred: 0\n",
|
|
" - Integracja z licznymi bibliotekami / chmurami\n",
|
|
" - Rozwiązanie OpenSource, stworzone przez firmę Databricks\n",
|
|
" - Dostępne różne wydania / opcje instalacji:\n",
|
|
" - płatne:\n",
|
|
" - Databricks Customers\n",
|
|
" - bezpłatne:\n",
|
|
" - Databricks Community Edition\n",
|
|
" - Self-managed MLflow\n",
|
|
" - Local Tracking Server\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## Komponenty\n",
|
|
"\n",
|
|
"MLflow składa się z czterech niezależnych komponentów:\n",
|
|
" - **MLflow Tracking** - pozwala śledzić zmiany parametrów, kodu, środowiska i ich wpływ na metryki. Jest to funkcjonalność bardzo zbliżona do tej, którą zapewnia Sacred\n",
|
|
" - **MLflow Projects** - umożliwia \"pakowanie\" kodu ekserymentów w taki sposób, żeby mogłby być w łatwy sposób zreprodukowane przez innych\n",
|
|
" - **MLflow Models** - ułatwia \"pakowanie\" modeli uczenia maszynowego\n",
|
|
" - **MLflow Registry** - zapewnia centralne miejsce do przechowywania i współdzielenia modeli. Zapewnia narzędzia do wersjonowania i śledzenia pochodzenia tych modeli.\n",
|
|
" \n",
|
|
"Komponenty te mogą być używane razem bądź oddzielnie."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## MLflow Tracking - przykład\n",
|
|
"(Poniższe przykłady kodu trenującego pochodzą z tutoriala MLflow)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%capture null\n",
|
|
"!pip install mlflow\n",
|
|
"!pip install sklearn"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"!mkdir -p IUM_08/examples/sklearn_elasticnet_wine/"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Overwriting IUM_08/examples/sklearn_elasticnet_wine/train.py\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%writefile IUM_08/examples/sklearn_elasticnet_wine/train.py\n",
|
|
"# The data set used in this example is from http://archive.ics.uci.edu/ml/datasets/Wine+Quality\n",
|
|
"# P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis.\n",
|
|
"# Modeling wine preferences by data mining from physicochemical properties. In Decision Support Systems, Elsevier, 47(4):547-553, 2009.\n",
|
|
"\n",
|
|
"import os\n",
|
|
"import warnings\n",
|
|
"import sys\n",
|
|
"\n",
|
|
"import pandas as pd\n",
|
|
"import numpy as np\n",
|
|
"from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score\n",
|
|
"from sklearn.model_selection import train_test_split\n",
|
|
"from sklearn.linear_model import ElasticNet\n",
|
|
"from urllib.parse import urlparse\n",
|
|
"import mlflow\n",
|
|
"import mlflow.sklearn\n",
|
|
"\n",
|
|
"import logging\n",
|
|
"\n",
|
|
"logging.basicConfig(level=logging.WARN)\n",
|
|
"logger = logging.getLogger(__name__)\n",
|
|
"\n",
|
|
"mlflow.set_tracking_uri(\"http://localhost:5000\")\n",
|
|
"mlflow.set_experiment(\"s123456\")\n",
|
|
"\n",
|
|
"def eval_metrics(actual, pred):\n",
|
|
" rmse = np.sqrt(mean_squared_error(actual, pred))\n",
|
|
" mae = mean_absolute_error(actual, pred)\n",
|
|
" r2 = r2_score(actual, pred)\n",
|
|
" return rmse, mae, r2\n",
|
|
"\n",
|
|
"\n",
|
|
"if __name__ == \"__main__\":\n",
|
|
" warnings.filterwarnings(\"ignore\")\n",
|
|
" np.random.seed(40)\n",
|
|
"\n",
|
|
" # Read the wine-quality csv file from the URL\n",
|
|
" csv_url = (\n",
|
|
" \"http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv\"\n",
|
|
" )\n",
|
|
" try:\n",
|
|
" data = pd.read_csv(csv_url, sep=\";\")\n",
|
|
" except Exception as e:\n",
|
|
" logger.exception(\n",
|
|
" \"Unable to download training & test CSV, check your internet connection. Error: %s\", e\n",
|
|
" )\n",
|
|
"\n",
|
|
" # Split the data into training and test sets. (0.75, 0.25) split.\n",
|
|
" train, test = train_test_split(data)\n",
|
|
"\n",
|
|
" # The predicted column is \"quality\" which is a scalar from [3, 9]\n",
|
|
" train_x = train.drop([\"quality\"], axis=1)\n",
|
|
" test_x = test.drop([\"quality\"], axis=1)\n",
|
|
" train_y = train[[\"quality\"]]\n",
|
|
" test_y = test[[\"quality\"]]\n",
|
|
"\n",
|
|
" \n",
|
|
" alpha = float(sys.argv[1]) if len(sys.argv) > 1 else 0.5\n",
|
|
" #alpha = 0.5\n",
|
|
" l1_ratio = float(sys.argv[2]) if len(sys.argv) > 2 else 0.5\n",
|
|
" #l1_ratio = 0.5\n",
|
|
"\n",
|
|
" with mlflow.start_run() as run:\n",
|
|
" print(\"MLflow run experiment_id: {0}\".format(run.info.experiment_id))\n",
|
|
" print(\"MLflow run artifact_uri: {0}\".format(run.info.artifact_uri))\n",
|
|
"\n",
|
|
" lr = ElasticNet(alpha=alpha, l1_ratio=l1_ratio, random_state=42)\n",
|
|
" lr.fit(train_x, train_y)\n",
|
|
"\n",
|
|
" predicted_qualities = lr.predict(test_x)\n",
|
|
"\n",
|
|
" (rmse, mae, r2) = eval_metrics(test_y, predicted_qualities)\n",
|
|
"\n",
|
|
" print(\"Elasticnet model (alpha=%f, l1_ratio=%f):\" % (alpha, l1_ratio))\n",
|
|
" print(\" RMSE: %s\" % rmse)\n",
|
|
" print(\" MAE: %s\" % mae)\n",
|
|
" print(\" R2: %s\" % r2)\n",
|
|
"\n",
|
|
" mlflow.log_param(\"alpha\", alpha)\n",
|
|
" mlflow.log_param(\"l1_ratio\", l1_ratio)\n",
|
|
" mlflow.log_metric(\"rmse\", rmse)\n",
|
|
" mlflow.log_metric(\"r2\", r2)\n",
|
|
" mlflow.log_metric(\"mae\", mae)\n",
|
|
" \n",
|
|
" # Infer model signature to log it\n",
|
|
" # Więcej o sygnaturach: https://mlflow.org/docs/latest/models.html?highlight=signature#model-signature\n",
|
|
" signature = mlflow.models.signature.infer_signature(train_x, lr.predict(train_x))\n",
|
|
"\n",
|
|
" tracking_url_type_store = urlparse(mlflow.get_tracking_uri()).scheme\n",
|
|
"\n",
|
|
" # Model registry does not work with file store\n",
|
|
" if tracking_url_type_store != \"file\":\n",
|
|
"\n",
|
|
" # Register the model\n",
|
|
" # There are other ways to use the Model Registry, which depends on the use case,\n",
|
|
" # please refer to the doc for more information:\n",
|
|
" # https://mlflow.org/docs/latest/model-registry.html#api-workflow\n",
|
|
" mlflow.sklearn.log_model(lr, \"wines-model\", registered_model_name=\"ElasticnetWineModel\", signature=signature)\n",
|
|
" else:\n",
|
|
" mlflow.sklearn.log_model(lr, \"model\", signature=signature)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"ls: cannot access '/tmp/mlruns': No such file or directory\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15f130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15f580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15f730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15f8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15fa90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f09fa15fc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f09fa15fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"! ls -l /tmp/mlruns\n",
|
|
"### Wtyrenujmy model z domyślnymi wartościami parametrów\n",
|
|
"! cd ./IUM_08/examples/; python sklearn_elasticnet_wine/train.py"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e5138e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513a90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f447e513c40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f447e513c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0db130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0db580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0db730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0db8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0dba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f000f0dbc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0dbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0dbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f000f0dbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e478e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47a90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f55f8e47c40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f55f8e47c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2f130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2f580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2f730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2f8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2fa90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa1f5f2fc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa1f5f2fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcb130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcb580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcb730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcb8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7d4fbcbc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d4fbcbc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5b130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5b580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5b730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5b8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5ba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fb7f3a5bc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb7f3a5bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976b130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976b580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976b730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976b8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976ba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa4b976bc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4b976bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9b130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9b580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9b730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9b8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9ba90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f531cc9bc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f531cc9bc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7f130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7f580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7f730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7f8e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7fa90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f911ff7fc40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f911ff7fc40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb78e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7a90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff9f0cb7c40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff9f0cb7c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7f160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7187f7fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7187f7fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e75b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f13007e7c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f13007e7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df675b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbe1df67c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe1df67c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c35b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f715f7c3c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f715f7c3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607f160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe00607fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe00607fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a375b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f95a1a37c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f95a1a37c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc075b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fab0dc07c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fab0dc07c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c475b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f2440c47c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2440c47c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce5435e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbdce543ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbdce543ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3b190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3b5e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3b790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3b940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3baf0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7efdebe3bca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3bca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3bca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efdebe3bca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd8975e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbafd897ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbafd897ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae35e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f28a6ae3ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f28a6ae3ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc7435e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa4cc743ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa4cc743ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae75e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fb408ae7ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb408ae7ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7eb160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7eb5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7eb760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7eb910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7ebac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f608c7ebc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7ebc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7ebc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f608c7ebc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e191075b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6e19107c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6e19107c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14b160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14b5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14b760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14b910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14bac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe7df14bc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7df14bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d935b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fcda6d93c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcda6d93c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a775b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3e90a77c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e90a77c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece575b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f9fece57c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9fece57c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5f160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7d77d5fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7d77d5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc69efc31c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc69efc35b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc69efc3760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc69efc3910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc69efc3ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fc69efc3c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc69efc3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc69efc3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc69efc3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f69c318b160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f69c318b5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f69c318b760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f69c318b910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f69c318bac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f69c318bc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f69c318bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f69c318bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f69c318bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f53ea4ef160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f53ea4ef5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f53ea4ef760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f53ea4ef910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f53ea4efac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f53ea4efc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f53ea4efc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f53ea4efc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f53ea4efc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7dd5caf160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7dd5caf5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7dd5caf760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7dd5caf910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7dd5cafac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f7dd5cafc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7dd5cafc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7dd5cafc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7dd5cafc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feba9c6f160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feba9c6f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feba9c6f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feba9c6f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feba9c6fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7feba9c6fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feba9c6fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feba9c6fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feba9c6fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd5940b3160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd5940b35b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd5940b3760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd5940b3910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd5940b3ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fd5940b3c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd5940b3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd5940b3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd5940b3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0319eb7160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0319eb75b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0319eb7760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0319eb7910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0319eb7ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0319eb7c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0319eb7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0319eb7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0319eb7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbbfb6b7160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbbfb6b75b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbbfb6b7760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbbfb6b7910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbbfb6b7ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbbfb6b7c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbbfb6b7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbbfb6b7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbbfb6b7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f368f86b160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f368f86b5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f368f86b760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f368f86b910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f368f86bac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f368f86bc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f368f86bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f368f86bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f368f86bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feb973d7160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feb973d75b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feb973d7760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feb973d7910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feb973d7ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7feb973d7c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feb973d7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feb973d7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feb973d7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa56c6cf190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa56c6cf5e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa56c6cf790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa56c6cf940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa56c6cfaf0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa56c6cfca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa56c6cfca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa56c6cfca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa56c6cfca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f406d6e3220>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f406d6e3610>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f406d6e37c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f406d6e3970>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f406d6e3b20>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f406d6e3cd0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f406d6e3cd0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f406d6e3cd0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f406d6e3cd0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d8a4231f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d8a4235e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d8a423790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d8a423940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d8a423af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f2d8a423ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d8a423ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d8a423ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2d8a423ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1d664e31f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1d664e35e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1d664e3790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1d664e3940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1d664e3af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f1d664e3ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1d664e3ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1d664e3ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1d664e3ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feaf01ef190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feaf01ef5e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feaf01ef790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feaf01ef940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feaf01efaf0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7feaf01efca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feaf01efca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feaf01efca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7feaf01efca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4119c9b190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4119c9b5e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4119c9b790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4119c9b940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4119c9baf0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f4119c9bca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4119c9bca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4119c9bca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4119c9bca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff7318ef190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff7318ef5e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff7318ef790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff7318ef940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff7318efaf0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7ff7318efca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff7318efca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff7318efca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff7318efca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc1bf2c7190>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc1bf2c75e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc1bf2c7790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc1bf2c7940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc1bf2c7af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fc1bf2c7ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc1bf2c7ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc1bf2c7ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc1bf2c7ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f5cc531f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f5cc535e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f5cc53790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f5cc53940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f5cc53af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f8f5cc53ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f5cc53ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f5cc53ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8f5cc53ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcbd30671f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcbd30675e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcbd3067790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcbd3067940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcbd3067af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fcbd3067ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcbd3067ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcbd3067ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fcbd3067ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d5ba5f1f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d5ba5f5e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d5ba5f790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d5ba5f940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d5ba5faf0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f8d5ba5fca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d5ba5fca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d5ba5fca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8d5ba5fca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f34c83071f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f34c83075e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f34c8307790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f34c8307940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f34c8307af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f34c8307ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f34c8307ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f34c8307ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f34c8307ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc9323871f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc9323875e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc932387790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc932387940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc932387af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fc932387ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc932387ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc932387ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc932387ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5fce1971f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5fce1975e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5fce197790>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5fce197940>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5fce197af0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f5fce197ca0>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5fce197ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5fce197ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5fce197ca0>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f56d07571c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f56d07575b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f56d0757760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f56d0757910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f56d0757ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f56d0757c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f56d0757c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f56d0757c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f56d0757c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4a95ebb1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4a95ebb5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4a95ebb760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4a95ebb910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4a95ebbac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f4a95ebbc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4a95ebbc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4a95ebbc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4a95ebbc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0816b5f1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0816b5f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0816b5f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0816b5f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0816b5fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0816b5fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0816b5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0816b5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0816b5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c76f471c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c76f475b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c76f47760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c76f47910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c76f47ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3c76f47c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c76f47c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c76f47c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3c76f47c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdc6ac4b1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdc6ac4b5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdc6ac4b760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdc6ac4b910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdc6ac4bac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fdc6ac4bc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdc6ac4bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdc6ac4bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fdc6ac4bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f23c5f9b1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f23c5f9b5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f23c5f9b760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f23c5f9b910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f23c5f9bac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f23c5f9bc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f23c5f9bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f23c5f9bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f23c5f9bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f94f59cb1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f94f59cb5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f94f59cb760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f94f59cb910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f94f59cbac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f94f59cbc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f94f59cbc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f94f59cbc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f94f59cbc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f078e8cb1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f078e8cb5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f078e8cb760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f078e8cb910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f078e8cbac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f078e8cbc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f078e8cbc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f078e8cbc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f078e8cbc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e1ebdf1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e1ebdf5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e1ebdf760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e1ebdf910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e1ebdfac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0e1ebdfc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e1ebdfc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e1ebdfc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0e1ebdfc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eff284eb1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eff284eb5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eff284eb760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eff284eb910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eff284ebac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7eff284ebc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eff284ebc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eff284ebc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eff284ebc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe14b2e31c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe14b2e35b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe14b2e3760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe14b2e3910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe14b2e3ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe14b2e3c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe14b2e3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe14b2e3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe14b2e3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0bae5171c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0bae5175b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0bae517760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0bae517910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0bae517ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0bae517c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0bae517c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0bae517c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0bae517c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f528799b160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f528799b5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f528799b760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f528799b910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f528799bac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f528799bc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f528799bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f528799bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f528799bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe544d1b160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe544d1b5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe544d1b760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe544d1b910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe544d1bac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe544d1bc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe544d1bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe544d1bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe544d1bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d27fe7160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d27fe75b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d27fe7760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d27fe7910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d27fe7ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3d27fe7c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d27fe7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d27fe7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3d27fe7c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb680787160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb6807875b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb680787760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb680787910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb680787ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fb680787c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb680787c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb680787c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fb680787c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa923b3b160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa923b3b5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa923b3b760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa923b3b910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa923b3bac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa923b3bc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa923b3bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa923b3bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa923b3bc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe4f9da3160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe4f9da35b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe4f9da3760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe4f9da3910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe4f9da3ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe4f9da3c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe4f9da3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe4f9da3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe4f9da3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3b4ec4f160>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3b4ec4f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3b4ec4f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3b4ec4f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3b4ec4fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3b4ec4fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3b4ec4fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3b4ec4fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3b4ec4fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe2e7797130>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe2e7797580>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe2e7797730>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe2e77978e0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe2e7797a90>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe2e7797c40>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe2e7797c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe2e7797c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe2e7797c40>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870117f1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870117f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870117f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870117f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870117fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f870117fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870117fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870117fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f870117fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f65437571c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f65437575b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6543757760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6543757910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6543757ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6543757c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6543757c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6543757c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6543757c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0984acf1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0984acf5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0984acf760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0984acf910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0984acfac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0984acfc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0984acfc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0984acfc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f0984acfc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f08d3b5f1c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f08d3b5f5b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f08d3b5f760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f08d3b5f910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f08d3b5fac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f08d3b5fc70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f08d3b5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f08d3b5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f08d3b5fc70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e1c5871c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e1c5875b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e1c587760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e1c587910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e1c587ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f5e1c587c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e1c587c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e1c587c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5e1c587c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=4, connect=4, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f59560e31c0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=3, connect=3, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f59560e35b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=2, connect=2, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f59560e3760>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=1, connect=1, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f59560e3910>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=0, read=5, redirect=5, status=5)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f59560e3ac0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 198, in _new_conn\n",
|
|
" sock = connection.create_connection(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 85, in create_connection\n",
|
|
" raise err\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/connection.py\", line 73, in create_connection\n",
|
|
" sock.connect(sa)\n",
|
|
"ConnectionRefusedError: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 793, in urlopen\n",
|
|
" response = self._make_request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 496, in _make_request\n",
|
|
" conn.request(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 400, in request\n",
|
|
" self.endheaders()\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1278, in endheaders\n",
|
|
" self._send_output(message_body, encode_chunked=encode_chunked)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 1038, in _send_output\n",
|
|
" self.send(msg)\n",
|
|
" File \"/usr/lib/python3.10/http/client.py\", line 976, in send\n",
|
|
" self.connect()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 238, in connect\n",
|
|
" self.sock = self._new_conn()\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connection.py\", line 213, in _new_conn\n",
|
|
" raise NewConnectionError(\n",
|
|
"urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f59560e3c70>: Failed to establish a new connection: [Errno 111] Connection refused\n",
|
|
"\n",
|
|
"The above exception was the direct cause of the following exception:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n",
|
|
" resp = conn.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 877, in urlopen\n",
|
|
" return self.urlopen(\n",
|
|
" [Previous line repeated 2 more times]\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 847, in urlopen\n",
|
|
" retries = retries.increment(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/urllib3/util/retry.py\", line 515, in increment\n",
|
|
" raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]\n",
|
|
"urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f59560e3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 128, in http_request\n",
|
|
" return _get_http_response_with_retries(\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/request_utils.py\", line 228, in _get_http_response_with_retries\n",
|
|
" return session.request(method, url, allow_redirects=allow_redirects, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n",
|
|
" resp = self.send(prep, **send_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n",
|
|
" r = adapter.send(request, **kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/requests/adapters.py\", line 519, in send\n",
|
|
" raise ConnectionError(e, request=request)\n",
|
|
"requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f59560e3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n",
|
|
"\n",
|
|
"During handling of the above exception, another exception occurred:\n",
|
|
"\n",
|
|
"Traceback (most recent call last):\n",
|
|
" File \"/home/pawel/ium/IUM_08/examples/sklearn_elasticnet_wine/train.py\", line 24, in <module>\n",
|
|
" mlflow.set_experiment(\"s123456\")\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/fluent.py\", line 143, in set_experiment\n",
|
|
" experiment = client.get_experiment_by_name(experiment_name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/client.py\", line 544, in get_experiment_by_name\n",
|
|
" return self._tracking_client.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/tracking/_tracking_service/client.py\", line 236, in get_experiment_by_name\n",
|
|
" return self.store.get_experiment_by_name(name)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 323, in get_experiment_by_name\n",
|
|
" response_proto = self._call_endpoint(GetExperimentByName, req_body)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/store/tracking/rest_store.py\", line 60, in _call_endpoint\n",
|
|
" return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 286, in call_endpoint\n",
|
|
" response = http_request(**call_kwargs)\n",
|
|
" File \"/home/pawel/ium/venv/lib/python3.10/site-packages/mlflow/utils/rest_utils.py\", line 151, in http_request\n",
|
|
" raise MlflowException(f\"API request to {url} failed with exception {e}\")\n",
|
|
"mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/experiments/get-by-name failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=s123456 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f59560e3c70>: Failed to establish a new connection: [Errno 111] Connection refused'))\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"### I jeszcze raz, tym razem ze zmienionymi wartościami parametrów\n",
|
|
"! cd ./IUM_08/examples/; for l in {1..9}; do for a in {1..9}; do python sklearn_elasticnet_wine/train.py 0.$a 0.$l; done; done"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"ls: cannot access 'IUM_08/examples/mlruns/0': No such file or directory\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"### Informacje o przebieagach eksperymentu zostały zapisane w katalogu mlruns\n",
|
|
"! ls -l IUM_08/examples/mlruns/0 | head"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 33,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"total 20\r\n",
|
|
"drwxrwxr-x 3 tomek tomek 4096 maj 17 08:43 artifacts\r\n",
|
|
"-rw-rw-r-- 1 tomek tomek 423 maj 17 08:43 meta.yaml\r\n",
|
|
"drwxrwxr-x 2 tomek tomek 4096 maj 17 08:43 metrics\r\n",
|
|
"drwxrwxr-x 2 tomek tomek 4096 maj 17 08:43 params\r\n",
|
|
"drwxrwxr-x 2 tomek tomek 4096 maj 17 08:43 tags\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"! ls -l IUM_08/examples/mlruns/0/375cde31bdd44a45a91fd7cee92ebcda"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[2021-05-16 17:58:43 +0200] [118029] [INFO] Starting gunicorn 20.1.0\n",
|
|
"[2021-05-16 17:58:43 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:43 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:44 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:44 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:45 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:45 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:46 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:46 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:47 +0200] [118029] [ERROR] Connection in use: ('127.0.0.1', 5000)\n",
|
|
"[2021-05-16 17:58:47 +0200] [118029] [ERROR] Retrying in 1 second.\n",
|
|
"[2021-05-16 17:58:48 +0200] [118029] [ERROR] Can't connect to ('127.0.0.1', 5000)\n",
|
|
"Running the mlflow server failed. Please see the logs above for details.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"### Możemy je obejrzeć w przeglądarce uruchamiając interfejs webowy:\n",
|
|
"### (powinniśmy to wywołać w normalnej konsoli, w jupyter będziemy mieli zablokowany kernel)\n",
|
|
"! cd IUM_08/examples/; mlflow ui"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Wygląd interfejsu webowego\n",
|
|
"<img width=\"75%\" src=\"IUM_08/mlflowui.png\"/>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Porównywanie wyników\n",
|
|
"<img width=\"75%\" src=\"IUM_08/compare-metrics.png\"/>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## Logowanie\n",
|
|
" - logowania metryk i parametrów można dokonać m.in. poprzez wywołania Python-owego API: `mlflow.log_param()` i `mlflow.log_metric()`. Więcej dostępnych funkcji: [link](https://mlflow.org/docs/latest/tracking.html#logging-functions)\n",
|
|
" - wywołania te muszą nastąpić po wykonaniu [`mlflow.start_run()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.start_run), najlepiej wewnątrz bloku:\n",
|
|
"```python\n",
|
|
" with mlflow.start_run():\n",
|
|
" \n",
|
|
" #[...]\n",
|
|
"\n",
|
|
" mlflow.log_param(\"alpha\", alpha)\n",
|
|
" mlflow.log_param(\"l1_ratio\", l1_ratio)\n",
|
|
"```\n",
|
|
" - jest też możliwość automatycznego logwania dla wybranych bibliotek: https://mlflow.org/docs/latest/tracking.html#automatic-logging"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"# MLflow Projects\n",
|
|
" - MLflow projects to zestaw konwencji i kilku narzędzi\n",
|
|
" - ułatwiają one uruchamianie eskperymentów"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Konfiguracja projektu\n",
|
|
" - W pliku `MLproject` zapisuje się konfigurację projektu ([specyfikacja](https://mlflow.org/docs/latest/projects.html))\n",
|
|
" - Zawiera ona:\n",
|
|
" - odnośnik do środowiska, w którym ma być wywołany eksperyment [szczegóły](https://mlflow.org/docs/latest/projects.html#specifying-an-environment):\n",
|
|
" - nazwa obrazu Docker\n",
|
|
" - albo ścieżka do pliku conda.yaml definiującego środowisko wykonania Conda\n",
|
|
" - parametry, z którymi można wywołać eksperyment\n",
|
|
" - polecenia służące do wywołania eksperymentu"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Overwriting IUM_08/examples/sklearn_elasticnet_wine/MLproject\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%writefile IUM_08/examples/sklearn_elasticnet_wine/MLproject\n",
|
|
"name: tutorial\n",
|
|
"\n",
|
|
"conda_env: conda.yaml #ścieżka do pliku conda.yaml z definicją środowiska\n",
|
|
" \n",
|
|
"#docker_env:\n",
|
|
"# image: mlflow-docker-example-environment\n",
|
|
"\n",
|
|
"entry_points:\n",
|
|
" main:\n",
|
|
" parameters:\n",
|
|
" alpha: {type: float, default: 0.5}\n",
|
|
" l1_ratio: {type: float, default: 0.1}\n",
|
|
" command: \"python train.py {alpha} {l1_ratio}\"\n",
|
|
" test:\n",
|
|
" parameters:\n",
|
|
" alpha: {type: cutoff, default: 0}\n",
|
|
" command: \"python test.py {cutoff}\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Środowisko Conda\n",
|
|
" </br><img style=\"height: 50px;\" src=\"img/environments/conda.png\"/>\n",
|
|
" - https://docs.conda.io\n",
|
|
" - Składnia plików conda.yaml definiujących środowisko: https://docs.conda.io/projects/conda/en/4.6.1/user-guide/tasks/manage-environments.html#create-env-file-manually\n",
|
|
" - Składnia YAML: [przystępnie](https://learnxinyminutes.com/docs/yaml/), [oficjalnie](https://yaml.org/spec/1.2/spec.html)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Overwriting IUM_08/examples/sklearn_elasticnet_wine/conda.yaml\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%writefile IUM_08/examples/sklearn_elasticnet_wine/conda.yaml\n",
|
|
"name: tutorial\n",
|
|
"channels:\n",
|
|
" - defaults\n",
|
|
"dependencies:\n",
|
|
" - python=3.6 #Te zależności będą zainstalowane za pomocą conda isntall\n",
|
|
" - pip\n",
|
|
" - pip: #Te ząś za pomocą pip install\n",
|
|
" - scikit-learn==0.23.2\n",
|
|
" - mlflow>=1.0"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Środowisko docker\n",
|
|
"- zamiast środowiska Conda możemy również podać nazwę obrazu docker, w którym ma być wywołany eksperyment.\n",
|
|
"- obraz będzie szukany lokalnie a następnie na DockerHub, lub w innym repozytorium dockera\n",
|
|
"- składnia specyfikacji ścieżki jest taka sama jak w przypadki poleceń dockera, np. docker pull [link](https://docs.docker.com/engine/reference/commandline/pull/#pull-from-a-different-registry)\n",
|
|
"- Można również podać katalogi do podmontowania wewnątrz kontenera oraz wartości zmiennych środowiskowych do ustawienia w kontenerze:\n",
|
|
"```yaml\n",
|
|
"docker_env:\n",
|
|
" image: mlflow-docker-example-environment\n",
|
|
" volumes: [\"/local/path:/container/mount/path\"]\n",
|
|
" environment: [[\"NEW_ENV_VAR\", \"new_var_value\"], \"VAR_TO_COPY_FROM_HOST_ENVIRONMENT\"]\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Parametry\n",
|
|
" - Specyfikacja parametrów w pliku MLproject pozwala na ich walidację i używanie wartości domyślnych\n",
|
|
" - Dostępne typy:\n",
|
|
" - String\n",
|
|
" - Float - dowolna liczba (MLflow waliduje, czy podana wartość jest liczbą)\n",
|
|
" - Path - pozwala podawać ścieżki względne (przekształca je na bezwzlędne) do plików lokalnych albo do plików zdalnych (np. do s3://) - zostaną wtedy ściągnięte lokalnie\n",
|
|
" - URI - podobnie jak path, ale do rozproszonych systemów plików\n",
|
|
"\n",
|
|
"- [Składnia](https://mlflow.org/docs/latest/projects.html#specifying-parameters)\n",
|
|
" \n",
|
|
"```yml:\n",
|
|
" parameter_name: {type: data_type, default: value} # Short syntax\n",
|
|
"\n",
|
|
" parameter_name: # Long syntax\n",
|
|
" type: data_type\n",
|
|
" default: value\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Uruchamianie projektu\n",
|
|
" - Projekt możemy uruchomić przy pomocy polecenia `mlflow run` ([dokumentacja](https://mlflow.org/docs/latest/cli.html#mlflow-run))\n",
|
|
" - Spowoduje to przygotowanie środowiska i uruchomienie eksperymentu wewnątrz środowiska\n",
|
|
" - domyślnie zostanie uruchomione polecenie zdefiniowane w \"entry point\" `main`. Żeby uruchomić inny \"entry point\", możemy użyć parametru `-e`, np:\n",
|
|
" ```bash\n",
|
|
" mlflow run sklearn_elasticnet_wine -e test\n",
|
|
" ```\n",
|
|
" - Parametry do naszego polecenia możemy przekazywać przy pomocy flagi `-P`"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"2021/05/16 17:59:10 INFO mlflow.projects.utils: === Created directory /tmp/tmprq4mdosv for downloading remote URIs passed to arguments of type 'path' ===\n",
|
|
"2021/05/16 17:59:10 INFO mlflow.projects.backend.local: === Running command 'source /home/tomek/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-5987e03d4dbaa5faa1a697bb113be9b9bdc39b29 1>&2 && python train.py 0.42 0.1' in run with ID '1860d321ea1545ff8866e4ba199d1712' === \n",
|
|
"Elasticnet model (alpha=0.420000, l1_ratio=0.100000):\n",
|
|
" RMSE: 0.7420620899060748\n",
|
|
" MAE: 0.5722846717246247\n",
|
|
" R2: 0.21978513651550236\n",
|
|
"2021/05/16 17:59:19 INFO mlflow.projects: === Run (ID '1860d321ea1545ff8866e4ba199d1712') succeeded ===\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"!cd IUM_08/examples/; mlflow run sklearn_elasticnet_wine -P alpha=0.42"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"# Zadania [10p pkt]\n",
|
|
"1. Dodaj do swojego projektu logowanie parametrów i metryk za pomocą MLflow (polecenia `mlflow.log_param` i `mlflow.log_metric`\n",
|
|
"2. Dodaj plik MLProject definiujący polecenia do trenowania i testowania, ich parametry wywołania oraz środowisko (Conda albo Docker)\n",
|
|
"\n",
|
|
"**Termin wykonania zadań**: **15 maja 2024**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## MLflow Models\n",
|
|
"\n",
|
|
"MLflow Models to konwencja zapisu modeli, która ułatwia potem ich załadowanie i użycie"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"Rodzaje modeli (\"flavors\") wspierane przez MLflow:\n",
|
|
"\n",
|
|
" - Python Function (python_function)\n",
|
|
" - PyTorch (pytorch)\n",
|
|
" - TensorFlow (tensorflow)\n",
|
|
" - Keras (keras)\n",
|
|
" - Scikit-learn (sklearn)\n",
|
|
" - Spacy(spaCy)\n",
|
|
" - ONNX (onnx)\n",
|
|
" - R Function (crate)\n",
|
|
" - H2O (h2o)\n",
|
|
" - MLeap (mleap)\n",
|
|
" - Spark MLlib (spark)\n",
|
|
" - MXNet Gluon (gluon)\n",
|
|
" - XGBoost (xgboost)\n",
|
|
" - LightGBM (lightgbm)\n",
|
|
" - CatBoost (catboost)\n",
|
|
" - Fastai(fastai)\n",
|
|
" - Statsmodels (statsmodels)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Zapisywanie modelu\n",
|
|
"Model ML można zapisać w MLflow przy pomocy jednej z dwóch funkcji z pakietu odpowiadającego używanej przez nas bibliotece:\n",
|
|
" - `save_model()` - zapisuje model na dysku\n",
|
|
" - `log_model()` - zapisuje model razem z innymi informacjami (metrykami, parametrami). W zależności od ustawień [\"tracking_uri\"](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.set_tracking_uri) może być to lokalny folder w `mlruns/ ` lub ścieżka na zdalnym serwerze MLflow\n",
|
|
"\n",
|
|
"```Python\n",
|
|
" mlflow.sklearn.save_model(lr, \"my_model\")\n",
|
|
"```\n",
|
|
"\n",
|
|
"```Python\n",
|
|
" mlflow.keras.save_model(lr, \"my_model\")\n",
|
|
"```\n",
|
|
"\n",
|
|
"Wywołanie tej funkcji spowoduje stworzenie katalogu \"my_model\" zawierającego:\n",
|
|
" - plik *MLmodel* zawierający informacje o sposobach, w jaki model można załadować (\"flavors\") oraz ścieżki do plików związanych z modelem, takich jak:\n",
|
|
" - *conda.yaml* - opis środowiska potrzebnego do załadowania modelu\n",
|
|
" - *model.pkl* - plik z zserializowanym modelem\n",
|
|
"\n",
|
|
"Tylko plik *MLmodel* jest specjalnym plikiem MLflow - reszta zależy od konkrentego \"flavour\"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"conda.yaml MLmodel model.pkl\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"ls IUM_08/examples/my_model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 39,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"total 12\r\n",
|
|
"-rw-rw-r-- 1 tomek tomek 153 maj 17 10:38 conda.yaml\r\n",
|
|
"-rw-rw-r-- 1 tomek tomek 958 maj 17 10:38 MLmodel\r\n",
|
|
"-rw-rw-r-- 1 tomek tomek 641 maj 17 10:38 model.pkl\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"! ls -l IUM_08/examples/mlruns/0/b395b55b47fc43de876b67f5a4a5dae9/artifacts/model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# %load IUM_08/examples/mlruns/0/b395b55b47fc43de876b67f5a4a5dae9/artifacts/model/MLmodel\n",
|
|
"artifact_path: model\n",
|
|
"flavors:\n",
|
|
" python_function:\n",
|
|
" env: conda.yaml\n",
|
|
" loader_module: mlflow.sklearn\n",
|
|
" model_path: model.pkl\n",
|
|
" python_version: 3.9.1\n",
|
|
" sklearn:\n",
|
|
" pickled_model: model.pkl\n",
|
|
" serialization_format: cloudpickle\n",
|
|
" sklearn_version: 0.24.2\n",
|
|
"run_id: b395b55b47fc43de876b67f5a4a5dae9\n",
|
|
"signature:\n",
|
|
" inputs: '[{\"name\": \"fixed acidity\", \"type\": \"double\"}, {\"name\": \"volatile acidity\",\n",
|
|
" \"type\": \"double\"}, {\"name\": \"citric acid\", \"type\": \"double\"}, {\"name\": \"residual\n",
|
|
" sugar\", \"type\": \"double\"}, {\"name\": \"chlorides\", \"type\": \"double\"}, {\"name\": \"free\n",
|
|
" sulfur dioxide\", \"type\": \"double\"}, {\"name\": \"total sulfur dioxide\", \"type\": \"double\"},\n",
|
|
" {\"name\": \"density\", \"type\": \"double\"}, {\"name\": \"pH\", \"type\": \"double\"}, {\"name\":\n",
|
|
" \"sulphates\", \"type\": \"double\"}, {\"name\": \"alcohol\", \"type\": \"double\"}]'\n",
|
|
" outputs: '[{\"type\": \"tensor\", \"tensor-spec\": {\"dtype\": \"float64\", \"shape\": [-1]}}]'\n",
|
|
"utc_time_created: '2021-05-17 08:38:41.749670'\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# %load IUM_08/examples/my_model/conda.yaml\n",
|
|
"channels:\n",
|
|
"- defaults\n",
|
|
"- conda-forge\n",
|
|
"dependencies:\n",
|
|
"- python=3.9.1\n",
|
|
"- pip\n",
|
|
"- pip:\n",
|
|
" - mlflow\n",
|
|
" - scikit-learn==0.24.2\n",
|
|
" - cloudpickle==1.6.0\n",
|
|
"name: mlflow-env"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Dodatkowe pola w MLmodel\n",
|
|
"\n",
|
|
"\n",
|
|
"- *utc_time_created* - timestamp z czasem stworzenia modelu\n",
|
|
"- *run_id* - ID uruchomienia (\"run\"), które stworzyło ten model, jeśli model był zapisany za pomocą MLflow Tracking.\n",
|
|
"- *signature* - opisa danych wejściowych i wyjściowych w formacie JSON\n",
|
|
"- *input_example* przykładowe wejście przyjmowane przez model. Można je podać poprzez parametr `input_example` funkcji [log_model](https://mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.log_model)\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 50,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"array([5.57688397, 5.50664777, 5.52550482, 5.50431125, 5.57688397])"
|
|
]
|
|
},
|
|
"execution_count": 50,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import mlflow\n",
|
|
"import pandas as pd\n",
|
|
"model = mlflow.sklearn.load_model(\"IUM_08/examples/mlruns/0/b395b55b47fc43de876b67f5a4a5dae9/artifacts/model\")\n",
|
|
"csv_url = \"http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv\"\n",
|
|
"data = pd.read_csv(csv_url, sep=\";\")\n",
|
|
"model.predict(data.drop([\"quality\"], axis=1).head())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"### Serwowanie modeli"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Usage: mlflow models [OPTIONS] COMMAND [ARGS]...\r\n",
|
|
"\r\n",
|
|
" Deploy MLflow models locally.\r\n",
|
|
"\r\n",
|
|
" To deploy a model associated with a run on a tracking server, set the\r\n",
|
|
" MLFLOW_TRACKING_URI environment variable to the URL of the desired server.\r\n",
|
|
"\r\n",
|
|
"Options:\r\n",
|
|
" --help Show this message and exit.\r\n",
|
|
"\r\n",
|
|
"Commands:\r\n",
|
|
" build-docker **EXPERIMENTAL**: Builds a Docker image whose default...\r\n",
|
|
" predict Generate predictions in json format using a saved MLflow...\r\n",
|
|
" prepare-env **EXPERIMENTAL**: Performs any preparation necessary to...\r\n",
|
|
" serve Serve a model saved with MLflow by launching a webserver on...\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"!cd IUM_08/examples/; mlflow models --help"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Usage: mlflow models serve [OPTIONS]\r\n",
|
|
"\r\n",
|
|
" Serve a model saved with MLflow by launching a webserver on the specified\r\n",
|
|
" host and port. The command supports models with the ``python_function`` or\r\n",
|
|
" ``crate`` (R Function) flavor. For information about the input data\r\n",
|
|
" formats accepted by the webserver, see the following documentation:\r\n",
|
|
" https://www.mlflow.org/docs/latest/models.html#built-in-deployment-tools.\r\n",
|
|
"\r\n",
|
|
" You can make requests to ``POST /invocations`` in pandas split- or record-\r\n",
|
|
" oriented formats.\r\n",
|
|
"\r\n",
|
|
" Example:\r\n",
|
|
"\r\n",
|
|
" .. code-block:: bash\r\n",
|
|
"\r\n",
|
|
" $ mlflow models serve -m runs:/my-run-id/model-path &\r\n",
|
|
"\r\n",
|
|
" $ curl http://127.0.0.1:5000/invocations -H 'Content-Type:\r\n",
|
|
" application/json' -d '{ \"columns\": [\"a\", \"b\", \"c\"],\r\n",
|
|
" \"data\": [[1, 2, 3], [4, 5, 6]] }'\r\n",
|
|
"\r\n",
|
|
"Options:\r\n",
|
|
" -m, --model-uri URI URI to the model. A local path, a 'runs:/' URI, or a\r\n",
|
|
" remote storage URI (e.g., an 's3://' URI). For more\r\n",
|
|
" information about supported remote URIs for model\r\n",
|
|
" artifacts, see\r\n",
|
|
" https://mlflow.org/docs/latest/tracking.html#artifact-\r\n",
|
|
" stores [required]\r\n",
|
|
"\r\n",
|
|
" -p, --port INTEGER The port to listen on (default: 5000).\r\n",
|
|
" -h, --host HOST The network address to listen on (default: 127.0.0.1).\r\n",
|
|
" Use 0.0.0.0 to bind to all addresses if you want to\r\n",
|
|
" access the tracking server from other machines.\r\n",
|
|
"\r\n",
|
|
" -w, --workers TEXT Number of gunicorn worker processes to handle requests\r\n",
|
|
" (default: 4).\r\n",
|
|
"\r\n",
|
|
" --no-conda If specified, will assume that MLmodel/MLproject is\r\n",
|
|
" running within a Conda environment with the necessary\r\n",
|
|
" dependencies for the current project instead of\r\n",
|
|
" attempting to create a new conda environment.\r\n",
|
|
"\r\n",
|
|
" --install-mlflow If specified and there is a conda environment to be\r\n",
|
|
" activated mlflow will be installed into the environment\r\n",
|
|
" after it has been activated. The version of installed\r\n",
|
|
" mlflow will be the same asthe one used to invoke this\r\n",
|
|
" command.\r\n",
|
|
"\r\n",
|
|
" --help Show this message and exit.\r\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"!cd IUM_08/examples/; mlflow models serve --help"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 54,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{\"columns\":[\"fixed acidity\",\"volatile acidity\",\"citric acid\",\"residual sugar\",\"chlorides\",\"free sulfur dioxide\",\"total sulfur dioxide\",\"density\",\"pH\",\"sulphates\",\"alcohol\"],\"index\":[0],\"data\":[[7.4,0.7,0.0,1.9,0.076,11.0,34.0,0.9978,3.51,0.56,9.4]]}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import pandas as pd\n",
|
|
"csv_url = \"http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv\"\n",
|
|
"data = pd.read_csv(csv_url, sep=\";\").drop([\"quality\"], axis=1).head(1).to_json(orient='split')\n",
|
|
"print(data)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 57,
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[5.576883967129615]"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"!curl http://127.0.0.1:5003/invocations -H 'Content-Type: application/json' -d '{\\\n",
|
|
" \"columns\":[\\\n",
|
|
" \"fixed acidity\",\"volatile acidity\",\"citric acid\",\"residual sugar\",\"chlorides\",\"free sulfur dioxide\",\"total sulfur dioxide\",\"density\",\"pH\",\"sulphates\",\"alcohol\"],\\\n",
|
|
" \"index\":[0],\\\n",
|
|
" \"data\":[[7.4,0.7,0.0,1.9,0.076,11.0,34.0,0.9978,3.51,0.56,9.4]]}'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"```\n",
|
|
"$ cd IUM_08/examples/\n",
|
|
"$ mlflow models serve -m my_model\n",
|
|
"2021/05/17 08:52:07 INFO mlflow.models.cli: Selected backend for flavor 'python_function'\n",
|
|
"2021/05/17 08:52:07 INFO mlflow.pyfunc.backend: === Running command 'source /home/tomek/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-503f0c7520a32f054a9d168bd099584a9439de9d 1>&2 && gunicorn --timeout=60 -b 127.0.0.1:5003 -w 1 ${GUNICORN_CMD_ARGS} -- mlflow.pyfunc.scoring_server.wsgi:app'\n",
|
|
"[2021-05-17 08:52:07 +0200] [291217] [INFO] Starting gunicorn 20.1.0\n",
|
|
"[2021-05-17 08:52:07 +0200] [291217] [INFO] Listening at: http://127.0.0.1:5003 (291217)\n",
|
|
"[2021-05-17 08:52:07 +0200] [291217] [INFO] Using worker: sync\n",
|
|
"[2021-05-17 08:52:07 +0200] [291221] [INFO] Booting worker with pid: 291221\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
}
|
|
},
|
|
"source": [
|
|
"## MLflow Registry\n",
|
|
" - umożliwia [zapisywanie](https://mlflow.org/docs/latest/model-registry.html#adding-an-mlflow-model-to-the-model-registry) i [ładowanie](https://mlflow.org/docs/latest/model-registry.html#fetching-an-mlflow-model-from-the-model-registry) modeli z centralnego rejestru\n",
|
|
" - Modele można też serwować bezpośrednio z rejestru:\n",
|
|
"\n",
|
|
"```bash\n",
|
|
"#!/usr/bin/env sh\n",
|
|
"\n",
|
|
"# Set environment variable for the tracking URL where the Model Registry resides\n",
|
|
"export MLFLOW_TRACKING_URI=http://localhost:5000\n",
|
|
"\n",
|
|
"# Serve the production model from the model registry\n",
|
|
"mlflow models serve -m \"models:/sk-learn-random-forest-reg-model/Production\"\n",
|
|
"```\n",
|
|
"\n",
|
|
"- Żeby było to możliwe, musimy mieć uruchomiony [serwer MLflow](https://mlflow.org/docs/latest/tracking.html#tracking-server)\n",
|
|
"- Umożliwia zarządzanie wersjami modeli i oznaczanie ich różnymi fazami, np. \"Staging\", \"Production\""
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"author": "Tomasz Ziętkiewicz",
|
|
"celltoolbar": "Slideshow",
|
|
"email": "tomasz.zietkiewicz@amu.edu.pl",
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"lang": "pl",
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.12"
|
|
},
|
|
"slideshow": {
|
|
"slide_type": "slide"
|
|
},
|
|
"subtitle": "8.MLFlow[laboratoria]",
|
|
"title": "Inżynieria uczenia maszynowego",
|
|
"year": "2021"
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|