522 KiB
522 KiB
Uczenie Głębokie - projekt
W projekcie wykorzystano dataset emotion, zawierający wpisy nacechowane określonymi emocjami.
Labels:
- 0 - sadness
- 1 - joy
- 2 - love
- 3 - anger
- 4 - fear
- 5 - surprise
REQUIREMENTS
!pip3 install transformers scikit-learn accelerate evaluate datasets torch sentencepiece torchvision sacrebleu
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Requirement already satisfied: transformers in /usr/local/lib/python3.8/dist-packages (4.26.1) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.8/dist-packages (1.0.2) Requirement already satisfied: accelerate in /usr/local/lib/python3.8/dist-packages (0.16.0) Requirement already satisfied: evaluate in /usr/local/lib/python3.8/dist-packages (0.4.0) Requirement already satisfied: datasets in /usr/local/lib/python3.8/dist-packages (2.9.0) Requirement already satisfied: torch in /usr/local/lib/python3.8/dist-packages (1.13.1+cu116) Requirement already satisfied: sentencepiece in /usr/local/lib/python3.8/dist-packages (0.1.97) Requirement already satisfied: torchvision in /usr/local/lib/python3.8/dist-packages (0.14.1+cu116) Collecting sacrebleu Downloading sacrebleu-2.3.1-py3-none-any.whl (118 kB) [2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m118.9/118.9 KB[0m [31m7.0 MB/s[0m eta [36m0:00:00[0m [?25hRequirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (0.12.0) Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6) Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (0.13.2) Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1) Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (23.0) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2) Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.25.1) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.8/dist-packages (from scikit-learn) (3.1.0) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.8/dist-packages (from scikit-learn) (1.2.0) Requirement already satisfied: scipy>=1.1.0 in /usr/local/lib/python3.8/dist-packages (from scikit-learn) (1.7.3) Requirement already satisfied: psutil in /usr/local/lib/python3.8/dist-packages (from accelerate) (5.4.8) Requirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.8/dist-packages (from evaluate) (2023.1.0) Requirement already satisfied: multiprocess in /usr/local/lib/python3.8/dist-packages (from evaluate) (0.70.14) Requirement already satisfied: pandas in /usr/local/lib/python3.8/dist-packages (from evaluate) (1.3.5) Requirement already satisfied: xxhash in /usr/local/lib/python3.8/dist-packages (from evaluate) (3.2.0) Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.8/dist-packages (from evaluate) (0.18.0) Requirement already satisfied: dill in /usr/local/lib/python3.8/dist-packages (from evaluate) (0.3.6) Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (9.0.0) Requirement already satisfied: aiohttp in /usr/local/lib/python3.8/dist-packages (from datasets) (3.8.3) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch) (4.4.0) Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.8/dist-packages (from torchvision) (7.1.2) Requirement already satisfied: lxml in /usr/local/lib/python3.8/dist-packages (from sacrebleu) (4.9.2) Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.8/dist-packages (from sacrebleu) (0.8.10) Collecting portalocker Downloading portalocker-2.7.0-py2.py3-none-any.whl (15 kB) Collecting colorama Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB) Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.1) Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.8.2) Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (6.0.4) Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (2.1.1) Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (4.0.2) Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (22.2.0) Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.3) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.26.14) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10) Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (4.0.0) Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas->evaluate) (2.8.2) Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas->evaluate) (2022.7.1) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.7.3->pandas->evaluate) (1.15.0) Installing collected packages: portalocker, colorama, sacrebleu Successfully installed colorama-0.4.6 portalocker-2.7.0 sacrebleu-2.3.1
import os
import json
from pathlib import Path
from typing import Dict, List
from datasets import load_dataset
import torch
import pandas as pd
os.environ['TOKENIZERS_PARALLELISM'] = 'true'
DATA PREP
!mkdir -p data
!python data_prep.py
Downloading builder script: 100% 3.97k/3.97k [00:00<00:00, 3.28MB/s] Downloading metadata: 100% 3.28k/3.28k [00:00<00:00, 2.79MB/s] Downloading readme: 100% 8.78k/8.78k [00:00<00:00, 7.56MB/s] No config specified, defaulting to: emotion/split Downloading and preparing dataset emotion/split to /root/.cache/huggingface/datasets/emotion/split/1.0.0/cca5efe2dfeb58c1d098e0f9eeb200e9927d889b5a03c67097275dfb5fe463bd... Downloading data files: 0% 0/3 [00:00<?, ?it/s] Downloading data: 0% 0.00/592k [00:00<?, ?B/s][A Downloading data: 9% 52.2k/592k [00:00<00:01, 438kB/s][A Downloading data: 100% 592k/592k [00:00<00:00, 2.36MB/s] Downloading data files: 33% 1/3 [00:01<00:02, 1.35s/it] Downloading data: 100% 74.0k/74.0k [00:00<00:00, 7.40MB/s] Downloading data files: 67% 2/3 [00:01<00:00, 1.10it/s] Downloading data: 100% 74.9k/74.9k [00:00<00:00, 8.28MB/s] Downloading data files: 100% 3/3 [00:02<00:00, 1.15it/s] Extracting data files: 100% 3/3 [00:00<00:00, 159.24it/s] Dataset emotion downloaded and prepared to /root/.cache/huggingface/datasets/emotion/split/1.0.0/cca5efe2dfeb58c1d098e0f9eeb200e9927d889b5a03c67097275dfb5fe463bd. Subsequent calls will reuse this data. 100% 3/3 [00:00<00:00, 736.49it/s] Saving into: data/train.json Saving into: data/s2s-train.json Saving into: data/valid.json Saving into: data/s2s-valid.json Saving into: data/test.json Saving into: data/s2s-test.json
!head data/train.json
{"label": 0, "text": "i didnt feel humiliated"} {"label": 0, "text": "i can go from feeling so hopeless to so damned hopeful just from being around someone who cares and is awake"} {"label": 3, "text": "im grabbing a minute to post i feel greedy wrong"} {"label": 2, "text": "i am ever feeling nostalgic about the fireplace i will know that it is still on the property"} {"label": 3, "text": "i am feeling grouchy"} {"label": 0, "text": "ive been feeling a little burdened lately wasnt sure why that was"} {"label": 5, "text": "ive been taking or milligrams or times recommended amount and ive fallen asleep a lot faster but i also feel like so funny"} {"label": 4, "text": "i feel as confused about life as a teenager or as jaded as a year old man"} {"label": 1, "text": "i have been with petronas for years i feel that petronas has performed well and made a huge profit"} {"label": 2, "text": "i feel romantic too"}
!head data/s2s-train.json
{"label": "sadness", "text": "i didnt feel humiliated"} {"label": "sadness", "text": "i can go from feeling so hopeless to so damned hopeful just from being around someone who cares and is awake"} {"label": "anger", "text": "im grabbing a minute to post i feel greedy wrong"} {"label": "love", "text": "i am ever feeling nostalgic about the fireplace i will know that it is still on the property"} {"label": "anger", "text": "i am feeling grouchy"} {"label": "sadness", "text": "ive been feeling a little burdened lately wasnt sure why that was"} {"label": "surprise", "text": "ive been taking or milligrams or times recommended amount and ive fallen asleep a lot faster but i also feel like so funny"} {"label": "fear", "text": "i feel as confused about life as a teenager or as jaded as a year old man"} {"label": "joy", "text": "i have been with petronas for years i feel that petronas has performed well and made a huge profit"} {"label": "love", "text": "i feel romantic too"}
!wc -l data/*
2000 data/s2s-test.json 16000 data/s2s-train.json 2000 data/s2s-valid.json 2000 data/test.json 16000 data/train.json 2000 data/valid.json 40000 total
!mkdir -p cache
ROBERTA
- full data
- model
roberta-base
- sequnece length: 128
- training epoch: 1
!python run_glue.py \
--cache_dir cache/roberta \
--model_name_or_path roberta-base \
--train_file data/train.json \
--validation_file data/valid.json \
--test_file data/test.json \
--per_device_train_batch_size 24 \
--per_device_eval_batch_size 24 \
--do_train \
--do_eval \
--do_predict \
--max_seq_length 128 \
--learning_rate 2e-5 \
--num_train_epochs 1 \
--output_dir out/emotion/roberta \
--overwrite_output_dir
2023-02-14 23:03:23.770695: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-02-14 23:03:24.398188: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2023-02-14 23:03:26.384825: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-02-14 23:03:26.384975: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-02-14 23:03:26.384997: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. WARNING:__main__:Process rank: -1, device: cuda:0, n_gpu: 1distributed training: False, 16-bits training: False INFO:__main__:Training/evaluation parameters TrainingArguments( _n_gpu=1, adafactor=False, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, auto_find_batch_size=False, bf16=False, bf16_full_eval=False, data_seed=None, dataloader_drop_last=False, dataloader_num_workers=0, dataloader_pin_memory=True, ddp_bucket_cap_mb=None, ddp_find_unused_parameters=None, ddp_timeout=1800, debug=[], deepspeed=None, disable_tqdm=False, do_eval=True, do_predict=True, do_train=True, eval_accumulation_steps=None, eval_delay=0, eval_steps=None, evaluation_strategy=no, fp16=False, fp16_backend=auto, fp16_full_eval=False, fp16_opt_level=O1, fsdp=[], fsdp_min_num_params=0, fsdp_transformer_layer_cls_to_wrap=None, full_determinism=False, gradient_accumulation_steps=1, gradient_checkpointing=False, greater_is_better=None, group_by_length=False, half_precision_backend=auto, hub_model_id=None, hub_private_repo=False, hub_strategy=every_save, hub_token=<HUB_TOKEN>, ignore_data_skip=False, include_inputs_for_metrics=False, jit_mode_eval=False, label_names=None, label_smoothing_factor=0.0, learning_rate=2e-05, length_column_name=length, load_best_model_at_end=False, local_rank=-1, log_level=passive, log_level_replica=passive, log_on_each_node=True, logging_dir=out/emotion/roberta/runs/Feb14_23-03-30_fc0011e45a00, logging_first_step=False, logging_nan_inf_filter=True, logging_steps=500, logging_strategy=steps, lr_scheduler_type=linear, max_grad_norm=1.0, max_steps=-1, metric_for_best_model=None, mp_parameters=, no_cuda=False, num_train_epochs=1.0, optim=adamw_hf, output_dir=out/emotion/roberta, overwrite_output_dir=True, past_index=-1, per_device_eval_batch_size=24, per_device_train_batch_size=24, prediction_loss_only=False, push_to_hub=False, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=<PUSH_TO_HUB_TOKEN>, ray_scope=last, remove_unused_columns=True, report_to=['tensorboard'], resume_from_checkpoint=None, run_name=out/emotion/roberta, save_on_each_node=False, save_steps=500, save_strategy=steps, save_total_limit=None, seed=42, sharded_ddp=[], skip_memory_metrics=True, tf32=None, torchdynamo=None, tpu_metrics_debug=False, tpu_num_cores=None, use_ipex=False, use_legacy_prediction_loop=False, use_mps_device=False, warmup_ratio=0.0, warmup_steps=0, weight_decay=0.0, xpu_backend=None, ) INFO:__main__:load a local file for train: data/train.json INFO:__main__:load a local file for validation: data/valid.json INFO:__main__:load a local file for test: data/test.json WARNING:datasets.builder:Using custom data configuration default-e1b3a7f886502404 INFO:datasets.info:Loading Dataset Infos from /usr/local/lib/python3.8/dist-packages/datasets/packaged_modules/json INFO:datasets.builder:Generating dataset json (/content/cache/roberta/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51) Downloading and preparing dataset json/default to /content/cache/roberta/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51... Downloading data files: 100% 3/3 [00:00<00:00, 14546.72it/s] INFO:datasets.download.download_manager:Downloading took 0.0 min INFO:datasets.download.download_manager:Checksum Computation took 0.0 min Extracting data files: 100% 3/3 [00:00<00:00, 2116.91it/s] INFO:datasets.utils.info_utils:Unable to verify checksums. INFO:datasets.builder:Generating train split INFO:datasets.builder:Generating validation split INFO:datasets.builder:Generating test split INFO:datasets.utils.info_utils:Unable to verify splits sizes. Dataset json downloaded and prepared to /content/cache/roberta/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51. Subsequent calls will reuse this data. 100% 3/3 [00:00<00:00, 650.01it/s] Downloading (…)lve/main/config.json: 100% 481/481 [00:00<00:00, 89.2kB/s] [INFO|configuration_utils.py:653] 2023-02-14 23:03:31,805 >> loading configuration file config.json from cache at cache/roberta/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/config.json [INFO|configuration_utils.py:705] 2023-02-14 23:03:31,805 >> Model config RobertaConfig { "_name_or_path": "roberta-base", "architectures": [ "RobertaForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "classifier_dropout": null, "eos_token_id": 2, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "id2label": { "0": "LABEL_0", "1": "LABEL_1", "2": "LABEL_2", "3": "LABEL_3", "4": "LABEL_4", "5": "LABEL_5" }, "initializer_range": 0.02, "intermediate_size": 3072, "label2id": { "LABEL_0": 0, "LABEL_1": 1, "LABEL_2": 2, "LABEL_3": 3, "LABEL_4": 4, "LABEL_5": 5 }, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "roberta", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 1, "position_embedding_type": "absolute", "transformers_version": "4.23.1", "type_vocab_size": 1, "use_cache": true, "vocab_size": 50265 } [INFO|tokenization_auto.py:418] 2023-02-14 23:03:31,898 >> Could not locate the tokenizer configuration file, will try to use the model config instead. [INFO|configuration_utils.py:653] 2023-02-14 23:03:31,989 >> loading configuration file config.json from cache at cache/roberta/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/config.json [INFO|configuration_utils.py:705] 2023-02-14 23:03:31,990 >> Model config RobertaConfig { "_name_or_path": "roberta-base", "architectures": [ "RobertaForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "classifier_dropout": null, "eos_token_id": 2, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "roberta", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 1, "position_embedding_type": "absolute", "transformers_version": "4.23.1", "type_vocab_size": 1, "use_cache": true, "vocab_size": 50265 } Downloading (…)olve/main/vocab.json: 100% 899k/899k [00:00<00:00, 9.68MB/s] Downloading (…)olve/main/merges.txt: 100% 456k/456k [00:00<00:00, 4.94MB/s] Downloading (…)/main/tokenizer.json: 100% 1.36M/1.36M [00:00<00:00, 11.9MB/s] [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:03:33,211 >> loading file vocab.json from cache at cache/roberta/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/vocab.json [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:03:33,211 >> loading file merges.txt from cache at cache/roberta/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/merges.txt [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:03:33,211 >> loading file tokenizer.json from cache at cache/roberta/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/tokenizer.json [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:03:33,211 >> loading file added_tokens.json from cache at None [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:03:33,211 >> loading file special_tokens_map.json from cache at None [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:03:33,211 >> loading file tokenizer_config.json from cache at None [INFO|configuration_utils.py:653] 2023-02-14 23:03:33,211 >> loading configuration file config.json from cache at cache/roberta/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/config.json [INFO|configuration_utils.py:705] 2023-02-14 23:03:33,212 >> Model config RobertaConfig { "_name_or_path": "roberta-base", "architectures": [ "RobertaForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "classifier_dropout": null, "eos_token_id": 2, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "roberta", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 1, "position_embedding_type": "absolute", "transformers_version": "4.23.1", "type_vocab_size": 1, "use_cache": true, "vocab_size": 50265 } INFO:__main__:Using implementation from class: AutoModelForSequenceClassification Downloading (…)"pytorch_model.bin";: 100% 501M/501M [00:04<00:00, 105MB/s] [INFO|modeling_utils.py:2156] 2023-02-14 23:03:38,350 >> loading weights file pytorch_model.bin from cache at cache/roberta/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/pytorch_model.bin [WARNING|modeling_utils.py:2596] 2023-02-14 23:03:39,757 >> Some weights of the model checkpoint at roberta-base were not used when initializing RobertaForSequenceClassification: ['roberta.pooler.dense.weight', 'lm_head.bias', 'lm_head.layer_norm.weight', 'lm_head.layer_norm.bias', 'lm_head.decoder.weight', 'roberta.pooler.dense.bias', 'lm_head.dense.bias', 'lm_head.dense.weight'] - This IS expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS NOT expected if you are initializing RobertaForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). [WARNING|modeling_utils.py:2608] 2023-02-14 23:03:39,757 >> Some weights of RobertaForSequenceClassification were not initialized from the model checkpoint at roberta-base and are newly initialized: ['classifier.dense.weight', 'classifier.out_proj.weight', 'classifier.dense.bias', 'classifier.out_proj.bias'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Frozen layers: [('roberta.encoder.layer.0.attention.self.query.weight', False), ('roberta.encoder.layer.0.attention.self.query.bias', False), ('roberta.encoder.layer.0.attention.self.key.weight', False), ('roberta.encoder.layer.0.attention.self.key.bias', False), ('roberta.encoder.layer.0.attention.self.value.weight', False), ('roberta.encoder.layer.0.attention.self.value.bias', False), ('roberta.encoder.layer.0.attention.output.dense.weight', False), ('roberta.encoder.layer.0.attention.output.dense.bias', False), ('roberta.encoder.layer.0.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.0.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.0.intermediate.dense.weight', False), ('roberta.encoder.layer.0.intermediate.dense.bias', False), ('roberta.encoder.layer.0.output.dense.weight', False), ('roberta.encoder.layer.0.output.dense.bias', False), ('roberta.encoder.layer.0.output.LayerNorm.weight', False), ('roberta.encoder.layer.0.output.LayerNorm.bias', False), ('roberta.encoder.layer.2.attention.self.query.weight', False), ('roberta.encoder.layer.2.attention.self.query.bias', False), ('roberta.encoder.layer.2.attention.self.key.weight', False), ('roberta.encoder.layer.2.attention.self.key.bias', False), ('roberta.encoder.layer.2.attention.self.value.weight', False), ('roberta.encoder.layer.2.attention.self.value.bias', False), ('roberta.encoder.layer.2.attention.output.dense.weight', False), ('roberta.encoder.layer.2.attention.output.dense.bias', False), ('roberta.encoder.layer.2.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.2.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.2.intermediate.dense.weight', False), ('roberta.encoder.layer.2.intermediate.dense.bias', False), ('roberta.encoder.layer.2.output.dense.weight', False), ('roberta.encoder.layer.2.output.dense.bias', False), ('roberta.encoder.layer.2.output.LayerNorm.weight', False), ('roberta.encoder.layer.2.output.LayerNorm.bias', False), ('roberta.encoder.layer.4.attention.self.query.weight', False), ('roberta.encoder.layer.4.attention.self.query.bias', False), ('roberta.encoder.layer.4.attention.self.key.weight', False), ('roberta.encoder.layer.4.attention.self.key.bias', False), ('roberta.encoder.layer.4.attention.self.value.weight', False), ('roberta.encoder.layer.4.attention.self.value.bias', False), ('roberta.encoder.layer.4.attention.output.dense.weight', False), ('roberta.encoder.layer.4.attention.output.dense.bias', False), ('roberta.encoder.layer.4.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.4.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.4.intermediate.dense.weight', False), ('roberta.encoder.layer.4.intermediate.dense.bias', False), ('roberta.encoder.layer.4.output.dense.weight', False), ('roberta.encoder.layer.4.output.dense.bias', False), ('roberta.encoder.layer.4.output.LayerNorm.weight', False), ('roberta.encoder.layer.4.output.LayerNorm.bias', False), ('roberta.encoder.layer.6.attention.self.query.weight', False), ('roberta.encoder.layer.6.attention.self.query.bias', False), ('roberta.encoder.layer.6.attention.self.key.weight', False), ('roberta.encoder.layer.6.attention.self.key.bias', False), ('roberta.encoder.layer.6.attention.self.value.weight', False), ('roberta.encoder.layer.6.attention.self.value.bias', False), ('roberta.encoder.layer.6.attention.output.dense.weight', False), ('roberta.encoder.layer.6.attention.output.dense.bias', False), ('roberta.encoder.layer.6.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.6.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.6.intermediate.dense.weight', False), ('roberta.encoder.layer.6.intermediate.dense.bias', False), ('roberta.encoder.layer.6.output.dense.weight', False), ('roberta.encoder.layer.6.output.dense.bias', False), ('roberta.encoder.layer.6.output.LayerNorm.weight', False), ('roberta.encoder.layer.6.output.LayerNorm.bias', False), ('roberta.encoder.layer.8.attention.self.query.weight', False), ('roberta.encoder.layer.8.attention.self.query.bias', False), ('roberta.encoder.layer.8.attention.self.key.weight', False), ('roberta.encoder.layer.8.attention.self.key.bias', False), ('roberta.encoder.layer.8.attention.self.value.weight', False), ('roberta.encoder.layer.8.attention.self.value.bias', False), ('roberta.encoder.layer.8.attention.output.dense.weight', False), ('roberta.encoder.layer.8.attention.output.dense.bias', False), ('roberta.encoder.layer.8.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.8.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.8.intermediate.dense.weight', False), ('roberta.encoder.layer.8.intermediate.dense.bias', False), ('roberta.encoder.layer.8.output.dense.weight', False), ('roberta.encoder.layer.8.output.dense.bias', False), ('roberta.encoder.layer.8.output.LayerNorm.weight', False), ('roberta.encoder.layer.8.output.LayerNorm.bias', False), ('roberta.encoder.layer.10.attention.self.query.weight', False), ('roberta.encoder.layer.10.attention.self.query.bias', False), ('roberta.encoder.layer.10.attention.self.key.weight', False), ('roberta.encoder.layer.10.attention.self.key.bias', False), ('roberta.encoder.layer.10.attention.self.value.weight', False), ('roberta.encoder.layer.10.attention.self.value.bias', False), ('roberta.encoder.layer.10.attention.output.dense.weight', False), ('roberta.encoder.layer.10.attention.output.dense.bias', False), ('roberta.encoder.layer.10.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.10.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.10.intermediate.dense.weight', False), ('roberta.encoder.layer.10.intermediate.dense.bias', False), ('roberta.encoder.layer.10.output.dense.weight', False), ('roberta.encoder.layer.10.output.dense.bias', False), ('roberta.encoder.layer.10.output.LayerNorm.weight', False), ('roberta.encoder.layer.10.output.LayerNorm.bias', False)] Running tokenizer on dataset: 0% 0/16 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/cache/roberta/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-92255ed6858d2f1f.arrow Running tokenizer on dataset: 100% 16/16 [00:00<00:00, 19.60ba/s] Running tokenizer on dataset: 0% 0/2 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/cache/roberta/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-541fdac4841bc78b.arrow Running tokenizer on dataset: 100% 2/2 [00:00<00:00, 7.48ba/s] Running tokenizer on dataset: 0% 0/2 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/cache/roberta/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-7c07449907d4dcfe.arrow Running tokenizer on dataset: 100% 2/2 [00:00<00:00, 20.93ba/s] INFO:__main__:Sample 10476 of the training set: {'label': 0, 'text': 'i do find new friends i m going to try extra hard to make them stay and if i decide that i don t want to feel hurt again and just ride out the last year of school on my own i m going to have to try extra hard not to care what people think of me being a loner', 'input_ids': [0, 118, 109, 465, 92, 964, 939, 475, 164, 7, 860, 1823, 543, 7, 146, 106, 1095, 8, 114, 939, 2845, 14, 939, 218, 326, 236, 7, 619, 2581, 456, 8, 95, 3068, 66, 5, 94, 76, 9, 334, 15, 127, 308, 939, 475, 164, 7, 33, 7, 860, 1823, 543, 45, 7, 575, 99, 82, 206, 9, 162, 145, 10, 784, 9604, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. INFO:__main__:Sample 1824 of the training set: {'label': 1, 'text': 'i asked them to join me in creating a world where all year old girls could grow up feeling hopeful and powerful', 'input_ids': [0, 118, 553, 106, 7, 1962, 162, 11, 2351, 10, 232, 147, 70, 76, 793, 1972, 115, 1733, 62, 2157, 7917, 8, 2247, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. INFO:__main__:Sample 409 of the training set: {'label': 2, 'text': 'i feel when you are a caring person you attract other caring people into your life', 'input_ids': [0, 118, 619, 77, 47, 32, 10, 10837, 621, 47, 5696, 97, 10837, 82, 88, 110, 301, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. [INFO|trainer.py:725] 2023-02-14 23:03:48,468 >> The following columns in the training set don't have a corresponding argument in `RobertaForSequenceClassification.forward` and have been ignored: text. If text are not expected by `RobertaForSequenceClassification.forward`, you can safely ignore this message. /usr/local/lib/python3.8/dist-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning warnings.warn( [INFO|trainer.py:1607] 2023-02-14 23:03:48,481 >> ***** Running training ***** [INFO|trainer.py:1608] 2023-02-14 23:03:48,481 >> Num examples = 16000 [INFO|trainer.py:1609] 2023-02-14 23:03:48,481 >> Num Epochs = 1 [INFO|trainer.py:1610] 2023-02-14 23:03:48,481 >> Instantaneous batch size per device = 24 [INFO|trainer.py:1611] 2023-02-14 23:03:48,481 >> Total train batch size (w. parallel, distributed & accumulation) = 24 [INFO|trainer.py:1612] 2023-02-14 23:03:48,481 >> Gradient Accumulation steps = 1 [INFO|trainer.py:1613] 2023-02-14 23:03:48,481 >> Total optimization steps = 667 {'loss': 0.8083, 'learning_rate': 5.0074962518740634e-06, 'epoch': 0.75} 75% 500/667 [00:58<00:18, 8.81it/s][INFO|trainer.py:2656] 2023-02-14 23:04:46,596 >> Saving model checkpoint to out/emotion/roberta/checkpoint-500 [INFO|configuration_utils.py:447] 2023-02-14 23:04:46,597 >> Configuration saved in out/emotion/roberta/checkpoint-500/config.json [INFO|modeling_utils.py:1624] 2023-02-14 23:04:47,328 >> Model weights saved in out/emotion/roberta/checkpoint-500/pytorch_model.bin [INFO|tokenization_utils_base.py:2123] 2023-02-14 23:04:47,329 >> tokenizer config file saved in out/emotion/roberta/checkpoint-500/tokenizer_config.json [INFO|tokenization_utils_base.py:2130] 2023-02-14 23:04:47,329 >> Special tokens file saved in out/emotion/roberta/checkpoint-500/special_tokens_map.json 100% 666/667 [01:18<00:00, 8.78it/s][INFO|trainer.py:1852] 2023-02-14 23:05:07,473 >> Training completed. Do not forget to share your model on huggingface.co/models =) {'train_runtime': 78.9923, 'train_samples_per_second': 202.551, 'train_steps_per_second': 8.444, 'train_loss': 0.7161429089227359, 'epoch': 1.0} 100% 667/667 [01:18<00:00, 8.45it/s] [INFO|trainer.py:2656] 2023-02-14 23:05:07,475 >> Saving model checkpoint to out/emotion/roberta [INFO|configuration_utils.py:447] 2023-02-14 23:05:07,476 >> Configuration saved in out/emotion/roberta/config.json [INFO|modeling_utils.py:1624] 2023-02-14 23:05:08,175 >> Model weights saved in out/emotion/roberta/pytorch_model.bin [INFO|tokenization_utils_base.py:2123] 2023-02-14 23:05:08,176 >> tokenizer config file saved in out/emotion/roberta/tokenizer_config.json [INFO|tokenization_utils_base.py:2130] 2023-02-14 23:05:08,176 >> Special tokens file saved in out/emotion/roberta/special_tokens_map.json ***** train metrics ***** epoch = 1.0 train_loss = 0.7161 train_runtime = 0:01:18.99 train_samples = 16000 train_samples_per_second = 202.551 train_steps_per_second = 8.444 INFO:__main__:*** Evaluate *** [INFO|trainer.py:725] 2023-02-14 23:05:08,275 >> The following columns in the evaluation set don't have a corresponding argument in `RobertaForSequenceClassification.forward` and have been ignored: text. If text are not expected by `RobertaForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:05:08,277 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:05:08,277 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:05:08,277 >> Batch size = 24 100% 84/84 [00:03<00:00, 23.63it/s] ***** eval metrics ***** epoch = 1.0 eval_accuracy = 0.889 eval_loss = 0.3302 eval_runtime = 0:00:03.60 eval_samples = 2000 eval_samples_per_second = 554.126 eval_steps_per_second = 23.273 INFO:__main__:*** Predict *** [INFO|trainer.py:725] 2023-02-14 23:05:11,890 >> The following columns in the test set don't have a corresponding argument in `RobertaForSequenceClassification.forward` and have been ignored: text. If text are not expected by `RobertaForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:05:11,891 >> ***** Running Prediction ***** [INFO|trainer.py:2909] 2023-02-14 23:05:11,891 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:05:11,891 >> Batch size = 24 100% 84/84 [00:03<00:00, 23.80it/s] INFO:__main__:***** Predict results None ***** [INFO|modelcard.py:444] 2023-02-14 23:05:15,585 >> Dropping the following result as it does not have all the necessary fields: {'task': {'name': 'Text Classification', 'type': 'text-classification'}, 'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.8889999985694885}]}
- full data
- sequence length: 128
- leakyRelu instad of relu
- every other layer frozen
- custom head
!python run_glue.py \
--cache_dir roberta_gpt_cache/roberta_custom \
--model_name_or_path roberta-base \
--custom_model roberta_custom \
--train_file data/train.json \
--validation_file data/valid.json \
--test_file data/test.json \
--per_device_train_batch_size 24 \
--per_device_eval_batch_size 24 \
--do_train \
--do_eval \
--do_predict \
--max_seq_length 128 \
--learning_rate 2e-5 \
--num_train_epochs 1 \
--output_dir roberta_gpt_out/roberta_custom \
--overwrite_output_dir
2023-02-15 16:41:47.706819: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-02-15 16:41:47.841368: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2023-02-15 16:41:48.597437: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-02-15 16:41:48.597583: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-02-15 16:41:48.597617: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. WARNING:__main__:Process rank: -1, device: cuda:0, n_gpu: 1distributed training: False, 16-bits training: False INFO:__main__:Training/evaluation parameters TrainingArguments( _n_gpu=1, adafactor=False, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, auto_find_batch_size=False, bf16=False, bf16_full_eval=False, data_seed=None, dataloader_drop_last=False, dataloader_num_workers=0, dataloader_pin_memory=True, ddp_bucket_cap_mb=None, ddp_find_unused_parameters=None, ddp_timeout=1800, debug=[], deepspeed=None, disable_tqdm=False, do_eval=True, do_predict=True, do_train=True, eval_accumulation_steps=None, eval_delay=0, eval_steps=None, evaluation_strategy=no, fp16=False, fp16_backend=auto, fp16_full_eval=False, fp16_opt_level=O1, fsdp=[], fsdp_min_num_params=0, fsdp_transformer_layer_cls_to_wrap=None, full_determinism=False, gradient_accumulation_steps=1, gradient_checkpointing=False, greater_is_better=None, group_by_length=False, half_precision_backend=auto, hub_model_id=None, hub_private_repo=False, hub_strategy=every_save, hub_token=<HUB_TOKEN>, ignore_data_skip=False, include_inputs_for_metrics=False, jit_mode_eval=False, label_names=None, label_smoothing_factor=0.0, learning_rate=2e-05, length_column_name=length, load_best_model_at_end=False, local_rank=-1, log_level=passive, log_level_replica=passive, log_on_each_node=True, logging_dir=roberta_gpt_out/roberta_custom/runs/Feb15_16-41-51_b7fb20e65b38, logging_first_step=False, logging_nan_inf_filter=True, logging_steps=500, logging_strategy=steps, lr_scheduler_type=linear, max_grad_norm=1.0, max_steps=-1, metric_for_best_model=None, mp_parameters=, no_cuda=False, num_train_epochs=1.0, optim=adamw_hf, optim_args=None, output_dir=roberta_gpt_out/roberta_custom, overwrite_output_dir=True, past_index=-1, per_device_eval_batch_size=24, per_device_train_batch_size=24, prediction_loss_only=False, push_to_hub=False, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=<PUSH_TO_HUB_TOKEN>, ray_scope=last, remove_unused_columns=True, report_to=['tensorboard'], resume_from_checkpoint=None, run_name=roberta_gpt_out/roberta_custom, save_on_each_node=False, save_steps=500, save_strategy=steps, save_total_limit=None, seed=42, sharded_ddp=[], skip_memory_metrics=True, tf32=None, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, torchdynamo=None, tpu_metrics_debug=False, tpu_num_cores=None, use_ipex=False, use_legacy_prediction_loop=False, use_mps_device=False, warmup_ratio=0.0, warmup_steps=0, weight_decay=0.0, xpu_backend=None, ) INFO:__main__:load a local file for train: data/train.json INFO:__main__:load a local file for validation: data/valid.json INFO:__main__:load a local file for test: data/test.json WARNING:datasets.builder:Using custom data configuration default-2fc7d0d25bce81a9 INFO:datasets.info:Loading Dataset Infos from /usr/local/lib/python3.8/dist-packages/datasets/packaged_modules/json INFO:datasets.builder:Generating dataset json (/content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51) Downloading and preparing dataset json/default to /content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51... Downloading data files: 100% 3/3 [00:00<00:00, 13706.88it/s] INFO:datasets.download.download_manager:Downloading took 0.0 min INFO:datasets.download.download_manager:Checksum Computation took 0.0 min Extracting data files: 100% 3/3 [00:00<00:00, 2053.35it/s] INFO:datasets.utils.info_utils:Unable to verify checksums. INFO:datasets.builder:Generating train split INFO:datasets.builder:Generating validation split INFO:datasets.builder:Generating test split INFO:datasets.utils.info_utils:Unable to verify splits sizes. Dataset json downloaded and prepared to /content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51. Subsequent calls will reuse this data. 100% 3/3 [00:00<00:00, 870.49it/s] Downloading (…)lve/main/config.json: 100% 481/481 [00:00<00:00, 84.7kB/s] [INFO|configuration_utils.py:660] 2023-02-15 16:41:53,771 >> loading configuration file config.json from cache at roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/config.json [INFO|configuration_utils.py:712] 2023-02-15 16:41:53,772 >> Model config RobertaConfig { "_name_or_path": "roberta-base", "architectures": [ "RobertaForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "classifier_dropout": null, "eos_token_id": 2, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "id2label": { "0": "LABEL_0", "1": "LABEL_1", "2": "LABEL_2", "3": "LABEL_3", "4": "LABEL_4", "5": "LABEL_5" }, "initializer_range": 0.02, "intermediate_size": 3072, "label2id": { "LABEL_0": 0, "LABEL_1": 1, "LABEL_2": 2, "LABEL_3": 3, "LABEL_4": 4, "LABEL_5": 5 }, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "roberta", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 1, "position_embedding_type": "absolute", "transformers_version": "4.26.1", "type_vocab_size": 1, "use_cache": true, "vocab_size": 50265 } [INFO|tokenization_auto.py:458] 2023-02-15 16:41:54,033 >> Could not locate the tokenizer configuration file, will try to use the model config instead. [INFO|configuration_utils.py:660] 2023-02-15 16:41:54,291 >> loading configuration file config.json from cache at roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/config.json [INFO|configuration_utils.py:712] 2023-02-15 16:41:54,292 >> Model config RobertaConfig { "_name_or_path": "roberta-base", "architectures": [ "RobertaForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "classifier_dropout": null, "eos_token_id": 2, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "roberta", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 1, "position_embedding_type": "absolute", "transformers_version": "4.26.1", "type_vocab_size": 1, "use_cache": true, "vocab_size": 50265 } Downloading (…)olve/main/vocab.json: 100% 899k/899k [00:00<00:00, 2.47MB/s] Downloading (…)olve/main/merges.txt: 100% 456k/456k [00:00<00:00, 1.51MB/s] Downloading (…)/main/tokenizer.json: 100% 1.36M/1.36M [00:00<00:00, 3.68MB/s] [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:41:57,754 >> loading file vocab.json from cache at roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/vocab.json [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:41:57,754 >> loading file merges.txt from cache at roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/merges.txt [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:41:57,754 >> loading file tokenizer.json from cache at roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/tokenizer.json [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:41:57,754 >> loading file added_tokens.json from cache at None [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:41:57,755 >> loading file special_tokens_map.json from cache at None [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:41:57,755 >> loading file tokenizer_config.json from cache at None [INFO|configuration_utils.py:660] 2023-02-15 16:41:57,755 >> loading configuration file config.json from cache at roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/config.json [INFO|configuration_utils.py:712] 2023-02-15 16:41:57,755 >> Model config RobertaConfig { "_name_or_path": "roberta-base", "architectures": [ "RobertaForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "bos_token_id": 0, "classifier_dropout": null, "eos_token_id": 2, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-05, "max_position_embeddings": 514, "model_type": "roberta", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 1, "position_embedding_type": "absolute", "transformers_version": "4.26.1", "type_vocab_size": 1, "use_cache": true, "vocab_size": 50265 } INFO:__main__:Using hidden states in model: False INFO:__main__:Using implementation from class: RobertaForSequenceClassificationCustomFIX Downloading (…)"pytorch_model.bin";: 100% 501M/501M [00:05<00:00, 89.8MB/s] [INFO|modeling_utils.py:2275] 2023-02-15 16:42:03,740 >> loading weights file pytorch_model.bin from cache at roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/pytorch_model.bin [WARNING|modeling_utils.py:2847] 2023-02-15 16:42:07,510 >> Some weights of the model checkpoint at roberta-base were not used when initializing RobertaForSequenceClassificationCustomFIX: ['lm_head.decoder.weight', 'roberta.pooler.dense.weight', 'lm_head.dense.bias', 'roberta.pooler.dense.bias', 'lm_head.layer_norm.bias', 'lm_head.bias', 'lm_head.dense.weight', 'lm_head.layer_norm.weight'] - This IS expected if you are initializing RobertaForSequenceClassificationCustomFIX from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS NOT expected if you are initializing RobertaForSequenceClassificationCustomFIX from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). [WARNING|modeling_utils.py:2859] 2023-02-15 16:42:07,510 >> Some weights of RobertaForSequenceClassificationCustomFIX were not initialized from the model checkpoint at roberta-base and are newly initialized: ['classifier.dense_4.weight', 'classifier.out_proj.bias', 'classifier.dense_1_input.bias', 'classifier.dense_3.bias', 'classifier.dense_4.bias', 'classifier.dense_3.weight', 'classifier.dense_1_hidden.bias', 'classifier.dense_1_input.weight', 'classifier.dense_2.weight', 'classifier.out_proj.weight', 'classifier.dense_2.bias', 'classifier.dense_1_hidden.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Frozen layers: [('roberta.encoder.layer.0.attention.self.query.weight', False), ('roberta.encoder.layer.0.attention.self.query.bias', False), ('roberta.encoder.layer.0.attention.self.key.weight', False), ('roberta.encoder.layer.0.attention.self.key.bias', False), ('roberta.encoder.layer.0.attention.self.value.weight', False), ('roberta.encoder.layer.0.attention.self.value.bias', False), ('roberta.encoder.layer.0.attention.output.dense.weight', False), ('roberta.encoder.layer.0.attention.output.dense.bias', False), ('roberta.encoder.layer.0.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.0.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.0.intermediate.dense.weight', False), ('roberta.encoder.layer.0.intermediate.dense.bias', False), ('roberta.encoder.layer.0.output.dense.weight', False), ('roberta.encoder.layer.0.output.dense.bias', False), ('roberta.encoder.layer.0.output.LayerNorm.weight', False), ('roberta.encoder.layer.0.output.LayerNorm.bias', False), ('roberta.encoder.layer.2.attention.self.query.weight', False), ('roberta.encoder.layer.2.attention.self.query.bias', False), ('roberta.encoder.layer.2.attention.self.key.weight', False), ('roberta.encoder.layer.2.attention.self.key.bias', False), ('roberta.encoder.layer.2.attention.self.value.weight', False), ('roberta.encoder.layer.2.attention.self.value.bias', False), ('roberta.encoder.layer.2.attention.output.dense.weight', False), ('roberta.encoder.layer.2.attention.output.dense.bias', False), ('roberta.encoder.layer.2.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.2.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.2.intermediate.dense.weight', False), ('roberta.encoder.layer.2.intermediate.dense.bias', False), ('roberta.encoder.layer.2.output.dense.weight', False), ('roberta.encoder.layer.2.output.dense.bias', False), ('roberta.encoder.layer.2.output.LayerNorm.weight', False), ('roberta.encoder.layer.2.output.LayerNorm.bias', False), ('roberta.encoder.layer.4.attention.self.query.weight', False), ('roberta.encoder.layer.4.attention.self.query.bias', False), ('roberta.encoder.layer.4.attention.self.key.weight', False), ('roberta.encoder.layer.4.attention.self.key.bias', False), ('roberta.encoder.layer.4.attention.self.value.weight', False), ('roberta.encoder.layer.4.attention.self.value.bias', False), ('roberta.encoder.layer.4.attention.output.dense.weight', False), ('roberta.encoder.layer.4.attention.output.dense.bias', False), ('roberta.encoder.layer.4.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.4.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.4.intermediate.dense.weight', False), ('roberta.encoder.layer.4.intermediate.dense.bias', False), ('roberta.encoder.layer.4.output.dense.weight', False), ('roberta.encoder.layer.4.output.dense.bias', False), ('roberta.encoder.layer.4.output.LayerNorm.weight', False), ('roberta.encoder.layer.4.output.LayerNorm.bias', False), ('roberta.encoder.layer.6.attention.self.query.weight', False), ('roberta.encoder.layer.6.attention.self.query.bias', False), ('roberta.encoder.layer.6.attention.self.key.weight', False), ('roberta.encoder.layer.6.attention.self.key.bias', False), ('roberta.encoder.layer.6.attention.self.value.weight', False), ('roberta.encoder.layer.6.attention.self.value.bias', False), ('roberta.encoder.layer.6.attention.output.dense.weight', False), ('roberta.encoder.layer.6.attention.output.dense.bias', False), ('roberta.encoder.layer.6.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.6.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.6.intermediate.dense.weight', False), ('roberta.encoder.layer.6.intermediate.dense.bias', False), ('roberta.encoder.layer.6.output.dense.weight', False), ('roberta.encoder.layer.6.output.dense.bias', False), ('roberta.encoder.layer.6.output.LayerNorm.weight', False), ('roberta.encoder.layer.6.output.LayerNorm.bias', False), ('roberta.encoder.layer.8.attention.self.query.weight', False), ('roberta.encoder.layer.8.attention.self.query.bias', False), ('roberta.encoder.layer.8.attention.self.key.weight', False), ('roberta.encoder.layer.8.attention.self.key.bias', False), ('roberta.encoder.layer.8.attention.self.value.weight', False), ('roberta.encoder.layer.8.attention.self.value.bias', False), ('roberta.encoder.layer.8.attention.output.dense.weight', False), ('roberta.encoder.layer.8.attention.output.dense.bias', False), ('roberta.encoder.layer.8.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.8.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.8.intermediate.dense.weight', False), ('roberta.encoder.layer.8.intermediate.dense.bias', False), ('roberta.encoder.layer.8.output.dense.weight', False), ('roberta.encoder.layer.8.output.dense.bias', False), ('roberta.encoder.layer.8.output.LayerNorm.weight', False), ('roberta.encoder.layer.8.output.LayerNorm.bias', False), ('roberta.encoder.layer.10.attention.self.query.weight', False), ('roberta.encoder.layer.10.attention.self.query.bias', False), ('roberta.encoder.layer.10.attention.self.key.weight', False), ('roberta.encoder.layer.10.attention.self.key.bias', False), ('roberta.encoder.layer.10.attention.self.value.weight', False), ('roberta.encoder.layer.10.attention.self.value.bias', False), ('roberta.encoder.layer.10.attention.output.dense.weight', False), ('roberta.encoder.layer.10.attention.output.dense.bias', False), ('roberta.encoder.layer.10.attention.output.LayerNorm.weight', False), ('roberta.encoder.layer.10.attention.output.LayerNorm.bias', False), ('roberta.encoder.layer.10.intermediate.dense.weight', False), ('roberta.encoder.layer.10.intermediate.dense.bias', False), ('roberta.encoder.layer.10.output.dense.weight', False), ('roberta.encoder.layer.10.output.dense.bias', False), ('roberta.encoder.layer.10.output.LayerNorm.weight', False), ('roberta.encoder.layer.10.output.LayerNorm.bias', False)] Running tokenizer on dataset: 0% 0/16 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-7f5564bde85bb8c0.arrow Running tokenizer on dataset: 100% 16/16 [00:00<00:00, 16.55ba/s] Running tokenizer on dataset: 0% 0/2 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-138d491b60344550.arrow Running tokenizer on dataset: 100% 2/2 [00:00<00:00, 20.08ba/s] Running tokenizer on dataset: 0% 0/2 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-de6cef99ad203f9d.arrow Running tokenizer on dataset: 100% 2/2 [00:00<00:00, 20.58ba/s] INFO:__main__:Sample 10476 of the training set: {'label': 0, 'text': 'i do find new friends i m going to try extra hard to make them stay and if i decide that i don t want to feel hurt again and just ride out the last year of school on my own i m going to have to try extra hard not to care what people think of me being a loner', 'input_ids': [0, 118, 109, 465, 92, 964, 939, 475, 164, 7, 860, 1823, 543, 7, 146, 106, 1095, 8, 114, 939, 2845, 14, 939, 218, 326, 236, 7, 619, 2581, 456, 8, 95, 3068, 66, 5, 94, 76, 9, 334, 15, 127, 308, 939, 475, 164, 7, 33, 7, 860, 1823, 543, 45, 7, 575, 99, 82, 206, 9, 162, 145, 10, 784, 9604, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. INFO:__main__:Sample 1824 of the training set: {'label': 1, 'text': 'i asked them to join me in creating a world where all year old girls could grow up feeling hopeful and powerful', 'input_ids': [0, 118, 553, 106, 7, 1962, 162, 11, 2351, 10, 232, 147, 70, 76, 793, 1972, 115, 1733, 62, 2157, 7917, 8, 2247, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. INFO:__main__:Sample 409 of the training set: {'label': 2, 'text': 'i feel when you are a caring person you attract other caring people into your life', 'input_ids': [0, 118, 619, 77, 47, 32, 10, 10837, 621, 47, 5696, 97, 10837, 82, 88, 110, 301, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. [INFO|trainer.py:710] 2023-02-15 16:42:10,962 >> The following columns in the training set don't have a corresponding argument in `RobertaForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `RobertaForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. /usr/local/lib/python3.8/dist-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning warnings.warn( [INFO|trainer.py:1650] 2023-02-15 16:42:10,969 >> ***** Running training ***** [INFO|trainer.py:1651] 2023-02-15 16:42:10,970 >> Num examples = 16000 [INFO|trainer.py:1652] 2023-02-15 16:42:10,970 >> Num Epochs = 1 [INFO|trainer.py:1653] 2023-02-15 16:42:10,970 >> Instantaneous batch size per device = 24 [INFO|trainer.py:1654] 2023-02-15 16:42:10,970 >> Total train batch size (w. parallel, distributed & accumulation) = 24 [INFO|trainer.py:1655] 2023-02-15 16:42:10,970 >> Gradient Accumulation steps = 1 [INFO|trainer.py:1656] 2023-02-15 16:42:10,970 >> Total optimization steps = 667 [INFO|trainer.py:1657] 2023-02-15 16:42:10,970 >> Number of trainable parameters = 187723014 {'loss': 0.7753, 'learning_rate': 5.0074962518740634e-06, 'epoch': 0.75} 75% 500/667 [01:00<00:19, 8.44it/s][INFO|trainer.py:2709] 2023-02-15 16:43:11,190 >> Saving model checkpoint to roberta_gpt_out/roberta_custom/checkpoint-500 [INFO|configuration_utils.py:453] 2023-02-15 16:43:11,191 >> Configuration saved in roberta_gpt_out/roberta_custom/checkpoint-500/config.json [INFO|modeling_utils.py:1704] 2023-02-15 16:43:12,662 >> Model weights saved in roberta_gpt_out/roberta_custom/checkpoint-500/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 16:43:12,662 >> tokenizer config file saved in roberta_gpt_out/roberta_custom/checkpoint-500/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 16:43:12,663 >> Special tokens file saved in roberta_gpt_out/roberta_custom/checkpoint-500/special_tokens_map.json 100% 666/667 [01:24<00:00, 8.45it/s][INFO|trainer.py:1901] 2023-02-15 16:43:35,735 >> Training completed. Do not forget to share your model on huggingface.co/models =) {'train_runtime': 84.7645, 'train_samples_per_second': 188.758, 'train_steps_per_second': 7.869, 'train_loss': 0.6754124654286626, 'epoch': 1.0} 100% 667/667 [01:24<00:00, 7.87it/s] [INFO|trainer.py:2709] 2023-02-15 16:43:35,737 >> Saving model checkpoint to roberta_gpt_out/roberta_custom [INFO|configuration_utils.py:453] 2023-02-15 16:43:35,738 >> Configuration saved in roberta_gpt_out/roberta_custom/config.json [INFO|modeling_utils.py:1704] 2023-02-15 16:43:37,210 >> Model weights saved in roberta_gpt_out/roberta_custom/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 16:43:37,211 >> tokenizer config file saved in roberta_gpt_out/roberta_custom/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 16:43:37,211 >> Special tokens file saved in roberta_gpt_out/roberta_custom/special_tokens_map.json ***** train metrics ***** epoch = 1.0 train_loss = 0.6754 train_runtime = 0:01:24.76 train_samples = 16000 train_samples_per_second = 188.758 train_steps_per_second = 7.869 INFO:__main__:*** Evaluate *** [INFO|trainer.py:710] 2023-02-15 16:43:37,321 >> The following columns in the evaluation set don't have a corresponding argument in `RobertaForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `RobertaForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:43:37,323 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:43:37,323 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:43:37,323 >> Batch size = 24 100% 84/84 [00:03<00:00, 23.11it/s] ***** eval metrics ***** epoch = 1.0 eval_accuracy = 0.902 eval_loss = 0.2834 eval_runtime = 0:00:03.67 eval_samples = 2000 eval_samples_per_second = 543.728 eval_steps_per_second = 22.837 INFO:__main__:*** Predict *** [INFO|trainer.py:710] 2023-02-15 16:43:41,005 >> The following columns in the test set don't have a corresponding argument in `RobertaForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `RobertaForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:43:41,006 >> ***** Running Prediction ***** [INFO|trainer.py:2966] 2023-02-15 16:43:41,006 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:43:41,006 >> Batch size = 24 100% 84/84 [00:03<00:00, 23.10it/s] INFO:__main__:***** Predict results None ***** [INFO|modelcard.py:449] 2023-02-15 16:43:44,966 >> Dropping the following result as it does not have all the necessary fields: {'task': {'name': 'Text Classification', 'type': 'text-classification'}, 'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.9020000100135803}]}
GPT2
- full data
- model
GPT2
- sequnece length: 128
- training epoch: 1
!python run_glue.py \
--cache_dir cache/gpt2 \
--model_name_or_path gpt2 \
--train_file data/train.json \
--validation_file data/valid.json \
--test_file data/test.json \
--per_device_train_batch_size 24 \
--per_device_eval_batch_size 24 \
--do_train \
--do_eval \
--do_predict \
--max_seq_length 128 \
--learning_rate 2e-5 \
--num_train_epochs 1 \
--output_dir out/gpt2 \
--overwrite_output_dir \
--eval_steps 250 \
--evaluation_strategy steps \
--metric_for_best_model accuracy \
--logging_steps 100 \
--save_total_limit 5 \
--max_steps 2500 \
--load_best_model_at_end True
2023-02-14 23:07:08.570069: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-02-14 23:07:08.727449: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2023-02-14 23:07:09.520905: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-02-14 23:07:09.521007: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-02-14 23:07:09.521044: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. WARNING:__main__:Process rank: -1, device: cuda:0, n_gpu: 1distributed training: False, 16-bits training: False INFO:__main__:Training/evaluation parameters TrainingArguments( _n_gpu=1, adafactor=False, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, auto_find_batch_size=False, bf16=False, bf16_full_eval=False, data_seed=None, dataloader_drop_last=False, dataloader_num_workers=0, dataloader_pin_memory=True, ddp_bucket_cap_mb=None, ddp_find_unused_parameters=None, ddp_timeout=1800, debug=[], deepspeed=None, disable_tqdm=False, do_eval=True, do_predict=True, do_train=True, eval_accumulation_steps=None, eval_delay=0, eval_steps=250, evaluation_strategy=steps, fp16=False, fp16_backend=auto, fp16_full_eval=False, fp16_opt_level=O1, fsdp=[], fsdp_min_num_params=0, fsdp_transformer_layer_cls_to_wrap=None, full_determinism=False, gradient_accumulation_steps=1, gradient_checkpointing=False, greater_is_better=True, group_by_length=False, half_precision_backend=auto, hub_model_id=None, hub_private_repo=False, hub_strategy=every_save, hub_token=<HUB_TOKEN>, ignore_data_skip=False, include_inputs_for_metrics=False, jit_mode_eval=False, label_names=None, label_smoothing_factor=0.0, learning_rate=2e-05, length_column_name=length, load_best_model_at_end=True, local_rank=-1, log_level=passive, log_level_replica=passive, log_on_each_node=True, logging_dir=out/emotion/gpt2/runs/Feb14_23-07-11_fc0011e45a00, logging_first_step=False, logging_nan_inf_filter=True, logging_steps=100, logging_strategy=steps, lr_scheduler_type=linear, max_grad_norm=1.0, max_steps=2500, metric_for_best_model=accuracy, mp_parameters=, no_cuda=False, num_train_epochs=1.0, optim=adamw_hf, output_dir=out/emotion/gpt2, overwrite_output_dir=True, past_index=-1, per_device_eval_batch_size=24, per_device_train_batch_size=24, prediction_loss_only=False, push_to_hub=False, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=<PUSH_TO_HUB_TOKEN>, ray_scope=last, remove_unused_columns=True, report_to=['tensorboard'], resume_from_checkpoint=None, run_name=out/emotion/gpt2, save_on_each_node=False, save_steps=500, save_strategy=steps, save_total_limit=5, seed=42, sharded_ddp=[], skip_memory_metrics=True, tf32=None, torchdynamo=None, tpu_metrics_debug=False, tpu_num_cores=None, use_ipex=False, use_legacy_prediction_loop=False, use_mps_device=False, warmup_ratio=0.0, warmup_steps=0, weight_decay=0.0, xpu_backend=None, ) INFO:__main__:load a local file for train: data/train.json INFO:__main__:load a local file for validation: data/valid.json INFO:__main__:load a local file for test: data/test.json WARNING:datasets.builder:Using custom data configuration default-e1b3a7f886502404 INFO:datasets.info:Loading Dataset Infos from /usr/local/lib/python3.8/dist-packages/datasets/packaged_modules/json INFO:datasets.builder:Generating dataset json (/content/cache/gpt2/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51) Downloading and preparing dataset json/default to /content/cache/gpt2/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51... Downloading data files: 100% 3/3 [00:00<00:00, 12932.08it/s] INFO:datasets.download.download_manager:Downloading took 0.0 min INFO:datasets.download.download_manager:Checksum Computation took 0.0 min Extracting data files: 100% 3/3 [00:00<00:00, 2039.37it/s] INFO:datasets.utils.info_utils:Unable to verify checksums. INFO:datasets.builder:Generating train split INFO:datasets.builder:Generating validation split INFO:datasets.builder:Generating test split INFO:datasets.utils.info_utils:Unable to verify splits sizes. Dataset json downloaded and prepared to /content/cache/gpt2/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51. Subsequent calls will reuse this data. 100% 3/3 [00:00<00:00, 604.92it/s] Downloading (…)lve/main/config.json: 100% 665/665 [00:00<00:00, 122kB/s] [INFO|configuration_utils.py:653] 2023-02-14 23:07:12,924 >> loading configuration file config.json from cache at cache/gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/config.json [INFO|configuration_utils.py:705] 2023-02-14 23:07:12,925 >> Model config GPT2Config { "_name_or_path": "gpt2", "activation_function": "gelu_new", "architectures": [ "GPT2LMHeadModel" ], "attn_pdrop": 0.1, "bos_token_id": 50256, "embd_pdrop": 0.1, "eos_token_id": 50256, "id2label": { "0": "LABEL_0", "1": "LABEL_1", "2": "LABEL_2", "3": "LABEL_3", "4": "LABEL_4", "5": "LABEL_5" }, "initializer_range": 0.02, "label2id": { "LABEL_0": 0, "LABEL_1": 1, "LABEL_2": 2, "LABEL_3": 3, "LABEL_4": 4, "LABEL_5": 5 }, "layer_norm_epsilon": 1e-05, "model_type": "gpt2", "n_ctx": 1024, "n_embd": 768, "n_head": 12, "n_inner": null, "n_layer": 12, "n_positions": 1024, "reorder_and_upcast_attn": false, "resid_pdrop": 0.1, "scale_attn_by_inverse_layer_idx": false, "scale_attn_weights": true, "summary_activation": null, "summary_first_dropout": 0.1, "summary_proj_to_labels": true, "summary_type": "cls_index", "summary_use_proj": true, "task_specific_params": { "text-generation": { "do_sample": true, "max_length": 50 } }, "transformers_version": "4.23.1", "use_cache": true, "vocab_size": 50257 } [INFO|tokenization_auto.py:418] 2023-02-14 23:07:13,018 >> Could not locate the tokenizer configuration file, will try to use the model config instead. [INFO|configuration_utils.py:653] 2023-02-14 23:07:13,111 >> loading configuration file config.json from cache at cache/gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/config.json [INFO|configuration_utils.py:705] 2023-02-14 23:07:13,111 >> Model config GPT2Config { "_name_or_path": "gpt2", "activation_function": "gelu_new", "architectures": [ "GPT2LMHeadModel" ], "attn_pdrop": 0.1, "bos_token_id": 50256, "embd_pdrop": 0.1, "eos_token_id": 50256, "initializer_range": 0.02, "layer_norm_epsilon": 1e-05, "model_type": "gpt2", "n_ctx": 1024, "n_embd": 768, "n_head": 12, "n_inner": null, "n_layer": 12, "n_positions": 1024, "reorder_and_upcast_attn": false, "resid_pdrop": 0.1, "scale_attn_by_inverse_layer_idx": false, "scale_attn_weights": true, "summary_activation": null, "summary_first_dropout": 0.1, "summary_proj_to_labels": true, "summary_type": "cls_index", "summary_use_proj": true, "task_specific_params": { "text-generation": { "do_sample": true, "max_length": 50 } }, "transformers_version": "4.23.1", "use_cache": true, "vocab_size": 50257 } Downloading (…)olve/main/vocab.json: 100% 1.04M/1.04M [00:00<00:00, 9.08MB/s] Downloading (…)olve/main/merges.txt: 100% 456k/456k [00:00<00:00, 5.01MB/s] Downloading (…)/main/tokenizer.json: 100% 1.36M/1.36M [00:00<00:00, 12.0MB/s] [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:07:14,433 >> loading file vocab.json from cache at cache/gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/vocab.json [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:07:14,433 >> loading file merges.txt from cache at cache/gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/merges.txt [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:07:14,434 >> loading file tokenizer.json from cache at cache/gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/tokenizer.json [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:07:14,434 >> loading file added_tokens.json from cache at None [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:07:14,434 >> loading file special_tokens_map.json from cache at None [INFO|tokenization_utils_base.py:1773] 2023-02-14 23:07:14,434 >> loading file tokenizer_config.json from cache at None [INFO|configuration_utils.py:653] 2023-02-14 23:07:14,434 >> loading configuration file config.json from cache at cache/gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/config.json [INFO|configuration_utils.py:705] 2023-02-14 23:07:14,435 >> Model config GPT2Config { "_name_or_path": "gpt2", "activation_function": "gelu_new", "architectures": [ "GPT2LMHeadModel" ], "attn_pdrop": 0.1, "bos_token_id": 50256, "embd_pdrop": 0.1, "eos_token_id": 50256, "initializer_range": 0.02, "layer_norm_epsilon": 1e-05, "model_type": "gpt2", "n_ctx": 1024, "n_embd": 768, "n_head": 12, "n_inner": null, "n_layer": 12, "n_positions": 1024, "reorder_and_upcast_attn": false, "resid_pdrop": 0.1, "scale_attn_by_inverse_layer_idx": false, "scale_attn_weights": true, "summary_activation": null, "summary_first_dropout": 0.1, "summary_proj_to_labels": true, "summary_type": "cls_index", "summary_use_proj": true, "task_specific_params": { "text-generation": { "do_sample": true, "max_length": 50 } }, "transformers_version": "4.23.1", "use_cache": true, "vocab_size": 50257 } INFO:__main__:Using implementation from class: AutoModelForSequenceClassification Downloading (…)"pytorch_model.bin";: 100% 548M/548M [00:05<00:00, 107MB/s] [INFO|modeling_utils.py:2156] 2023-02-14 23:07:19,840 >> loading weights file pytorch_model.bin from cache at cache/gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/pytorch_model.bin [INFO|modeling_utils.py:2606] 2023-02-14 23:07:21,267 >> All model checkpoint weights were used when initializing GPT2ForSequenceClassification. [WARNING|modeling_utils.py:2608] 2023-02-14 23:07:21,267 >> Some weights of GPT2ForSequenceClassification were not initialized from the model checkpoint at gpt2 and are newly initialized: ['score.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. [ERROR|tokenization_utils_base.py:1019] 2023-02-14 23:07:21,277 >> Using pad_token, but it is not set yet. INFO:__main__:Set PAD token to EOS: <|endoftext|> Running tokenizer on dataset: 0% 0/16 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/cache/gpt2/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-cf43a94824d333a0.arrow Running tokenizer on dataset: 100% 16/16 [00:00<00:00, 19.87ba/s] Running tokenizer on dataset: 0% 0/2 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/cache/gpt2/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-742ee95b271da600.arrow Running tokenizer on dataset: 100% 2/2 [00:00<00:00, 20.69ba/s] Running tokenizer on dataset: 0% 0/2 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/cache/gpt2/json/default-e1b3a7f886502404/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-9e616ad6e069d412.arrow Running tokenizer on dataset: 100% 2/2 [00:00<00:00, 20.84ba/s] INFO:__main__:Sample 10476 of the training set: {'label': 0, 'text': 'i do find new friends i m going to try extra hard to make them stay and if i decide that i don t want to feel hurt again and just ride out the last year of school on my own i m going to have to try extra hard not to care what people think of me being a loner', 'input_ids': [72, 466, 1064, 649, 2460, 1312, 285, 1016, 284, 1949, 3131, 1327, 284, 787, 606, 2652, 290, 611, 1312, 5409, 326, 1312, 836, 256, 765, 284, 1254, 5938, 757, 290, 655, 6594, 503, 262, 938, 614, 286, 1524, 319, 616, 898, 1312, 285, 1016, 284, 423, 284, 1949, 3131, 1327, 407, 284, 1337, 644, 661, 892, 286, 502, 852, 257, 300, 14491, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. INFO:__main__:Sample 1824 of the training set: {'label': 1, 'text': 'i asked them to join me in creating a world where all year old girls could grow up feeling hopeful and powerful', 'input_ids': [72, 1965, 606, 284, 4654, 502, 287, 4441, 257, 995, 810, 477, 614, 1468, 4813, 714, 1663, 510, 4203, 17836, 290, 3665, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. INFO:__main__:Sample 409 of the training set: {'label': 2, 'text': 'i feel when you are a caring person you attract other caring people into your life', 'input_ids': [72, 1254, 618, 345, 389, 257, 18088, 1048, 345, 4729, 584, 18088, 661, 656, 534, 1204, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. [INFO|trainer.py:503] 2023-02-14 23:07:24,803 >> max_steps is given, it will override any value given in num_train_epochs [INFO|trainer.py:725] 2023-02-14 23:07:24,804 >> The following columns in the training set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. /usr/local/lib/python3.8/dist-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning warnings.warn( [INFO|trainer.py:1607] 2023-02-14 23:07:24,810 >> ***** Running training ***** [INFO|trainer.py:1608] 2023-02-14 23:07:24,810 >> Num examples = 16000 [INFO|trainer.py:1609] 2023-02-14 23:07:24,810 >> Num Epochs = 4 [INFO|trainer.py:1610] 2023-02-14 23:07:24,810 >> Instantaneous batch size per device = 24 [INFO|trainer.py:1611] 2023-02-14 23:07:24,810 >> Total train batch size (w. parallel, distributed & accumulation) = 24 [INFO|trainer.py:1612] 2023-02-14 23:07:24,810 >> Gradient Accumulation steps = 1 [INFO|trainer.py:1613] 2023-02-14 23:07:24,810 >> Total optimization steps = 2500 {'loss': 2.3442, 'learning_rate': 1.9200000000000003e-05, 'epoch': 0.15} {'loss': 1.3126, 'learning_rate': 1.8400000000000003e-05, 'epoch': 0.3} 10% 250/2500 [00:37<05:29, 6.83it/s][INFO|trainer.py:725] 2023-02-14 23:08:02,490 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:08:02,492 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:08:02,492 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:08:02,492 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 27.92it/s][A 8% 7/84 [00:00<00:03, 23.90it/s][A 12% 10/84 [00:00<00:03, 22.68it/s][A 15% 13/84 [00:00<00:03, 22.09it/s][A 19% 16/84 [00:00<00:03, 21.73it/s][A 23% 19/84 [00:00<00:03, 21.53it/s][A 26% 22/84 [00:00<00:02, 21.41it/s][A 30% 25/84 [00:01<00:02, 21.30it/s][A 33% 28/84 [00:01<00:02, 21.25it/s][A 37% 31/84 [00:01<00:02, 21.24it/s][A 40% 34/84 [00:01<00:02, 21.23it/s][A 44% 37/84 [00:01<00:02, 21.21it/s][A 48% 40/84 [00:01<00:02, 21.21it/s][A 51% 43/84 [00:01<00:01, 21.19it/s][A 55% 46/84 [00:02<00:01, 21.14it/s][A 58% 49/84 [00:02<00:01, 21.15it/s][A 62% 52/84 [00:02<00:01, 21.16it/s][A 65% 55/84 [00:02<00:01, 21.17it/s][A 69% 58/84 [00:02<00:01, 21.18it/s][A 73% 61/84 [00:02<00:01, 21.18it/s][A 76% 64/84 [00:02<00:00, 21.18it/s][A 80% 67/84 [00:03<00:00, 21.10it/s][A 83% 70/84 [00:03<00:00, 21.12it/s][A 87% 73/84 [00:03<00:00, 21.13it/s][A 90% 76/84 [00:03<00:00, 21.14it/s][A 94% 79/84 [00:03<00:00, 21.17it/s][A 98% 82/84 [00:03<00:00, 21.18it/s][A {'eval_loss': 0.7983964085578918, 'eval_accuracy': 0.7465000152587891, 'eval_runtime': 3.9478, 'eval_samples_per_second': 506.612, 'eval_steps_per_second': 21.278, 'epoch': 0.37} 10% 250/2500 [00:41<05:29, 6.83it/s] {'loss': 0.7216, 'learning_rate': 1.76e-05, 'epoch': 0.45} {'loss': 0.5032, 'learning_rate': 1.6800000000000002e-05, 'epoch': 0.6} {'loss': 0.3904, 'learning_rate': 1.6000000000000003e-05, 'epoch': 0.75} 20% 500/2500 [01:18<04:53, 6.82it/s][INFO|trainer.py:725] 2023-02-14 23:08:43,155 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:08:43,157 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:08:43,157 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:08:43,157 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 28.21it/s][A 8% 7/84 [00:00<00:03, 24.02it/s][A 12% 10/84 [00:00<00:03, 22.66it/s][A 15% 13/84 [00:00<00:03, 21.98it/s][A 19% 16/84 [00:00<00:03, 21.63it/s][A 23% 19/84 [00:00<00:03, 21.38it/s][A 26% 22/84 [00:00<00:02, 21.30it/s][A 30% 25/84 [00:01<00:02, 21.19it/s][A 33% 28/84 [00:01<00:02, 21.17it/s][A 37% 31/84 [00:01<00:02, 20.95it/s][A 40% 34/84 [00:01<00:02, 20.96it/s][A 44% 37/84 [00:01<00:02, 20.95it/s][A 48% 40/84 [00:01<00:02, 20.95it/s][A 51% 43/84 [00:02<00:01, 20.99it/s][A 55% 46/84 [00:02<00:01, 21.01it/s][A 58% 49/84 [00:02<00:01, 21.02it/s][A 62% 52/84 [00:02<00:01, 21.02it/s][A 65% 55/84 [00:02<00:01, 21.02it/s][A 69% 58/84 [00:02<00:01, 21.05it/s][A 73% 61/84 [00:02<00:01, 21.06it/s][A 76% 64/84 [00:02<00:00, 21.10it/s][A 80% 67/84 [00:03<00:00, 21.11it/s][A 83% 70/84 [00:03<00:00, 21.13it/s][A 87% 73/84 [00:03<00:00, 21.11it/s][A 90% 76/84 [00:03<00:00, 21.08it/s][A 94% 79/84 [00:03<00:00, 21.08it/s][A 98% 82/84 [00:03<00:00, 21.11it/s][A {'eval_loss': 0.29131895303726196, 'eval_accuracy': 0.9035000205039978, 'eval_runtime': 3.9682, 'eval_samples_per_second': 504.003, 'eval_steps_per_second': 21.168, 'epoch': 0.75} 20% 500/2500 [01:22<04:53, 6.82it/s] [A[INFO|trainer.py:2656] 2023-02-14 23:08:47,127 >> Saving model checkpoint to out/emotion/gpt2/checkpoint-500 [INFO|configuration_utils.py:447] 2023-02-14 23:08:47,128 >> Configuration saved in out/emotion/gpt2/checkpoint-500/config.json [INFO|modeling_utils.py:1624] 2023-02-14 23:08:47,844 >> Model weights saved in out/emotion/gpt2/checkpoint-500/pytorch_model.bin [INFO|tokenization_utils_base.py:2123] 2023-02-14 23:08:47,845 >> tokenizer config file saved in out/emotion/gpt2/checkpoint-500/tokenizer_config.json [INFO|tokenization_utils_base.py:2130] 2023-02-14 23:08:47,845 >> Special tokens file saved in out/emotion/gpt2/checkpoint-500/special_tokens_map.json {'loss': 0.3554, 'learning_rate': 1.5200000000000002e-05, 'epoch': 0.9} {'loss': 0.2871, 'learning_rate': 1.4400000000000001e-05, 'epoch': 1.05} 30% 750/2500 [02:01<04:16, 6.82it/s][INFO|trainer.py:725] 2023-02-14 23:09:26,109 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:09:26,111 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:09:26,111 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:09:26,111 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 28.28it/s][A 8% 7/84 [00:00<00:03, 24.09it/s][A 12% 10/84 [00:00<00:03, 22.54it/s][A 15% 13/84 [00:00<00:03, 21.98it/s][A 19% 16/84 [00:00<00:03, 21.65it/s][A 23% 19/84 [00:00<00:03, 21.44it/s][A 26% 22/84 [00:00<00:02, 21.34it/s][A 30% 25/84 [00:01<00:02, 21.27it/s][A 33% 28/84 [00:01<00:02, 21.21it/s][A 37% 31/84 [00:01<00:02, 21.18it/s][A 40% 34/84 [00:01<00:02, 21.16it/s][A 44% 37/84 [00:01<00:02, 21.13it/s][A 48% 40/84 [00:01<00:02, 21.11it/s][A 51% 43/84 [00:01<00:01, 21.10it/s][A 55% 46/84 [00:02<00:01, 21.12it/s][A 58% 49/84 [00:02<00:01, 21.11it/s][A 62% 52/84 [00:02<00:01, 21.04it/s][A 65% 55/84 [00:02<00:01, 21.06it/s][A 69% 58/84 [00:02<00:01, 21.05it/s][A 73% 61/84 [00:02<00:01, 21.06it/s][A 76% 64/84 [00:02<00:00, 21.07it/s][A 80% 67/84 [00:03<00:00, 21.10it/s][A 83% 70/84 [00:03<00:00, 21.11it/s][A 87% 73/84 [00:03<00:00, 21.13it/s][A 90% 76/84 [00:03<00:00, 21.15it/s][A 94% 79/84 [00:03<00:00, 21.12it/s][A 98% 82/84 [00:03<00:00, 21.12it/s][A {'eval_loss': 0.2168988287448883, 'eval_accuracy': 0.9235000014305115, 'eval_runtime': 3.9585, 'eval_samples_per_second': 505.237, 'eval_steps_per_second': 21.22, 'epoch': 1.12} 30% 750/2500 [02:05<04:16, 6.82it/s] {'loss': 0.2285, 'learning_rate': 1.3600000000000002e-05, 'epoch': 1.2} {'loss': 0.1888, 'learning_rate': 1.2800000000000001e-05, 'epoch': 1.35} {'loss': 0.2106, 'learning_rate': 1.2e-05, 'epoch': 1.5} 40% 1000/2500 [02:41<03:39, 6.82it/s][INFO|trainer.py:725] 2023-02-14 23:10:06,735 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:10:06,737 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:10:06,737 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:10:06,737 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 28.21it/s][A 8% 7/84 [00:00<00:03, 24.04it/s][A 12% 10/84 [00:00<00:03, 22.74it/s][A 15% 13/84 [00:00<00:03, 22.13it/s][A 19% 16/84 [00:00<00:03, 21.78it/s][A 23% 19/84 [00:00<00:03, 21.58it/s][A 26% 22/84 [00:00<00:02, 21.42it/s][A 30% 25/84 [00:01<00:02, 21.34it/s][A 33% 28/84 [00:01<00:02, 21.28it/s][A 37% 31/84 [00:01<00:02, 21.23it/s][A 40% 34/84 [00:01<00:02, 21.17it/s][A 44% 37/84 [00:01<00:02, 21.09it/s][A 48% 40/84 [00:01<00:02, 20.95it/s][A 51% 43/84 [00:01<00:01, 20.98it/s][A 55% 46/84 [00:02<00:01, 21.03it/s][A 58% 49/84 [00:02<00:01, 21.06it/s][A 62% 52/84 [00:02<00:01, 21.08it/s][A 65% 55/84 [00:02<00:01, 21.07it/s][A 69% 58/84 [00:02<00:01, 21.10it/s][A 73% 61/84 [00:02<00:01, 21.04it/s][A 76% 64/84 [00:02<00:00, 21.08it/s][A 80% 67/84 [00:03<00:00, 21.07it/s][A 83% 70/84 [00:03<00:00, 21.08it/s][A 87% 73/84 [00:03<00:00, 21.10it/s][A 90% 76/84 [00:03<00:00, 21.07it/s][A 94% 79/84 [00:03<00:00, 21.09it/s][A 98% 82/84 [00:03<00:00, 21.08it/s][A {'eval_loss': 0.19490236043930054, 'eval_accuracy': 0.9259999990463257, 'eval_runtime': 3.9583, 'eval_samples_per_second': 505.269, 'eval_steps_per_second': 21.221, 'epoch': 1.5} 40% 1000/2500 [02:45<03:39, 6.82it/s] [A[INFO|trainer.py:2656] 2023-02-14 23:10:10,696 >> Saving model checkpoint to out/emotion/gpt2/checkpoint-1000 [INFO|configuration_utils.py:447] 2023-02-14 23:10:10,697 >> Configuration saved in out/emotion/gpt2/checkpoint-1000/config.json [INFO|modeling_utils.py:1624] 2023-02-14 23:10:11,400 >> Model weights saved in out/emotion/gpt2/checkpoint-1000/pytorch_model.bin [INFO|tokenization_utils_base.py:2123] 2023-02-14 23:10:11,401 >> tokenizer config file saved in out/emotion/gpt2/checkpoint-1000/tokenizer_config.json [INFO|tokenization_utils_base.py:2130] 2023-02-14 23:10:11,401 >> Special tokens file saved in out/emotion/gpt2/checkpoint-1000/special_tokens_map.json {'loss': 0.1906, 'learning_rate': 1.1200000000000001e-05, 'epoch': 1.65} {'loss': 0.1793, 'learning_rate': 1.04e-05, 'epoch': 1.8} 50% 1250/2500 [03:24<03:03, 6.83it/s][INFO|trainer.py:725] 2023-02-14 23:10:49,631 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:10:49,633 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:10:49,633 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:10:49,633 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 27.62it/s][A 8% 7/84 [00:00<00:03, 23.92it/s][A 12% 10/84 [00:00<00:03, 22.65it/s][A 15% 13/84 [00:00<00:03, 22.07it/s][A 19% 16/84 [00:00<00:03, 21.74it/s][A 23% 19/84 [00:00<00:03, 21.56it/s][A 26% 22/84 [00:00<00:02, 21.42it/s][A 30% 25/84 [00:01<00:02, 21.33it/s][A 33% 28/84 [00:01<00:02, 21.28it/s][A 37% 31/84 [00:01<00:02, 21.24it/s][A 40% 34/84 [00:01<00:02, 21.21it/s][A 44% 37/84 [00:01<00:02, 21.19it/s][A 48% 40/84 [00:01<00:02, 21.08it/s][A 51% 43/84 [00:01<00:01, 21.11it/s][A 55% 46/84 [00:02<00:01, 21.11it/s][A 58% 49/84 [00:02<00:01, 21.08it/s][A 62% 52/84 [00:02<00:01, 21.10it/s][A 65% 55/84 [00:02<00:01, 21.12it/s][A 69% 58/84 [00:02<00:01, 21.12it/s][A 73% 61/84 [00:02<00:01, 21.13it/s][A 76% 64/84 [00:02<00:00, 21.12it/s][A 80% 67/84 [00:03<00:00, 21.13it/s][A 83% 70/84 [00:03<00:00, 21.12it/s][A 87% 73/84 [00:03<00:00, 21.12it/s][A 90% 76/84 [00:03<00:00, 21.06it/s][A 94% 79/84 [00:03<00:00, 21.05it/s][A 98% 82/84 [00:03<00:00, 21.07it/s][A {'eval_loss': 0.1607103943824768, 'eval_accuracy': 0.9319999814033508, 'eval_runtime': 3.9556, 'eval_samples_per_second': 505.611, 'eval_steps_per_second': 21.236, 'epoch': 1.87} 50% 1250/2500 [03:28<03:03, 6.83it/s] {'loss': 0.2116, 'learning_rate': 9.600000000000001e-06, 'epoch': 1.95} {'loss': 0.1536, 'learning_rate': 8.8e-06, 'epoch': 2.1} {'loss': 0.1518, 'learning_rate': 8.000000000000001e-06, 'epoch': 2.25} 60% 1500/2500 [04:05<02:26, 6.84it/s][INFO|trainer.py:725] 2023-02-14 23:11:30,201 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:11:30,203 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:11:30,203 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:11:30,203 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 28.19it/s][A 8% 7/84 [00:00<00:03, 24.04it/s][A 12% 10/84 [00:00<00:03, 22.73it/s][A 15% 13/84 [00:00<00:03, 22.14it/s][A 19% 16/84 [00:00<00:03, 21.77it/s][A 23% 19/84 [00:00<00:03, 21.58it/s][A 26% 22/84 [00:00<00:02, 21.44it/s][A 30% 25/84 [00:01<00:02, 21.35it/s][A 33% 28/84 [00:01<00:02, 21.29it/s][A 37% 31/84 [00:01<00:02, 21.26it/s][A 40% 34/84 [00:01<00:02, 21.22it/s][A 44% 37/84 [00:01<00:02, 21.22it/s][A 48% 40/84 [00:01<00:02, 21.20it/s][A 51% 43/84 [00:01<00:01, 21.21it/s][A 55% 46/84 [00:02<00:01, 21.19it/s][A 58% 49/84 [00:02<00:01, 21.09it/s][A 62% 52/84 [00:02<00:01, 21.11it/s][A 65% 55/84 [00:02<00:01, 21.14it/s][A 69% 58/84 [00:02<00:01, 21.16it/s][A 73% 61/84 [00:02<00:01, 21.17it/s][A 76% 64/84 [00:02<00:00, 21.18it/s][A 80% 67/84 [00:03<00:00, 21.19it/s][A 83% 70/84 [00:03<00:00, 21.15it/s][A 87% 73/84 [00:03<00:00, 21.14it/s][A 90% 76/84 [00:03<00:00, 21.14it/s][A 94% 79/84 [00:03<00:00, 21.16it/s][A 98% 82/84 [00:03<00:00, 21.16it/s][A {'eval_loss': 0.160899356007576, 'eval_accuracy': 0.9330000281333923, 'eval_runtime': 3.9453, 'eval_samples_per_second': 506.928, 'eval_steps_per_second': 21.291, 'epoch': 2.25} 60% 1500/2500 [04:09<02:26, 6.84it/s] [A[INFO|trainer.py:2656] 2023-02-14 23:11:34,149 >> Saving model checkpoint to out/emotion/gpt2/checkpoint-1500 [INFO|configuration_utils.py:447] 2023-02-14 23:11:34,150 >> Configuration saved in out/emotion/gpt2/checkpoint-1500/config.json [INFO|modeling_utils.py:1624] 2023-02-14 23:11:34,855 >> Model weights saved in out/emotion/gpt2/checkpoint-1500/pytorch_model.bin [INFO|tokenization_utils_base.py:2123] 2023-02-14 23:11:34,856 >> tokenizer config file saved in out/emotion/gpt2/checkpoint-1500/tokenizer_config.json [INFO|tokenization_utils_base.py:2130] 2023-02-14 23:11:34,856 >> Special tokens file saved in out/emotion/gpt2/checkpoint-1500/special_tokens_map.json {'loss': 0.157, 'learning_rate': 7.2000000000000005e-06, 'epoch': 2.4} {'loss': 0.141, 'learning_rate': 6.4000000000000006e-06, 'epoch': 2.55} 70% 1750/2500 [04:48<01:49, 6.83it/s][INFO|trainer.py:725] 2023-02-14 23:12:13,242 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:12:13,244 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:12:13,244 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:12:13,244 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 28.13it/s][A 8% 7/84 [00:00<00:03, 24.00it/s][A 12% 10/84 [00:00<00:03, 22.68it/s][A 15% 13/84 [00:00<00:03, 22.01it/s][A 19% 16/84 [00:00<00:03, 21.66it/s][A 23% 19/84 [00:00<00:03, 21.47it/s][A 26% 22/84 [00:00<00:02, 21.35it/s][A 30% 25/84 [00:01<00:02, 21.28it/s][A 33% 28/84 [00:01<00:02, 21.11it/s][A 37% 31/84 [00:01<00:02, 21.10it/s][A 40% 34/84 [00:01<00:02, 21.11it/s][A 44% 37/84 [00:01<00:02, 21.10it/s][A 48% 40/84 [00:01<00:02, 21.12it/s][A 51% 43/84 [00:01<00:01, 21.13it/s][A 55% 46/84 [00:02<00:01, 21.12it/s][A 58% 49/84 [00:02<00:01, 21.12it/s][A 62% 52/84 [00:02<00:01, 21.11it/s][A 65% 55/84 [00:02<00:01, 21.11it/s][A 69% 58/84 [00:02<00:01, 21.13it/s][A 73% 61/84 [00:02<00:01, 21.14it/s][A 76% 64/84 [00:02<00:00, 21.15it/s][A 80% 67/84 [00:03<00:00, 21.15it/s][A 83% 70/84 [00:03<00:00, 21.07it/s][A 87% 73/84 [00:03<00:00, 21.08it/s][A 90% 76/84 [00:03<00:00, 21.05it/s][A 94% 79/84 [00:03<00:00, 21.09it/s][A 98% 82/84 [00:03<00:00, 21.11it/s][A {'eval_loss': 0.15204769372940063, 'eval_accuracy': 0.9319999814033508, 'eval_runtime': 3.9593, 'eval_samples_per_second': 505.142, 'eval_steps_per_second': 21.216, 'epoch': 2.62} 70% 1750/2500 [04:52<01:49, 6.83it/s] {'loss': 0.1426, 'learning_rate': 5.600000000000001e-06, 'epoch': 2.7} {'loss': 0.1463, 'learning_rate': 4.800000000000001e-06, 'epoch': 2.85} {'loss': 0.1403, 'learning_rate': 4.000000000000001e-06, 'epoch': 3.0} 80% 2000/2500 [05:29<01:13, 6.79it/s][INFO|trainer.py:725] 2023-02-14 23:12:53,905 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:12:53,907 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:12:53,907 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:12:53,907 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 28.19it/s][A 8% 7/84 [00:00<00:03, 23.99it/s][A 12% 10/84 [00:00<00:03, 22.40it/s][A 15% 13/84 [00:00<00:03, 21.92it/s][A 19% 16/84 [00:00<00:03, 21.62it/s][A 23% 19/84 [00:00<00:03, 21.45it/s][A 26% 22/84 [00:00<00:02, 21.36it/s][A 30% 25/84 [00:01<00:02, 21.31it/s][A 33% 28/84 [00:01<00:02, 21.28it/s][A 37% 31/84 [00:01<00:02, 21.26it/s][A 40% 34/84 [00:01<00:02, 21.24it/s][A 44% 37/84 [00:01<00:02, 21.21it/s][A 48% 40/84 [00:01<00:02, 21.15it/s][A 51% 43/84 [00:01<00:01, 21.12it/s][A 55% 46/84 [00:02<00:01, 21.13it/s][A 58% 49/84 [00:02<00:01, 21.15it/s][A 62% 52/84 [00:02<00:01, 21.16it/s][A 65% 55/84 [00:02<00:01, 21.11it/s][A 69% 58/84 [00:02<00:01, 21.11it/s][A 73% 61/84 [00:02<00:01, 21.11it/s][A 76% 64/84 [00:02<00:00, 21.08it/s][A 80% 67/84 [00:03<00:00, 21.12it/s][A 83% 70/84 [00:03<00:00, 21.13it/s][A 87% 73/84 [00:03<00:00, 21.16it/s][A 90% 76/84 [00:03<00:00, 21.09it/s][A 94% 79/84 [00:03<00:00, 21.13it/s][A 98% 82/84 [00:03<00:00, 21.13it/s][A {'eval_loss': 0.14609387516975403, 'eval_accuracy': 0.9290000200271606, 'eval_runtime': 3.9539, 'eval_samples_per_second': 505.831, 'eval_steps_per_second': 21.245, 'epoch': 3.0} 80% 2000/2500 [05:33<01:13, 6.79it/s] [A[INFO|trainer.py:2656] 2023-02-14 23:12:57,862 >> Saving model checkpoint to out/emotion/gpt2/checkpoint-2000 [INFO|configuration_utils.py:447] 2023-02-14 23:12:57,863 >> Configuration saved in out/emotion/gpt2/checkpoint-2000/config.json [INFO|modeling_utils.py:1624] 2023-02-14 23:12:58,567 >> Model weights saved in out/emotion/gpt2/checkpoint-2000/pytorch_model.bin [INFO|tokenization_utils_base.py:2123] 2023-02-14 23:12:58,568 >> tokenizer config file saved in out/emotion/gpt2/checkpoint-2000/tokenizer_config.json [INFO|tokenization_utils_base.py:2130] 2023-02-14 23:12:58,568 >> Special tokens file saved in out/emotion/gpt2/checkpoint-2000/special_tokens_map.json {'loss': 0.1256, 'learning_rate': 3.2000000000000003e-06, 'epoch': 3.15} {'loss': 0.1246, 'learning_rate': 2.4000000000000003e-06, 'epoch': 3.3} 90% 2250/2500 [06:12<00:36, 6.83it/s][INFO|trainer.py:725] 2023-02-14 23:13:36,839 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:13:36,841 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:13:36,841 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:13:36,842 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 28.26it/s][A 8% 7/84 [00:00<00:03, 24.11it/s][A 12% 10/84 [00:00<00:03, 22.76it/s][A 15% 13/84 [00:00<00:03, 22.15it/s][A 19% 16/84 [00:00<00:03, 21.80it/s][A 23% 19/84 [00:00<00:03, 21.61it/s][A 26% 22/84 [00:00<00:02, 21.43it/s][A 30% 25/84 [00:01<00:02, 21.33it/s][A 33% 28/84 [00:01<00:02, 21.27it/s][A 37% 31/84 [00:01<00:02, 21.25it/s][A 40% 34/84 [00:01<00:02, 21.24it/s][A 44% 37/84 [00:01<00:02, 21.21it/s][A 48% 40/84 [00:01<00:02, 21.21it/s][A 51% 43/84 [00:01<00:01, 21.10it/s][A 55% 46/84 [00:02<00:01, 21.12it/s][A 58% 49/84 [00:02<00:01, 21.15it/s][A 62% 52/84 [00:02<00:01, 21.11it/s][A 65% 55/84 [00:02<00:01, 21.08it/s][A 69% 58/84 [00:02<00:01, 21.10it/s][A 73% 61/84 [00:02<00:01, 21.13it/s][A 76% 64/84 [00:02<00:00, 21.11it/s][A 80% 67/84 [00:03<00:00, 21.11it/s][A 83% 70/84 [00:03<00:00, 21.14it/s][A 87% 73/84 [00:03<00:00, 21.16it/s][A 90% 76/84 [00:03<00:00, 21.17it/s][A 94% 79/84 [00:03<00:00, 21.17it/s][A 98% 82/84 [00:03<00:00, 21.19it/s][A {'eval_loss': 0.15553689002990723, 'eval_accuracy': 0.9294999837875366, 'eval_runtime': 3.9468, 'eval_samples_per_second': 506.742, 'eval_steps_per_second': 21.283, 'epoch': 3.37} 90% 2250/2500 [06:15<00:36, 6.83it/s] {'loss': 0.1174, 'learning_rate': 1.6000000000000001e-06, 'epoch': 3.45} {'loss': 0.1374, 'learning_rate': 8.000000000000001e-07, 'epoch': 3.6} {'loss': 0.1207, 'learning_rate': 0.0, 'epoch': 3.75} 100% 2500/2500 [06:52<00:00, 6.78it/s][INFO|trainer.py:725] 2023-02-14 23:14:17,492 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:14:17,494 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:14:17,494 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:14:17,494 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 5% 4/84 [00:00<00:02, 28.12it/s][A 8% 7/84 [00:00<00:03, 23.96it/s][A 12% 10/84 [00:00<00:03, 22.65it/s][A 15% 13/84 [00:00<00:03, 22.05it/s][A 19% 16/84 [00:00<00:03, 21.68it/s][A 23% 19/84 [00:00<00:03, 21.31it/s][A 26% 22/84 [00:00<00:02, 21.26it/s][A 30% 25/84 [00:01<00:02, 21.23it/s][A 33% 28/84 [00:01<00:02, 21.18it/s][A 37% 31/84 [00:01<00:02, 21.16it/s][A 40% 34/84 [00:01<00:02, 21.14it/s][A 44% 37/84 [00:01<00:02, 21.14it/s][A 48% 40/84 [00:01<00:02, 21.11it/s][A 51% 43/84 [00:01<00:01, 21.13it/s][A 55% 46/84 [00:02<00:01, 21.13it/s][A 58% 49/84 [00:02<00:01, 21.15it/s][A 62% 52/84 [00:02<00:01, 21.16it/s][A 65% 55/84 [00:02<00:01, 21.16it/s][A 69% 58/84 [00:02<00:01, 21.17it/s][A 73% 61/84 [00:02<00:01, 21.18it/s][A 76% 64/84 [00:02<00:00, 21.16it/s][A 80% 67/84 [00:03<00:00, 21.18it/s][A 83% 70/84 [00:03<00:00, 21.12it/s][A 87% 73/84 [00:03<00:00, 21.13it/s][A 90% 76/84 [00:03<00:00, 21.14it/s][A 94% 79/84 [00:03<00:00, 21.13it/s][A 98% 82/84 [00:03<00:00, 21.13it/s][A {'eval_loss': 0.15162073075771332, 'eval_accuracy': 0.9309999942779541, 'eval_runtime': 3.9562, 'eval_samples_per_second': 505.538, 'eval_steps_per_second': 21.233, 'epoch': 3.75} 100% 2500/2500 [06:56<00:00, 6.78it/s] [A[INFO|trainer.py:2656] 2023-02-14 23:14:21,452 >> Saving model checkpoint to out/emotion/gpt2/checkpoint-2500 [INFO|configuration_utils.py:447] 2023-02-14 23:14:21,452 >> Configuration saved in out/emotion/gpt2/checkpoint-2500/config.json [INFO|modeling_utils.py:1624] 2023-02-14 23:14:22,156 >> Model weights saved in out/emotion/gpt2/checkpoint-2500/pytorch_model.bin [INFO|tokenization_utils_base.py:2123] 2023-02-14 23:14:22,157 >> tokenizer config file saved in out/emotion/gpt2/checkpoint-2500/tokenizer_config.json [INFO|tokenization_utils_base.py:2130] 2023-02-14 23:14:22,157 >> Special tokens file saved in out/emotion/gpt2/checkpoint-2500/special_tokens_map.json [INFO|trainer.py:1852] 2023-02-14 23:14:23,723 >> Training completed. Do not forget to share your model on huggingface.co/models =) [INFO|trainer.py:1946] 2023-02-14 23:14:23,723 >> Loading best model from out/emotion/gpt2/checkpoint-1500 (score: 0.9330000281333923). {'train_runtime': 419.344, 'train_samples_per_second': 143.081, 'train_steps_per_second': 5.962, 'train_loss': 0.351297896194458, 'epoch': 3.75} 100% 2500/2500 [06:59<00:00, 5.96it/s] [INFO|trainer.py:2656] 2023-02-14 23:14:24,156 >> Saving model checkpoint to out/emotion/gpt2 [INFO|configuration_utils.py:447] 2023-02-14 23:14:24,157 >> Configuration saved in out/emotion/gpt2/config.json [INFO|modeling_utils.py:1624] 2023-02-14 23:14:25,141 >> Model weights saved in out/emotion/gpt2/pytorch_model.bin [INFO|tokenization_utils_base.py:2123] 2023-02-14 23:14:25,142 >> tokenizer config file saved in out/emotion/gpt2/tokenizer_config.json [INFO|tokenization_utils_base.py:2130] 2023-02-14 23:14:25,142 >> Special tokens file saved in out/emotion/gpt2/special_tokens_map.json ***** train metrics ***** epoch = 3.75 train_loss = 0.3513 train_runtime = 0:06:59.34 train_samples = 16000 train_samples_per_second = 143.081 train_steps_per_second = 5.962 INFO:__main__:*** Evaluate *** [INFO|trainer.py:725] 2023-02-14 23:14:25,244 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:14:25,246 >> ***** Running Evaluation ***** [INFO|trainer.py:2909] 2023-02-14 23:14:25,246 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:14:25,246 >> Batch size = 24 100% 84/84 [00:03<00:00, 21.33it/s] ***** eval metrics ***** epoch = 3.75 eval_accuracy = 0.933 eval_loss = 0.1609 eval_runtime = 0:00:03.99 eval_samples = 2000 eval_samples_per_second = 500.658 eval_steps_per_second = 21.028 INFO:__main__:*** Predict *** [INFO|trainer.py:725] 2023-02-14 23:14:29,244 >> The following columns in the test set don't have a corresponding argument in `GPT2ForSequenceClassification.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassification.forward`, you can safely ignore this message. [INFO|trainer.py:2907] 2023-02-14 23:14:29,245 >> ***** Running Prediction ***** [INFO|trainer.py:2909] 2023-02-14 23:14:29,246 >> Num examples = 2000 [INFO|trainer.py:2912] 2023-02-14 23:14:29,246 >> Batch size = 24 100% 84/84 [00:03<00:00, 21.53it/s] INFO:__main__:***** Predict results None ***** [INFO|modelcard.py:444] 2023-02-14 23:14:33,320 >> Dropping the following result as it does not have all the necessary fields: {'task': {'name': 'Text Classification', 'type': 'text-classification'}, 'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.9330000281333923}]}
- full dataset
- custom head
!python run_glue.py \
--cache_dir roberta_gpt_cache/custom_gpt2 \
--model_name_or_path gpt2 \
--custom_model gpt2_custom \
--train_file data/train.json \
--validation_file data/valid.json \
--test_file data/test.json \
--per_device_train_batch_size 24 \
--per_device_eval_batch_size 24 \
--do_train \
--do_eval \
--do_predict \
--max_seq_length 128 \
--learning_rate 2e-5 \
--num_train_epochs 1 \
--output_dir roberta_gpt_out/gpt2_custom \
--overwrite_output_dir \
--eval_steps 250 \
--evaluation_strategy steps \
--metric_for_best_model accuracy \
--logging_steps 100 \
--save_total_limit 5 \
--max_steps 2500 \
--load_best_model_at_end True
2023-02-15 16:43:47.454540: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-02-15 16:43:47.587338: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2023-02-15 16:43:48.351405: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-02-15 16:43:48.351528: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-02-15 16:43:48.351554: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. WARNING:__main__:Process rank: -1, device: cuda:0, n_gpu: 1distributed training: False, 16-bits training: False INFO:__main__:Training/evaluation parameters TrainingArguments( _n_gpu=1, adafactor=False, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, auto_find_batch_size=False, bf16=False, bf16_full_eval=False, data_seed=None, dataloader_drop_last=False, dataloader_num_workers=0, dataloader_pin_memory=True, ddp_bucket_cap_mb=None, ddp_find_unused_parameters=None, ddp_timeout=1800, debug=[], deepspeed=None, disable_tqdm=False, do_eval=True, do_predict=True, do_train=True, eval_accumulation_steps=None, eval_delay=0, eval_steps=250, evaluation_strategy=steps, fp16=False, fp16_backend=auto, fp16_full_eval=False, fp16_opt_level=O1, fsdp=[], fsdp_min_num_params=0, fsdp_transformer_layer_cls_to_wrap=None, full_determinism=False, gradient_accumulation_steps=1, gradient_checkpointing=False, greater_is_better=True, group_by_length=False, half_precision_backend=auto, hub_model_id=None, hub_private_repo=False, hub_strategy=every_save, hub_token=<HUB_TOKEN>, ignore_data_skip=False, include_inputs_for_metrics=False, jit_mode_eval=False, label_names=None, label_smoothing_factor=0.0, learning_rate=2e-05, length_column_name=length, load_best_model_at_end=True, local_rank=-1, log_level=passive, log_level_replica=passive, log_on_each_node=True, logging_dir=roberta_gpt_out/gpt2_custom/runs/Feb15_16-43-51_b7fb20e65b38, logging_first_step=False, logging_nan_inf_filter=True, logging_steps=100, logging_strategy=steps, lr_scheduler_type=linear, max_grad_norm=1.0, max_steps=2500, metric_for_best_model=accuracy, mp_parameters=, no_cuda=False, num_train_epochs=1.0, optim=adamw_hf, optim_args=None, output_dir=roberta_gpt_out/gpt2_custom, overwrite_output_dir=True, past_index=-1, per_device_eval_batch_size=24, per_device_train_batch_size=24, prediction_loss_only=False, push_to_hub=False, push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=<PUSH_TO_HUB_TOKEN>, ray_scope=last, remove_unused_columns=True, report_to=['tensorboard'], resume_from_checkpoint=None, run_name=roberta_gpt_out/gpt2_custom, save_on_each_node=False, save_steps=500, save_strategy=steps, save_total_limit=5, seed=42, sharded_ddp=[], skip_memory_metrics=True, tf32=None, torch_compile=False, torch_compile_backend=None, torch_compile_mode=None, torchdynamo=None, tpu_metrics_debug=False, tpu_num_cores=None, use_ipex=False, use_legacy_prediction_loop=False, use_mps_device=False, warmup_ratio=0.0, warmup_steps=0, weight_decay=0.0, xpu_backend=None, ) INFO:__main__:load a local file for train: data/train.json INFO:__main__:load a local file for validation: data/valid.json INFO:__main__:load a local file for test: data/test.json WARNING:datasets.builder:Using custom data configuration default-2fc7d0d25bce81a9 INFO:datasets.info:Loading Dataset Infos from /usr/local/lib/python3.8/dist-packages/datasets/packaged_modules/json INFO:datasets.builder:Generating dataset json (/content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51) Downloading and preparing dataset json/default to /content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51... Downloading data files: 100% 3/3 [00:00<00:00, 14529.92it/s] INFO:datasets.download.download_manager:Downloading took 0.0 min INFO:datasets.download.download_manager:Checksum Computation took 0.0 min Extracting data files: 100% 3/3 [00:00<00:00, 1944.51it/s] INFO:datasets.utils.info_utils:Unable to verify checksums. INFO:datasets.builder:Generating train split INFO:datasets.builder:Generating validation split INFO:datasets.builder:Generating test split INFO:datasets.utils.info_utils:Unable to verify splits sizes. Dataset json downloaded and prepared to /content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51. Subsequent calls will reuse this data. 100% 3/3 [00:00<00:00, 936.72it/s] Downloading (…)lve/main/config.json: 100% 665/665 [00:00<00:00, 116kB/s] [INFO|configuration_utils.py:660] 2023-02-15 16:43:53,242 >> loading configuration file config.json from cache at roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/config.json [INFO|configuration_utils.py:712] 2023-02-15 16:43:53,243 >> Model config GPT2Config { "_name_or_path": "gpt2", "activation_function": "gelu_new", "architectures": [ "GPT2LMHeadModel" ], "attn_pdrop": 0.1, "bos_token_id": 50256, "embd_pdrop": 0.1, "eos_token_id": 50256, "id2label": { "0": "LABEL_0", "1": "LABEL_1", "2": "LABEL_2", "3": "LABEL_3", "4": "LABEL_4", "5": "LABEL_5" }, "initializer_range": 0.02, "label2id": { "LABEL_0": 0, "LABEL_1": 1, "LABEL_2": 2, "LABEL_3": 3, "LABEL_4": 4, "LABEL_5": 5 }, "layer_norm_epsilon": 1e-05, "model_type": "gpt2", "n_ctx": 1024, "n_embd": 768, "n_head": 12, "n_inner": null, "n_layer": 12, "n_positions": 1024, "reorder_and_upcast_attn": false, "resid_pdrop": 0.1, "scale_attn_by_inverse_layer_idx": false, "scale_attn_weights": true, "summary_activation": null, "summary_first_dropout": 0.1, "summary_proj_to_labels": true, "summary_type": "cls_index", "summary_use_proj": true, "task_specific_params": { "text-generation": { "do_sample": true, "max_length": 50 } }, "transformers_version": "4.26.1", "use_cache": true, "vocab_size": 50257 } [INFO|tokenization_auto.py:458] 2023-02-15 16:43:53,504 >> Could not locate the tokenizer configuration file, will try to use the model config instead. [INFO|configuration_utils.py:660] 2023-02-15 16:43:53,769 >> loading configuration file config.json from cache at roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/config.json [INFO|configuration_utils.py:712] 2023-02-15 16:43:53,769 >> Model config GPT2Config { "_name_or_path": "gpt2", "activation_function": "gelu_new", "architectures": [ "GPT2LMHeadModel" ], "attn_pdrop": 0.1, "bos_token_id": 50256, "embd_pdrop": 0.1, "eos_token_id": 50256, "initializer_range": 0.02, "layer_norm_epsilon": 1e-05, "model_type": "gpt2", "n_ctx": 1024, "n_embd": 768, "n_head": 12, "n_inner": null, "n_layer": 12, "n_positions": 1024, "reorder_and_upcast_attn": false, "resid_pdrop": 0.1, "scale_attn_by_inverse_layer_idx": false, "scale_attn_weights": true, "summary_activation": null, "summary_first_dropout": 0.1, "summary_proj_to_labels": true, "summary_type": "cls_index", "summary_use_proj": true, "task_specific_params": { "text-generation": { "do_sample": true, "max_length": 50 } }, "transformers_version": "4.26.1", "use_cache": true, "vocab_size": 50257 } Downloading (…)olve/main/vocab.json: 100% 1.04M/1.04M [00:00<00:00, 3.44MB/s] Downloading (…)olve/main/merges.txt: 100% 456k/456k [00:00<00:00, 1.88MB/s] Downloading (…)/main/tokenizer.json: 100% 1.36M/1.36M [00:00<00:00, 3.74MB/s] [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:43:57,245 >> loading file vocab.json from cache at roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/vocab.json [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:43:57,245 >> loading file merges.txt from cache at roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/merges.txt [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:43:57,245 >> loading file tokenizer.json from cache at roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/tokenizer.json [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:43:57,245 >> loading file added_tokens.json from cache at None [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:43:57,245 >> loading file special_tokens_map.json from cache at None [INFO|tokenization_utils_base.py:1802] 2023-02-15 16:43:57,245 >> loading file tokenizer_config.json from cache at None [INFO|configuration_utils.py:660] 2023-02-15 16:43:57,245 >> loading configuration file config.json from cache at roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/config.json [INFO|configuration_utils.py:712] 2023-02-15 16:43:57,246 >> Model config GPT2Config { "_name_or_path": "gpt2", "activation_function": "gelu_new", "architectures": [ "GPT2LMHeadModel" ], "attn_pdrop": 0.1, "bos_token_id": 50256, "embd_pdrop": 0.1, "eos_token_id": 50256, "initializer_range": 0.02, "layer_norm_epsilon": 1e-05, "model_type": "gpt2", "n_ctx": 1024, "n_embd": 768, "n_head": 12, "n_inner": null, "n_layer": 12, "n_positions": 1024, "reorder_and_upcast_attn": false, "resid_pdrop": 0.1, "scale_attn_by_inverse_layer_idx": false, "scale_attn_weights": true, "summary_activation": null, "summary_first_dropout": 0.1, "summary_proj_to_labels": true, "summary_type": "cls_index", "summary_use_proj": true, "task_specific_params": { "text-generation": { "do_sample": true, "max_length": 50 } }, "transformers_version": "4.26.1", "use_cache": true, "vocab_size": 50257 } INFO:__main__:Using hidden states in model: False INFO:__main__:Using implementation from class: GPT2ForSequenceClassificationCustomFIX Downloading (…)"pytorch_model.bin";: 100% 548M/548M [00:03<00:00, 166MB/s] [INFO|modeling_utils.py:2275] 2023-02-15 16:44:01,011 >> loading weights file pytorch_model.bin from cache at roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/pytorch_model.bin [INFO|modeling_utils.py:2857] 2023-02-15 16:44:03,582 >> All model checkpoint weights were used when initializing GPT2ForSequenceClassificationCustomFIX. [WARNING|modeling_utils.py:2859] 2023-02-15 16:44:03,582 >> Some weights of GPT2ForSequenceClassificationCustomFIX were not initialized from the model checkpoint at gpt2 and are newly initialized: ['score.dense_3_hidden.weight', 'score.dense_1_input.bias', 'score.dense_2_hidden.bias', 'score.out_proj.weight', 'score.dense_2.bias', 'score.dense_4.bias', 'score.dense_2_hidden.weight', 'score.dense_3.weight', 'score.dense_4.weight', 'score.dense_2.weight', 'score.dense_1_input.weight', 'score.dense_1_hidden.weight', 'score.dense_1_hidden.bias', 'score.dense_3_hidden.bias', 'score.dense_3.bias'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. [ERROR|tokenization_utils_base.py:1042] 2023-02-15 16:44:03,591 >> Using pad_token, but it is not set yet. INFO:__main__:Set PAD token to EOS: <|endoftext|> transformer.wte.weight transformer.wpe.weight transformer.h.0.ln_1.weight transformer.h.0.ln_1.bias transformer.h.0.attn.c_attn.weight transformer.h.0.attn.c_attn.bias transformer.h.0.attn.c_proj.weight transformer.h.0.attn.c_proj.bias transformer.h.0.ln_2.weight transformer.h.0.ln_2.bias transformer.h.0.mlp.c_fc.weight transformer.h.0.mlp.c_fc.bias transformer.h.0.mlp.c_proj.weight transformer.h.0.mlp.c_proj.bias transformer.h.1.ln_1.weight transformer.h.1.ln_1.bias transformer.h.1.attn.c_attn.weight transformer.h.1.attn.c_attn.bias transformer.h.1.attn.c_proj.weight transformer.h.1.attn.c_proj.bias transformer.h.1.ln_2.weight transformer.h.1.ln_2.bias transformer.h.1.mlp.c_fc.weight transformer.h.1.mlp.c_fc.bias transformer.h.1.mlp.c_proj.weight transformer.h.1.mlp.c_proj.bias transformer.h.2.ln_1.weight transformer.h.2.ln_1.bias transformer.h.2.attn.c_attn.weight transformer.h.2.attn.c_attn.bias transformer.h.2.attn.c_proj.weight transformer.h.2.attn.c_proj.bias transformer.h.2.ln_2.weight transformer.h.2.ln_2.bias transformer.h.2.mlp.c_fc.weight transformer.h.2.mlp.c_fc.bias transformer.h.2.mlp.c_proj.weight transformer.h.2.mlp.c_proj.bias transformer.h.3.ln_1.weight transformer.h.3.ln_1.bias transformer.h.3.attn.c_attn.weight transformer.h.3.attn.c_attn.bias transformer.h.3.attn.c_proj.weight transformer.h.3.attn.c_proj.bias transformer.h.3.ln_2.weight transformer.h.3.ln_2.bias transformer.h.3.mlp.c_fc.weight transformer.h.3.mlp.c_fc.bias transformer.h.3.mlp.c_proj.weight transformer.h.3.mlp.c_proj.bias transformer.h.4.ln_1.weight transformer.h.4.ln_1.bias transformer.h.4.attn.c_attn.weight transformer.h.4.attn.c_attn.bias transformer.h.4.attn.c_proj.weight transformer.h.4.attn.c_proj.bias transformer.h.4.ln_2.weight transformer.h.4.ln_2.bias transformer.h.4.mlp.c_fc.weight transformer.h.4.mlp.c_fc.bias transformer.h.4.mlp.c_proj.weight transformer.h.4.mlp.c_proj.bias transformer.h.5.ln_1.weight transformer.h.5.ln_1.bias transformer.h.5.attn.c_attn.weight transformer.h.5.attn.c_attn.bias transformer.h.5.attn.c_proj.weight transformer.h.5.attn.c_proj.bias transformer.h.5.ln_2.weight transformer.h.5.ln_2.bias transformer.h.5.mlp.c_fc.weight transformer.h.5.mlp.c_fc.bias transformer.h.5.mlp.c_proj.weight transformer.h.5.mlp.c_proj.bias transformer.h.6.ln_1.weight transformer.h.6.ln_1.bias transformer.h.6.attn.c_attn.weight transformer.h.6.attn.c_attn.bias transformer.h.6.attn.c_proj.weight transformer.h.6.attn.c_proj.bias transformer.h.6.ln_2.weight transformer.h.6.ln_2.bias transformer.h.6.mlp.c_fc.weight transformer.h.6.mlp.c_fc.bias transformer.h.6.mlp.c_proj.weight transformer.h.6.mlp.c_proj.bias transformer.h.7.ln_1.weight transformer.h.7.ln_1.bias transformer.h.7.attn.c_attn.weight transformer.h.7.attn.c_attn.bias transformer.h.7.attn.c_proj.weight transformer.h.7.attn.c_proj.bias transformer.h.7.ln_2.weight transformer.h.7.ln_2.bias transformer.h.7.mlp.c_fc.weight transformer.h.7.mlp.c_fc.bias transformer.h.7.mlp.c_proj.weight transformer.h.7.mlp.c_proj.bias transformer.h.8.ln_1.weight transformer.h.8.ln_1.bias transformer.h.8.attn.c_attn.weight transformer.h.8.attn.c_attn.bias transformer.h.8.attn.c_proj.weight transformer.h.8.attn.c_proj.bias transformer.h.8.ln_2.weight transformer.h.8.ln_2.bias transformer.h.8.mlp.c_fc.weight transformer.h.8.mlp.c_fc.bias transformer.h.8.mlp.c_proj.weight transformer.h.8.mlp.c_proj.bias transformer.h.9.ln_1.weight transformer.h.9.ln_1.bias transformer.h.9.attn.c_attn.weight transformer.h.9.attn.c_attn.bias transformer.h.9.attn.c_proj.weight transformer.h.9.attn.c_proj.bias transformer.h.9.ln_2.weight transformer.h.9.ln_2.bias transformer.h.9.mlp.c_fc.weight transformer.h.9.mlp.c_fc.bias transformer.h.9.mlp.c_proj.weight transformer.h.9.mlp.c_proj.bias transformer.h.10.ln_1.weight transformer.h.10.ln_1.bias transformer.h.10.attn.c_attn.weight transformer.h.10.attn.c_attn.bias transformer.h.10.attn.c_proj.weight transformer.h.10.attn.c_proj.bias transformer.h.10.ln_2.weight transformer.h.10.ln_2.bias transformer.h.10.mlp.c_fc.weight transformer.h.10.mlp.c_fc.bias transformer.h.10.mlp.c_proj.weight transformer.h.10.mlp.c_proj.bias transformer.h.11.ln_1.weight transformer.h.11.ln_1.bias transformer.h.11.attn.c_attn.weight transformer.h.11.attn.c_attn.bias transformer.h.11.attn.c_proj.weight transformer.h.11.attn.c_proj.bias transformer.h.11.ln_2.weight transformer.h.11.ln_2.bias transformer.h.11.mlp.c_fc.weight transformer.h.11.mlp.c_fc.bias transformer.h.11.mlp.c_proj.weight transformer.h.11.mlp.c_proj.bias transformer.ln_f.weight transformer.ln_f.bias score.dense_1_input.weight score.dense_1_input.bias score.dense_1_hidden.weight score.dense_1_hidden.bias score.dense_2.weight score.dense_2.bias score.dense_2_hidden.weight score.dense_2_hidden.bias score.dense_3.weight score.dense_3.bias score.dense_3_hidden.weight score.dense_3_hidden.bias score.dense_4.weight score.dense_4.bias score.out_proj.weight Running tokenizer on dataset: 0% 0/16 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-bea742f4a59644e1.arrow Running tokenizer on dataset: 100% 16/16 [00:00<00:00, 16.16ba/s] Running tokenizer on dataset: 0% 0/2 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-70f9aa9f07dd2d49.arrow Running tokenizer on dataset: 100% 2/2 [00:00<00:00, 18.31ba/s] Running tokenizer on dataset: 0% 0/2 [00:00<?, ?ba/s]INFO:datasets.arrow_dataset:Caching processed dataset at /content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-d96eda4fa0456cad.arrow Running tokenizer on dataset: 100% 2/2 [00:00<00:00, 19.03ba/s] INFO:__main__:Sample 10476 of the training set: {'label': 0, 'text': 'i do find new friends i m going to try extra hard to make them stay and if i decide that i don t want to feel hurt again and just ride out the last year of school on my own i m going to have to try extra hard not to care what people think of me being a loner', 'input_ids': [72, 466, 1064, 649, 2460, 1312, 285, 1016, 284, 1949, 3131, 1327, 284, 787, 606, 2652, 290, 611, 1312, 5409, 326, 1312, 836, 256, 765, 284, 1254, 5938, 757, 290, 655, 6594, 503, 262, 938, 614, 286, 1524, 319, 616, 898, 1312, 285, 1016, 284, 423, 284, 1949, 3131, 1327, 407, 284, 1337, 644, 661, 892, 286, 502, 852, 257, 300, 14491, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. INFO:__main__:Sample 1824 of the training set: {'label': 1, 'text': 'i asked them to join me in creating a world where all year old girls could grow up feeling hopeful and powerful', 'input_ids': [72, 1965, 606, 284, 4654, 502, 287, 4441, 257, 995, 810, 477, 614, 1468, 4813, 714, 1663, 510, 4203, 17836, 290, 3665, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. INFO:__main__:Sample 409 of the training set: {'label': 2, 'text': 'i feel when you are a caring person you attract other caring people into your life', 'input_ids': [72, 1254, 618, 345, 389, 257, 18088, 1048, 345, 4729, 584, 18088, 661, 656, 534, 1204, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}. [INFO|trainer.py:511] 2023-02-15 16:44:06,852 >> max_steps is given, it will override any value given in num_train_epochs [INFO|trainer.py:710] 2023-02-15 16:44:06,852 >> The following columns in the training set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. /usr/local/lib/python3.8/dist-packages/transformers/optimization.py:306: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning warnings.warn( [INFO|trainer.py:1650] 2023-02-15 16:44:06,859 >> ***** Running training ***** [INFO|trainer.py:1651] 2023-02-15 16:44:06,859 >> Num examples = 16000 [INFO|trainer.py:1652] 2023-02-15 16:44:06,859 >> Num Epochs = 4 [INFO|trainer.py:1653] 2023-02-15 16:44:06,859 >> Instantaneous batch size per device = 24 [INFO|trainer.py:1654] 2023-02-15 16:44:06,859 >> Total train batch size (w. parallel, distributed & accumulation) = 24 [INFO|trainer.py:1655] 2023-02-15 16:44:06,859 >> Gradient Accumulation steps = 1 [INFO|trainer.py:1656] 2023-02-15 16:44:06,859 >> Total optimization steps = 2500 [INFO|trainer.py:1657] 2023-02-15 16:44:06,860 >> Number of trainable parameters = 135071232 {'loss': 1.6316, 'learning_rate': 1.9200000000000003e-05, 'epoch': 0.15} {'loss': 1.4361, 'learning_rate': 1.8400000000000003e-05, 'epoch': 0.3} 10% 250/2500 [00:43<06:23, 5.87it/s][INFO|trainer.py:710] 2023-02-15 16:44:50,580 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:44:50,582 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:44:50,582 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:44:50,582 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.51it/s][A 7% 6/84 [00:00<00:03, 21.26it/s][A 11% 9/84 [00:00<00:03, 19.78it/s][A 14% 12/84 [00:00<00:03, 19.17it/s][A 17% 14/84 [00:00<00:03, 18.92it/s][A 19% 16/84 [00:00<00:03, 18.74it/s][A 21% 18/84 [00:00<00:03, 18.61it/s][A 24% 20/84 [00:01<00:03, 18.52it/s][A 26% 22/84 [00:01<00:03, 18.47it/s][A 29% 24/84 [00:01<00:03, 18.43it/s][A 31% 26/84 [00:01<00:03, 18.35it/s][A 33% 28/84 [00:01<00:03, 18.34it/s][A 36% 30/84 [00:01<00:02, 18.34it/s][A 38% 32/84 [00:01<00:02, 18.34it/s][A 40% 34/84 [00:01<00:02, 18.34it/s][A 43% 36/84 [00:01<00:02, 18.32it/s][A 45% 38/84 [00:02<00:02, 18.32it/s][A 48% 40/84 [00:02<00:02, 18.29it/s][A 50% 42/84 [00:02<00:02, 18.29it/s][A 52% 44/84 [00:02<00:02, 18.29it/s][A 55% 46/84 [00:02<00:02, 18.26it/s][A 57% 48/84 [00:02<00:02, 17.77it/s][A 60% 50/84 [00:02<00:01, 17.92it/s][A 62% 52/84 [00:02<00:01, 18.04it/s][A 64% 54/84 [00:02<00:01, 18.13it/s][A 67% 56/84 [00:03<00:01, 18.19it/s][A 69% 58/84 [00:03<00:01, 18.23it/s][A 71% 60/84 [00:03<00:01, 18.21it/s][A 74% 62/84 [00:03<00:01, 18.20it/s][A 76% 64/84 [00:03<00:01, 18.24it/s][A 79% 66/84 [00:03<00:00, 18.20it/s][A 81% 68/84 [00:03<00:00, 18.23it/s][A 83% 70/84 [00:03<00:00, 18.26it/s][A 86% 72/84 [00:03<00:00, 18.24it/s][A 88% 74/84 [00:04<00:00, 18.27it/s][A 90% 76/84 [00:04<00:00, 18.28it/s][A 93% 78/84 [00:04<00:00, 18.22it/s][A 95% 80/84 [00:04<00:00, 18.25it/s][A 98% 82/84 [00:04<00:00, 18.26it/s][A {'eval_loss': 0.8915246725082397, 'eval_accuracy': 0.671999990940094, 'eval_runtime': 4.5739, 'eval_samples_per_second': 437.261, 'eval_steps_per_second': 18.365, 'epoch': 0.37} 10% 250/2500 [00:48<06:23, 5.87it/s] {'loss': 0.9392, 'learning_rate': 1.76e-05, 'epoch': 0.45} {'loss': 0.6756, 'learning_rate': 1.6800000000000002e-05, 'epoch': 0.6} {'loss': 0.4982, 'learning_rate': 1.6000000000000003e-05, 'epoch': 0.75} 20% 500/2500 [01:30<05:40, 5.88it/s][INFO|trainer.py:710] 2023-02-15 16:45:37,856 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:45:37,857 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:45:37,858 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:45:37,858 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.34it/s][A 7% 6/84 [00:00<00:03, 21.07it/s][A 11% 9/84 [00:00<00:03, 19.58it/s][A 14% 12/84 [00:00<00:03, 19.09it/s][A 17% 14/84 [00:00<00:03, 18.89it/s][A 19% 16/84 [00:00<00:03, 18.74it/s][A 21% 18/84 [00:00<00:03, 18.57it/s][A 24% 20/84 [00:01<00:03, 18.51it/s][A 26% 22/84 [00:01<00:03, 18.46it/s][A 29% 24/84 [00:01<00:03, 18.40it/s][A 31% 26/84 [00:01<00:03, 18.39it/s][A 33% 28/84 [00:01<00:03, 18.38it/s][A 36% 30/84 [00:01<00:02, 18.38it/s][A 38% 32/84 [00:01<00:02, 18.38it/s][A 40% 34/84 [00:01<00:02, 18.38it/s][A 43% 36/84 [00:01<00:02, 18.38it/s][A 45% 38/84 [00:02<00:02, 18.36it/s][A 48% 40/84 [00:02<00:02, 18.27it/s][A 50% 42/84 [00:02<00:02, 18.19it/s][A 52% 44/84 [00:02<00:02, 17.80it/s][A 55% 46/84 [00:02<00:02, 17.94it/s][A 57% 48/84 [00:02<00:01, 18.07it/s][A 60% 50/84 [00:02<00:01, 18.13it/s][A 62% 52/84 [00:02<00:01, 18.20it/s][A 64% 54/84 [00:02<00:01, 18.23it/s][A 67% 56/84 [00:03<00:01, 18.24it/s][A 69% 58/84 [00:03<00:01, 18.25it/s][A 71% 60/84 [00:03<00:01, 18.26it/s][A 74% 62/84 [00:03<00:01, 18.27it/s][A 76% 64/84 [00:03<00:01, 18.29it/s][A 79% 66/84 [00:03<00:00, 18.32it/s][A 81% 68/84 [00:03<00:00, 18.33it/s][A 83% 70/84 [00:03<00:00, 18.34it/s][A 86% 72/84 [00:03<00:00, 18.33it/s][A 88% 74/84 [00:03<00:00, 18.32it/s][A 90% 76/84 [00:04<00:00, 18.30it/s][A 93% 78/84 [00:04<00:00, 18.30it/s][A 95% 80/84 [00:04<00:00, 18.31it/s][A 98% 82/84 [00:04<00:00, 18.33it/s][A {'eval_loss': 0.37129098176956177, 'eval_accuracy': 0.8725000023841858, 'eval_runtime': 4.5697, 'eval_samples_per_second': 437.663, 'eval_steps_per_second': 18.382, 'epoch': 0.75} 20% 500/2500 [01:35<05:40, 5.88it/s] [A[INFO|trainer.py:2709] 2023-02-15 16:45:42,428 >> Saving model checkpoint to roberta_gpt_out/gpt2_custom/checkpoint-500 [INFO|configuration_utils.py:453] 2023-02-15 16:45:42,429 >> Configuration saved in roberta_gpt_out/gpt2_custom/checkpoint-500/config.json [INFO|modeling_utils.py:1704] 2023-02-15 16:45:43,202 >> Model weights saved in roberta_gpt_out/gpt2_custom/checkpoint-500/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 16:45:43,202 >> tokenizer config file saved in roberta_gpt_out/gpt2_custom/checkpoint-500/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 16:45:43,203 >> Special tokens file saved in roberta_gpt_out/gpt2_custom/checkpoint-500/special_tokens_map.json {'loss': 0.4123, 'learning_rate': 1.5200000000000002e-05, 'epoch': 0.9} {'loss': 0.3322, 'learning_rate': 1.4400000000000001e-05, 'epoch': 1.05} 30% 750/2500 [02:20<04:58, 5.87it/s][INFO|trainer.py:710] 2023-02-15 16:46:27,753 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:46:27,754 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:46:27,754 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:46:27,755 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.45it/s][A 7% 6/84 [00:00<00:03, 21.13it/s][A 11% 9/84 [00:00<00:03, 19.76it/s][A 14% 12/84 [00:00<00:03, 19.18it/s][A 17% 14/84 [00:00<00:03, 18.93it/s][A 19% 16/84 [00:00<00:03, 18.75it/s][A 21% 18/84 [00:00<00:03, 18.63it/s][A 24% 20/84 [00:01<00:03, 18.54it/s][A 26% 22/84 [00:01<00:03, 18.48it/s][A 29% 24/84 [00:01<00:03, 18.43it/s][A 31% 26/84 [00:01<00:03, 18.39it/s][A 33% 28/84 [00:01<00:03, 18.37it/s][A 36% 30/84 [00:01<00:02, 18.33it/s][A 38% 32/84 [00:01<00:02, 18.30it/s][A 40% 34/84 [00:01<00:02, 18.30it/s][A 43% 36/84 [00:01<00:02, 18.29it/s][A 45% 38/84 [00:02<00:02, 18.30it/s][A 48% 40/84 [00:02<00:02, 18.28it/s][A 50% 42/84 [00:02<00:02, 18.21it/s][A 52% 44/84 [00:02<00:02, 18.23it/s][A 55% 46/84 [00:02<00:02, 18.26it/s][A 57% 48/84 [00:02<00:01, 18.28it/s][A 60% 50/84 [00:02<00:01, 18.28it/s][A 62% 52/84 [00:02<00:01, 18.26it/s][A 64% 54/84 [00:02<00:01, 18.29it/s][A 67% 56/84 [00:03<00:01, 18.30it/s][A 69% 58/84 [00:03<00:01, 18.25it/s][A 71% 60/84 [00:03<00:01, 18.27it/s][A 74% 62/84 [00:03<00:01, 18.30it/s][A 76% 64/84 [00:03<00:01, 18.30it/s][A 79% 66/84 [00:03<00:00, 18.32it/s][A 81% 68/84 [00:03<00:00, 18.33it/s][A 83% 70/84 [00:03<00:00, 18.32it/s][A 86% 72/84 [00:03<00:00, 18.28it/s][A 88% 74/84 [00:03<00:00, 18.29it/s][A 90% 76/84 [00:04<00:00, 18.31it/s][A 93% 78/84 [00:04<00:00, 18.29it/s][A 95% 80/84 [00:04<00:00, 18.28it/s][A 98% 82/84 [00:04<00:00, 18.30it/s][A {'eval_loss': 0.24756397306919098, 'eval_accuracy': 0.9194999933242798, 'eval_runtime': 4.5615, 'eval_samples_per_second': 438.453, 'eval_steps_per_second': 18.415, 'epoch': 1.12} 30% 750/2500 [02:25<04:58, 5.87it/s] {'loss': 0.2514, 'learning_rate': 1.3600000000000002e-05, 'epoch': 1.2} {'loss': 0.2498, 'learning_rate': 1.2800000000000001e-05, 'epoch': 1.35} {'loss': 0.2478, 'learning_rate': 1.2e-05, 'epoch': 1.5} 40% 1000/2500 [03:08<04:15, 5.86it/s][INFO|trainer.py:710] 2023-02-15 16:47:15,045 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:47:15,047 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:47:15,047 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:47:15,047 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.55it/s][A 7% 6/84 [00:00<00:03, 21.29it/s][A 11% 9/84 [00:00<00:03, 19.85it/s][A 14% 12/84 [00:00<00:03, 19.24it/s][A 17% 14/84 [00:00<00:03, 18.98it/s][A 19% 16/84 [00:00<00:03, 18.80it/s][A 21% 18/84 [00:00<00:03, 18.64it/s][A 24% 20/84 [00:01<00:03, 18.52it/s][A 26% 22/84 [00:01<00:03, 18.47it/s][A 29% 24/84 [00:01<00:03, 18.44it/s][A 31% 26/84 [00:01<00:03, 18.42it/s][A 33% 28/84 [00:01<00:03, 17.97it/s][A 36% 30/84 [00:01<00:02, 18.08it/s][A 38% 32/84 [00:01<00:02, 18.14it/s][A 40% 34/84 [00:01<00:02, 18.21it/s][A 43% 36/84 [00:01<00:02, 18.22it/s][A 45% 38/84 [00:02<00:02, 18.25it/s][A 48% 40/84 [00:02<00:02, 18.26it/s][A 50% 42/84 [00:02<00:02, 18.28it/s][A 52% 44/84 [00:02<00:02, 18.31it/s][A 55% 46/84 [00:02<00:02, 18.33it/s][A 57% 48/84 [00:02<00:01, 18.34it/s][A 60% 50/84 [00:02<00:01, 18.34it/s][A 62% 52/84 [00:02<00:01, 18.35it/s][A 64% 54/84 [00:02<00:01, 18.34it/s][A 67% 56/84 [00:03<00:01, 18.30it/s][A 69% 58/84 [00:03<00:01, 18.30it/s][A 71% 60/84 [00:03<00:01, 18.30it/s][A 74% 62/84 [00:03<00:01, 18.32it/s][A 76% 64/84 [00:03<00:01, 18.32it/s][A 79% 66/84 [00:03<00:00, 18.32it/s][A 81% 68/84 [00:03<00:00, 18.32it/s][A 83% 70/84 [00:03<00:00, 18.33it/s][A 86% 72/84 [00:03<00:00, 18.32it/s][A 88% 74/84 [00:03<00:00, 18.31it/s][A 90% 76/84 [00:04<00:00, 18.30it/s][A 93% 78/84 [00:04<00:00, 18.32it/s][A 95% 80/84 [00:04<00:00, 18.33it/s][A 98% 82/84 [00:04<00:00, 18.34it/s][A {'eval_loss': 0.19117043912410736, 'eval_accuracy': 0.9279999732971191, 'eval_runtime': 4.5621, 'eval_samples_per_second': 438.393, 'eval_steps_per_second': 18.412, 'epoch': 1.5} 40% 1000/2500 [03:12<04:15, 5.86it/s] [A[INFO|trainer.py:2709] 2023-02-15 16:47:19,610 >> Saving model checkpoint to roberta_gpt_out/gpt2_custom/checkpoint-1000 [INFO|configuration_utils.py:453] 2023-02-15 16:47:19,611 >> Configuration saved in roberta_gpt_out/gpt2_custom/checkpoint-1000/config.json [INFO|modeling_utils.py:1704] 2023-02-15 16:47:20,386 >> Model weights saved in roberta_gpt_out/gpt2_custom/checkpoint-1000/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 16:47:20,387 >> tokenizer config file saved in roberta_gpt_out/gpt2_custom/checkpoint-1000/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 16:47:20,387 >> Special tokens file saved in roberta_gpt_out/gpt2_custom/checkpoint-1000/special_tokens_map.json {'loss': 0.2036, 'learning_rate': 1.1200000000000001e-05, 'epoch': 1.65} {'loss': 0.2155, 'learning_rate': 1.04e-05, 'epoch': 1.8} 50% 1250/2500 [03:58<03:34, 5.84it/s][INFO|trainer.py:710] 2023-02-15 16:48:04,913 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:48:04,915 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:48:04,915 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:48:04,915 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.17it/s][A 7% 6/84 [00:00<00:03, 21.17it/s][A 11% 9/84 [00:00<00:03, 19.76it/s][A 14% 12/84 [00:00<00:03, 19.17it/s][A 17% 14/84 [00:00<00:03, 18.83it/s][A 19% 16/84 [00:00<00:03, 18.66it/s][A 21% 18/84 [00:00<00:03, 18.55it/s][A 24% 20/84 [00:01<00:03, 18.40it/s][A 26% 22/84 [00:01<00:03, 18.33it/s][A 29% 24/84 [00:01<00:03, 18.32it/s][A 31% 26/84 [00:01<00:03, 18.30it/s][A 33% 28/84 [00:01<00:03, 18.26it/s][A 36% 30/84 [00:01<00:02, 18.28it/s][A 38% 32/84 [00:01<00:02, 18.30it/s][A 40% 34/84 [00:01<00:02, 18.30it/s][A 43% 36/84 [00:01<00:02, 18.30it/s][A 45% 38/84 [00:02<00:02, 18.30it/s][A 48% 40/84 [00:02<00:02, 18.28it/s][A 50% 42/84 [00:02<00:02, 18.29it/s][A 52% 44/84 [00:02<00:02, 18.29it/s][A 55% 46/84 [00:02<00:02, 18.30it/s][A 57% 48/84 [00:02<00:01, 18.31it/s][A 60% 50/84 [00:02<00:01, 18.32it/s][A 62% 52/84 [00:02<00:01, 18.31it/s][A 64% 54/84 [00:02<00:01, 18.32it/s][A 67% 56/84 [00:03<00:01, 18.32it/s][A 69% 58/84 [00:03<00:01, 18.31it/s][A 71% 60/84 [00:03<00:01, 18.33it/s][A 74% 62/84 [00:03<00:01, 18.34it/s][A 76% 64/84 [00:03<00:01, 18.34it/s][A 79% 66/84 [00:03<00:00, 18.35it/s][A 81% 68/84 [00:03<00:00, 18.37it/s][A 83% 70/84 [00:03<00:00, 18.36it/s][A 86% 72/84 [00:03<00:00, 18.33it/s][A 88% 74/84 [00:03<00:00, 18.35it/s][A 90% 76/84 [00:04<00:00, 18.34it/s][A 93% 78/84 [00:04<00:00, 18.34it/s][A 95% 80/84 [00:04<00:00, 18.35it/s][A 98% 82/84 [00:04<00:00, 18.35it/s][A {'eval_loss': 0.16847124695777893, 'eval_accuracy': 0.9319999814033508, 'eval_runtime': 4.5608, 'eval_samples_per_second': 438.519, 'eval_steps_per_second': 18.418, 'epoch': 1.87} 50% 1250/2500 [04:02<03:34, 5.84it/s] {'loss': 0.2466, 'learning_rate': 9.600000000000001e-06, 'epoch': 1.95} {'loss': 0.1791, 'learning_rate': 8.8e-06, 'epoch': 2.1} {'loss': 0.172, 'learning_rate': 8.000000000000001e-06, 'epoch': 2.25} 60% 1500/2500 [04:45<02:51, 5.84it/s][INFO|trainer.py:710] 2023-02-15 16:48:52,173 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:48:52,174 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:48:52,175 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:48:52,175 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.56it/s][A 7% 6/84 [00:00<00:03, 21.30it/s][A 11% 9/84 [00:00<00:03, 19.85it/s][A 14% 12/84 [00:00<00:03, 19.17it/s][A 17% 14/84 [00:00<00:03, 18.94it/s][A 19% 16/84 [00:00<00:03, 18.75it/s][A 21% 18/84 [00:00<00:03, 18.62it/s][A 24% 20/84 [00:01<00:03, 18.50it/s][A 26% 22/84 [00:01<00:03, 18.43it/s][A 29% 24/84 [00:01<00:03, 18.38it/s][A 31% 26/84 [00:01<00:03, 18.36it/s][A 33% 28/84 [00:01<00:03, 18.27it/s][A 36% 30/84 [00:01<00:02, 18.29it/s][A 38% 32/84 [00:01<00:02, 18.29it/s][A 40% 34/84 [00:01<00:02, 18.29it/s][A 43% 36/84 [00:01<00:02, 18.27it/s][A 45% 38/84 [00:02<00:02, 18.28it/s][A 48% 40/84 [00:02<00:02, 18.30it/s][A 50% 42/84 [00:02<00:02, 18.27it/s][A 52% 44/84 [00:02<00:02, 18.27it/s][A 55% 46/84 [00:02<00:02, 18.30it/s][A 57% 48/84 [00:02<00:01, 18.32it/s][A 60% 50/84 [00:02<00:01, 18.34it/s][A 62% 52/84 [00:02<00:01, 18.34it/s][A 64% 54/84 [00:02<00:01, 18.36it/s][A 67% 56/84 [00:03<00:01, 18.34it/s][A 69% 58/84 [00:03<00:01, 18.36it/s][A 71% 60/84 [00:03<00:01, 18.37it/s][A 74% 62/84 [00:03<00:01, 18.38it/s][A 76% 64/84 [00:03<00:01, 18.37it/s][A 79% 66/84 [00:03<00:00, 18.37it/s][A 81% 68/84 [00:03<00:00, 18.37it/s][A 83% 70/84 [00:03<00:00, 18.36it/s][A 86% 72/84 [00:03<00:00, 18.36it/s][A 88% 74/84 [00:03<00:00, 18.36it/s][A 90% 76/84 [00:04<00:00, 18.37it/s][A 93% 78/84 [00:04<00:00, 18.39it/s][A 95% 80/84 [00:04<00:00, 18.39it/s][A 98% 82/84 [00:04<00:00, 18.39it/s][A {'eval_loss': 0.18199807405471802, 'eval_accuracy': 0.9304999709129333, 'eval_runtime': 4.5537, 'eval_samples_per_second': 439.207, 'eval_steps_per_second': 18.447, 'epoch': 2.25} 60% 1500/2500 [04:49<02:51, 5.84it/s] [A[INFO|trainer.py:2709] 2023-02-15 16:48:56,729 >> Saving model checkpoint to roberta_gpt_out/gpt2_custom/checkpoint-1500 [INFO|configuration_utils.py:453] 2023-02-15 16:48:56,730 >> Configuration saved in roberta_gpt_out/gpt2_custom/checkpoint-1500/config.json [INFO|modeling_utils.py:1704] 2023-02-15 16:48:57,516 >> Model weights saved in roberta_gpt_out/gpt2_custom/checkpoint-1500/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 16:48:57,517 >> tokenizer config file saved in roberta_gpt_out/gpt2_custom/checkpoint-1500/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 16:48:57,517 >> Special tokens file saved in roberta_gpt_out/gpt2_custom/checkpoint-1500/special_tokens_map.json {'loss': 0.1795, 'learning_rate': 7.2000000000000005e-06, 'epoch': 2.4} {'loss': 0.1792, 'learning_rate': 6.4000000000000006e-06, 'epoch': 2.55} 70% 1750/2500 [05:35<02:09, 5.81it/s][INFO|trainer.py:710] 2023-02-15 16:49:42,128 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:49:42,130 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:49:42,130 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:49:42,130 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.47it/s][A 7% 6/84 [00:00<00:03, 21.26it/s][A 11% 9/84 [00:00<00:03, 19.82it/s][A 14% 12/84 [00:00<00:03, 19.21it/s][A 17% 14/84 [00:00<00:03, 18.96it/s][A 19% 16/84 [00:00<00:03, 18.74it/s][A 21% 18/84 [00:00<00:03, 18.61it/s][A 24% 20/84 [00:01<00:03, 18.50it/s][A 26% 22/84 [00:01<00:03, 18.44it/s][A 29% 24/84 [00:01<00:03, 18.39it/s][A 31% 26/84 [00:01<00:03, 18.34it/s][A 33% 28/84 [00:01<00:03, 18.33it/s][A 36% 30/84 [00:01<00:02, 18.32it/s][A 38% 32/84 [00:01<00:02, 18.30it/s][A 40% 34/84 [00:01<00:02, 18.29it/s][A 43% 36/84 [00:01<00:02, 18.30it/s][A 45% 38/84 [00:02<00:02, 18.28it/s][A 48% 40/84 [00:02<00:02, 18.28it/s][A 50% 42/84 [00:02<00:02, 18.28it/s][A 52% 44/84 [00:02<00:02, 18.29it/s][A 55% 46/84 [00:02<00:02, 18.24it/s][A 57% 48/84 [00:02<00:01, 18.27it/s][A 60% 50/84 [00:02<00:01, 18.30it/s][A 62% 52/84 [00:02<00:01, 18.31it/s][A 64% 54/84 [00:02<00:01, 18.31it/s][A 67% 56/84 [00:03<00:01, 18.32it/s][A 69% 58/84 [00:03<00:01, 18.32it/s][A 71% 60/84 [00:03<00:01, 18.34it/s][A 74% 62/84 [00:03<00:01, 18.35it/s][A 76% 64/84 [00:03<00:01, 18.36it/s][A 79% 66/84 [00:03<00:00, 18.34it/s][A 81% 68/84 [00:03<00:00, 18.35it/s][A 83% 70/84 [00:03<00:00, 18.33it/s][A 86% 72/84 [00:03<00:00, 18.33it/s][A 88% 74/84 [00:03<00:00, 18.34it/s][A 90% 76/84 [00:04<00:00, 18.35it/s][A 93% 78/84 [00:04<00:00, 18.37it/s][A 95% 80/84 [00:04<00:00, 18.38it/s][A 98% 82/84 [00:04<00:00, 18.39it/s][A {'eval_loss': 0.1523813158273697, 'eval_accuracy': 0.9334999918937683, 'eval_runtime': 4.5557, 'eval_samples_per_second': 439.008, 'eval_steps_per_second': 18.438, 'epoch': 2.62} 70% 1750/2500 [05:39<02:09, 5.81it/s] {'loss': 0.1561, 'learning_rate': 5.600000000000001e-06, 'epoch': 2.7} {'loss': 0.1587, 'learning_rate': 4.800000000000001e-06, 'epoch': 2.85} {'loss': 0.1505, 'learning_rate': 4.000000000000001e-06, 'epoch': 3.0} 80% 2000/2500 [06:22<01:25, 5.88it/s][INFO|trainer.py:710] 2023-02-15 16:50:29,449 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:50:29,451 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:50:29,451 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:50:29,451 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.14it/s][A 7% 6/84 [00:00<00:03, 21.20it/s][A 11% 9/84 [00:00<00:03, 19.77it/s][A 14% 12/84 [00:00<00:03, 19.12it/s][A 17% 14/84 [00:00<00:03, 18.88it/s][A 19% 16/84 [00:00<00:03, 18.72it/s][A 21% 18/84 [00:00<00:03, 18.60it/s][A 24% 20/84 [00:01<00:03, 18.52it/s][A 26% 22/84 [00:01<00:03, 18.47it/s][A 29% 24/84 [00:01<00:03, 18.43it/s][A 31% 26/84 [00:01<00:03, 18.39it/s][A 33% 28/84 [00:01<00:03, 18.37it/s][A 36% 30/84 [00:01<00:02, 18.36it/s][A 38% 32/84 [00:01<00:02, 18.35it/s][A 40% 34/84 [00:01<00:02, 18.34it/s][A 43% 36/84 [00:01<00:02, 18.33it/s][A 45% 38/84 [00:02<00:02, 18.32it/s][A 48% 40/84 [00:02<00:02, 18.34it/s][A 50% 42/84 [00:02<00:02, 18.33it/s][A 52% 44/84 [00:02<00:02, 18.32it/s][A 55% 46/84 [00:02<00:02, 18.32it/s][A 57% 48/84 [00:02<00:01, 18.28it/s][A 60% 50/84 [00:02<00:01, 18.12it/s][A 62% 52/84 [00:02<00:01, 18.17it/s][A 64% 54/84 [00:02<00:01, 18.22it/s][A 67% 56/84 [00:03<00:01, 18.26it/s][A 69% 58/84 [00:03<00:01, 18.28it/s][A 71% 60/84 [00:03<00:01, 18.29it/s][A 74% 62/84 [00:03<00:01, 18.30it/s][A 76% 64/84 [00:03<00:01, 18.32it/s][A 79% 66/84 [00:03<00:00, 18.33it/s][A 81% 68/84 [00:03<00:00, 18.30it/s][A 83% 70/84 [00:03<00:00, 18.29it/s][A 86% 72/84 [00:03<00:00, 18.28it/s][A 88% 74/84 [00:03<00:00, 18.27it/s][A 90% 76/84 [00:04<00:00, 18.28it/s][A 93% 78/84 [00:04<00:00, 18.27it/s][A 95% 80/84 [00:04<00:00, 18.27it/s][A 98% 82/84 [00:04<00:00, 18.25it/s][A {'eval_loss': 0.15084920823574066, 'eval_accuracy': 0.9375, 'eval_runtime': 4.5639, 'eval_samples_per_second': 438.222, 'eval_steps_per_second': 18.405, 'epoch': 3.0} 80% 2000/2500 [06:27<01:25, 5.88it/s] [A[INFO|trainer.py:2709] 2023-02-15 16:50:34,016 >> Saving model checkpoint to roberta_gpt_out/gpt2_custom/checkpoint-2000 [INFO|configuration_utils.py:453] 2023-02-15 16:50:34,017 >> Configuration saved in roberta_gpt_out/gpt2_custom/checkpoint-2000/config.json [INFO|modeling_utils.py:1704] 2023-02-15 16:50:34,796 >> Model weights saved in roberta_gpt_out/gpt2_custom/checkpoint-2000/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 16:50:34,797 >> tokenizer config file saved in roberta_gpt_out/gpt2_custom/checkpoint-2000/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 16:50:34,797 >> Special tokens file saved in roberta_gpt_out/gpt2_custom/checkpoint-2000/special_tokens_map.json {'loss': 0.137, 'learning_rate': 3.2000000000000003e-06, 'epoch': 3.15} {'loss': 0.1429, 'learning_rate': 2.4000000000000003e-06, 'epoch': 3.3} 90% 2250/2500 [07:12<00:42, 5.87it/s][INFO|trainer.py:710] 2023-02-15 16:51:19,304 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:51:19,305 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:51:19,306 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:51:19,306 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.44it/s][A 7% 6/84 [00:00<00:03, 21.26it/s][A 11% 9/84 [00:00<00:03, 19.82it/s][A 14% 12/84 [00:00<00:03, 19.22it/s][A 17% 14/84 [00:00<00:03, 18.93it/s][A 19% 16/84 [00:00<00:03, 18.74it/s][A 21% 18/84 [00:00<00:03, 18.59it/s][A 24% 20/84 [00:01<00:03, 18.51it/s][A 26% 22/84 [00:01<00:03, 18.43it/s][A 29% 24/84 [00:01<00:03, 18.41it/s][A 31% 26/84 [00:01<00:03, 18.30it/s][A 33% 28/84 [00:01<00:03, 18.31it/s][A 36% 30/84 [00:01<00:02, 18.31it/s][A 38% 32/84 [00:01<00:02, 18.30it/s][A 40% 34/84 [00:01<00:02, 18.28it/s][A 43% 36/84 [00:01<00:02, 18.28it/s][A 45% 38/84 [00:02<00:02, 18.29it/s][A 48% 40/84 [00:02<00:02, 18.29it/s][A 50% 42/84 [00:02<00:02, 18.30it/s][A 52% 44/84 [00:02<00:02, 18.29it/s][A 55% 46/84 [00:02<00:02, 18.31it/s][A 57% 48/84 [00:02<00:01, 18.29it/s][A 60% 50/84 [00:02<00:01, 18.28it/s][A 62% 52/84 [00:02<00:01, 18.28it/s][A 64% 54/84 [00:02<00:01, 18.28it/s][A 67% 56/84 [00:03<00:01, 18.30it/s][A 69% 58/84 [00:03<00:01, 18.32it/s][A 71% 60/84 [00:03<00:01, 18.32it/s][A 74% 62/84 [00:03<00:01, 18.32it/s][A 76% 64/84 [00:03<00:01, 18.30it/s][A 79% 66/84 [00:03<00:00, 18.29it/s][A 81% 68/84 [00:03<00:00, 18.28it/s][A 83% 70/84 [00:03<00:00, 18.21it/s][A 86% 72/84 [00:03<00:00, 18.22it/s][A 88% 74/84 [00:03<00:00, 17.99it/s][A 90% 76/84 [00:04<00:00, 18.07it/s][A 93% 78/84 [00:04<00:00, 18.13it/s][A 95% 80/84 [00:04<00:00, 18.14it/s][A 98% 82/84 [00:04<00:00, 18.18it/s][A {'eval_loss': 0.15722069144248962, 'eval_accuracy': 0.9319999814033508, 'eval_runtime': 4.5701, 'eval_samples_per_second': 437.626, 'eval_steps_per_second': 18.38, 'epoch': 3.37} 90% 2250/2500 [07:17<00:42, 5.87it/s] {'loss': 0.1575, 'learning_rate': 1.6000000000000001e-06, 'epoch': 3.45} {'loss': 0.1367, 'learning_rate': 8.000000000000001e-07, 'epoch': 3.6} {'loss': 0.1238, 'learning_rate': 0.0, 'epoch': 3.75} 100% 2500/2500 [07:59<00:00, 5.86it/s][INFO|trainer.py:710] 2023-02-15 16:52:06,584 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:52:06,586 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:52:06,586 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:52:06,586 >> Batch size = 24 0% 0/84 [00:00<?, ?it/s][A 4% 3/84 [00:00<00:02, 27.52it/s][A 7% 6/84 [00:00<00:03, 21.17it/s][A 11% 9/84 [00:00<00:03, 19.74it/s][A 14% 12/84 [00:00<00:03, 19.15it/s][A 17% 14/84 [00:00<00:03, 18.93it/s][A 19% 16/84 [00:00<00:03, 18.77it/s][A 21% 18/84 [00:00<00:03, 18.65it/s][A 24% 20/84 [00:01<00:03, 18.58it/s][A 26% 22/84 [00:01<00:03, 18.52it/s][A 29% 24/84 [00:01<00:03, 18.41it/s][A 31% 26/84 [00:01<00:03, 18.38it/s][A 33% 28/84 [00:01<00:03, 18.37it/s][A 36% 30/84 [00:01<00:02, 18.35it/s][A 38% 32/84 [00:01<00:02, 18.36it/s][A 40% 34/84 [00:01<00:02, 18.35it/s][A 43% 36/84 [00:01<00:02, 18.36it/s][A 45% 38/84 [00:02<00:02, 18.36it/s][A 48% 40/84 [00:02<00:02, 18.36it/s][A 50% 42/84 [00:02<00:02, 18.37it/s][A 52% 44/84 [00:02<00:02, 18.32it/s][A 55% 46/84 [00:02<00:02, 18.34it/s][A 57% 48/84 [00:02<00:01, 18.35it/s][A 60% 50/84 [00:02<00:01, 18.36it/s][A 62% 52/84 [00:02<00:01, 18.27it/s][A 64% 54/84 [00:02<00:01, 18.31it/s][A 67% 56/84 [00:03<00:01, 18.34it/s][A 69% 58/84 [00:03<00:01, 18.35it/s][A 71% 60/84 [00:03<00:01, 18.35it/s][A 74% 62/84 [00:03<00:01, 18.35it/s][A 76% 64/84 [00:03<00:01, 18.32it/s][A 79% 66/84 [00:03<00:00, 18.34it/s][A 81% 68/84 [00:03<00:00, 18.35it/s][A 83% 70/84 [00:03<00:00, 18.37it/s][A 86% 72/84 [00:03<00:00, 18.37it/s][A 88% 74/84 [00:03<00:00, 18.38it/s][A 90% 76/84 [00:04<00:00, 18.39it/s][A 93% 78/84 [00:04<00:00, 18.39it/s][A 95% 80/84 [00:04<00:00, 18.38it/s][A 98% 82/84 [00:04<00:00, 18.35it/s][A {'eval_loss': 0.14900894463062286, 'eval_accuracy': 0.934499979019165, 'eval_runtime': 4.5505, 'eval_samples_per_second': 439.516, 'eval_steps_per_second': 18.46, 'epoch': 3.75} 100% 2500/2500 [08:04<00:00, 5.86it/s] [A[INFO|trainer.py:2709] 2023-02-15 16:52:11,137 >> Saving model checkpoint to roberta_gpt_out/gpt2_custom/checkpoint-2500 [INFO|configuration_utils.py:453] 2023-02-15 16:52:11,138 >> Configuration saved in roberta_gpt_out/gpt2_custom/checkpoint-2500/config.json [INFO|modeling_utils.py:1704] 2023-02-15 16:52:11,913 >> Model weights saved in roberta_gpt_out/gpt2_custom/checkpoint-2500/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 16:52:11,914 >> tokenizer config file saved in roberta_gpt_out/gpt2_custom/checkpoint-2500/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 16:52:11,914 >> Special tokens file saved in roberta_gpt_out/gpt2_custom/checkpoint-2500/special_tokens_map.json [INFO|trainer.py:1901] 2023-02-15 16:52:13,776 >> Training completed. Do not forget to share your model on huggingface.co/models =) [INFO|trainer.py:2025] 2023-02-15 16:52:13,776 >> Loading best model from roberta_gpt_out/gpt2_custom/checkpoint-2000 (score: 0.9375). {'train_runtime': 487.3719, 'train_samples_per_second': 123.109, 'train_steps_per_second': 5.13, 'train_loss': 0.36852474327087403, 'epoch': 3.75} 100% 2500/2500 [08:07<00:00, 5.13it/s] [INFO|trainer.py:2709] 2023-02-15 16:52:14,234 >> Saving model checkpoint to roberta_gpt_out/gpt2_custom [INFO|configuration_utils.py:453] 2023-02-15 16:52:14,235 >> Configuration saved in roberta_gpt_out/gpt2_custom/config.json [INFO|modeling_utils.py:1704] 2023-02-15 16:52:15,266 >> Model weights saved in roberta_gpt_out/gpt2_custom/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 16:52:15,267 >> tokenizer config file saved in roberta_gpt_out/gpt2_custom/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 16:52:15,267 >> Special tokens file saved in roberta_gpt_out/gpt2_custom/special_tokens_map.json ***** train metrics ***** epoch = 3.75 train_loss = 0.3685 train_runtime = 0:08:07.37 train_samples = 16000 train_samples_per_second = 123.109 train_steps_per_second = 5.13 INFO:__main__:*** Evaluate *** [INFO|trainer.py:710] 2023-02-15 16:52:15,371 >> The following columns in the evaluation set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:52:15,373 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 16:52:15,373 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:52:15,373 >> Batch size = 24 100% 84/84 [00:04<00:00, 18.51it/s] ***** eval metrics ***** epoch = 3.75 eval_accuracy = 0.9375 eval_loss = 0.1508 eval_runtime = 0:00:04.60 eval_samples = 2000 eval_samples_per_second = 434.443 eval_steps_per_second = 18.247 INFO:__main__:*** Predict *** [INFO|trainer.py:710] 2023-02-15 16:52:19,980 >> The following columns in the test set don't have a corresponding argument in `GPT2ForSequenceClassificationCustomFIX.forward` and have been ignored: text. If text are not expected by `GPT2ForSequenceClassificationCustomFIX.forward`, you can safely ignore this message. [INFO|trainer.py:2964] 2023-02-15 16:52:19,981 >> ***** Running Prediction ***** [INFO|trainer.py:2966] 2023-02-15 16:52:19,981 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 16:52:19,981 >> Batch size = 24 100% 84/84 [00:04<00:00, 18.73it/s] INFO:__main__:***** Predict results None ***** [INFO|modelcard.py:449] 2023-02-15 16:52:24,864 >> Dropping the following result as it does not have all the necessary fields: {'task': {'name': 'Text Classification', 'type': 'text-classification'}, 'metrics': [{'name': 'Accuracy', 'type': 'accuracy', 'value': 0.9375}]}
T5
- full data
- model
T5
- sequnece length: 128
- training epoch: 1
- first few layers frozen
!python run_translation.py \
--cache_dir t5_large_cache/t5_v1_1 \
--model_name_or_path "google/t5-v1_1-large" \
--train_file data/s2s-train.json \
--validation_file data/s2s-valid.json \
--test_file data/s2s-test.json \
--per_device_train_batch_size 8 \
--per_device_eval_batch_size 8 \
--source_lang "text" \
--target_lang "label" \
--source_prefix "emotion classification" \
--max_source_length 256 \
--max_target_length 128 \
--generation_max_length 128 \
--do_train \
--do_eval \
--do_predict \
--predict_with_generate \
--num_train_epochs 1 \
--output_dir t5_large_out/t5_v1_1 \
--overwrite_output_dir \
--eval_steps 250 \
--evaluation_strategy steps \
--metric_for_best_model accuracy \
--logging_steps 100 \
--save_total_limit 5 \
--max_steps 2500 \
--load_best_model_at_end True
[1;30;43mStreaming output truncated to the last 5000 lines.[0m "transformers_version": "4.26.1" } 28% 71/250 [00:14<00:36, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:28,987 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 29% 72/250 [00:14<00:39, 4.56it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:29,238 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 29% 73/250 [00:14<00:38, 4.59it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:29,453 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 30% 74/250 [00:15<00:37, 4.68it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:29,656 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 30% 75/250 [00:15<00:36, 4.78it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:29,856 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 30% 76/250 [00:15<00:36, 4.73it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:30,072 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 31% 77/250 [00:15<00:36, 4.78it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:30,276 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 31% 78/250 [00:15<00:35, 4.84it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:30,476 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 32% 79/250 [00:16<00:34, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:30,676 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 32% 80/250 [00:16<00:34, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:30,880 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 32% 81/250 [00:16<00:34, 4.88it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:31,086 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 33% 82/250 [00:16<00:34, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:31,285 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 33% 83/250 [00:16<00:33, 4.93it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:31,486 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 34% 84/250 [00:17<00:33, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:31,689 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 34% 85/250 [00:17<00:33, 4.99it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:31,885 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 34% 86/250 [00:17<00:33, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:32,092 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 35% 87/250 [00:17<00:32, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:32,294 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 35% 88/250 [00:17<00:32, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:32,500 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 36% 89/250 [00:18<00:32, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:32,704 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 36% 90/250 [00:18<00:32, 4.95it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:32,903 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 36% 91/250 [00:18<00:32, 4.84it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:33,119 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 37% 92/250 [00:18<00:32, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:33,319 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 37% 93/250 [00:18<00:31, 4.93it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:33,518 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 38% 94/250 [00:19<00:31, 4.99it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:33,713 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 38% 95/250 [00:19<00:31, 4.97it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:33,916 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 38% 96/250 [00:19<00:31, 4.97it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:34,118 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 39% 97/250 [00:19<00:30, 4.98it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:34,317 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 39% 98/250 [00:19<00:30, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:34,523 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 40% 99/250 [00:20<00:30, 4.96it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:34,724 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 40% 100/250 [00:20<00:30, 4.95it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:34,926 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 40% 101/250 [00:20<00:30, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:35,130 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 41% 102/250 [00:20<00:30, 4.88it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:35,341 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 41% 103/250 [00:20<00:29, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:35,540 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 42% 104/250 [00:21<00:29, 4.95it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:35,739 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 42% 105/250 [00:21<00:29, 4.95it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:35,941 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 42% 106/250 [00:21<00:29, 4.93it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:36,145 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 43% 107/250 [00:21<00:29, 4.93it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:36,348 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 43% 108/250 [00:21<00:28, 4.97it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:36,545 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 44% 109/250 [00:22<00:29, 4.81it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:36,768 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 44% 110/250 [00:22<00:28, 4.83it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:36,974 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 44% 111/250 [00:22<00:28, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:37,175 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 45% 112/250 [00:22<00:28, 4.84it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:37,388 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 45% 113/250 [00:22<00:28, 4.84it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:37,591 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 46% 114/250 [00:23<00:28, 4.85it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:37,796 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 46% 115/250 [00:23<00:27, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:38,000 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 46% 116/250 [00:23<00:27, 4.84it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:38,209 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 47% 117/250 [00:23<00:27, 4.85it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:38,414 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 47% 118/250 [00:24<00:27, 4.81it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:38,626 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 48% 119/250 [00:24<00:27, 4.80it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:38,836 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 48% 120/250 [00:24<00:27, 4.69it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:39,060 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 48% 121/250 [00:24<00:28, 4.46it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:39,311 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 49% 122/250 [00:24<00:27, 4.63it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:39,506 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 49% 123/250 [00:25<00:26, 4.72it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:39,709 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 50% 124/250 [00:25<00:26, 4.80it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:39,909 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 50% 125/250 [00:25<00:25, 4.81it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:40,115 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 50% 126/250 [00:25<00:25, 4.86it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:40,316 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 51% 127/250 [00:25<00:25, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:40,514 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 51% 128/250 [00:26<00:24, 4.97it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:40,711 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 52% 129/250 [00:26<00:24, 4.93it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:40,917 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 52% 130/250 [00:26<00:24, 4.95it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:41,117 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 52% 131/250 [00:26<00:24, 4.93it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:41,322 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 53% 132/250 [00:26<00:23, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:41,524 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 53% 133/250 [00:27<00:24, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:41,736 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 54% 134/250 [00:27<00:23, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:41,940 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 54% 135/250 [00:27<00:23, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:42,144 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 54% 136/250 [00:27<00:23, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:42,344 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 55% 137/250 [00:27<00:22, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:42,545 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 55% 138/250 [00:28<00:22, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:42,751 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 56% 139/250 [00:28<00:22, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:42,955 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 56% 140/250 [00:28<00:22, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:43,157 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 56% 141/250 [00:28<00:22, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:43,357 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 57% 142/250 [00:28<00:21, 4.99it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:43,553 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 57% 143/250 [00:29<00:21, 4.99it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:43,753 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 58% 144/250 [00:29<00:21, 4.98it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:43,955 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 58% 145/250 [00:29<00:21, 4.95it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:44,160 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 58% 146/250 [00:29<00:21, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:44,363 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 59% 147/250 [00:29<00:20, 4.98it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:44,560 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 59% 148/250 [00:30<00:20, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:44,766 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 60% 149/250 [00:30<00:20, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:44,968 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 60% 150/250 [00:30<00:20, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:45,173 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 60% 151/250 [00:30<00:20, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:45,375 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 61% 152/250 [00:30<00:19, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:45,579 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 61% 153/250 [00:31<00:19, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:45,784 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 62% 154/250 [00:31<00:19, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:45,987 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 62% 155/250 [00:31<00:19, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:46,197 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 62% 156/250 [00:31<00:19, 4.79it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:46,414 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 63% 157/250 [00:31<00:19, 4.85it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:46,614 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 63% 158/250 [00:32<00:18, 4.86it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:46,819 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 64% 159/250 [00:32<00:18, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:47,018 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 64% 160/250 [00:32<00:18, 4.95it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:47,216 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 64% 161/250 [00:32<00:18, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:47,422 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 65% 162/250 [00:33<00:17, 4.96it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:47,620 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 65% 163/250 [00:33<00:17, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:47,834 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 66% 164/250 [00:33<00:17, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:48,033 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 66% 165/250 [00:33<00:17, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:48,237 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 66% 166/250 [00:33<00:17, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:48,445 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 67% 167/250 [00:34<00:17, 4.85it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:48,653 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 67% 168/250 [00:34<00:16, 4.85it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:48,860 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 68% 169/250 [00:34<00:16, 4.86it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:49,064 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 68% 170/250 [00:34<00:16, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:49,266 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 68% 171/250 [00:34<00:16, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:49,469 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 69% 172/250 [00:35<00:15, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:49,671 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 69% 173/250 [00:35<00:15, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:49,873 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 70% 174/250 [00:35<00:15, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:50,079 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 70% 175/250 [00:35<00:15, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:50,284 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 70% 176/250 [00:35<00:15, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:50,486 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 71% 177/250 [00:36<00:14, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:50,690 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 71% 178/250 [00:36<00:14, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:50,895 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 72% 179/250 [00:36<00:14, 4.86it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:51,105 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 72% 180/250 [00:36<00:14, 4.86it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:51,311 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 72% 181/250 [00:36<00:14, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:51,511 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 73% 182/250 [00:37<00:13, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:51,715 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 73% 183/250 [00:37<00:13, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:51,918 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 74% 184/250 [00:37<00:13, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:52,122 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 74% 185/250 [00:37<00:13, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:52,331 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 74% 186/250 [00:37<00:12, 4.93it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:52,528 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 75% 187/250 [00:38<00:12, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:52,733 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 75% 188/250 [00:38<00:12, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:52,935 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 76% 189/250 [00:38<00:12, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:53,139 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 76% 190/250 [00:38<00:12, 4.95it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:53,338 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 76% 191/250 [00:38<00:11, 4.96it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:53,539 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 77% 192/250 [00:39<00:11, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:53,743 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 77% 193/250 [00:39<00:11, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:53,949 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 78% 194/250 [00:39<00:11, 4.83it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:54,165 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 78% 195/250 [00:39<00:11, 4.83it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:54,371 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 78% 196/250 [00:39<00:11, 4.86it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:54,575 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 79% 197/250 [00:40<00:10, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:54,775 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 79% 198/250 [00:40<00:10, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:54,983 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 80% 199/250 [00:40<00:10, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:55,181 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 80% 200/250 [00:40<00:10, 4.97it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:55,378 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 80% 201/250 [00:40<00:09, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:55,583 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 81% 202/250 [00:41<00:09, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:55,786 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 81% 203/250 [00:41<00:09, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:55,999 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 82% 204/250 [00:41<00:10, 4.56it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:56,250 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 82% 205/250 [00:41<00:09, 4.65it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:56,455 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 82% 206/250 [00:42<00:09, 4.71it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:56,661 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 83% 207/250 [00:42<00:08, 4.79it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:56,862 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 83% 208/250 [00:42<00:08, 4.69it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:57,085 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 84% 209/250 [00:42<00:08, 4.75it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:57,289 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 84% 210/250 [00:42<00:08, 4.80it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:57,493 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 84% 211/250 [00:43<00:08, 4.86it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:57,692 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 85% 212/250 [00:43<00:07, 4.86it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:57,897 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 85% 213/250 [00:43<00:07, 4.84it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:58,107 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 86% 214/250 [00:43<00:07, 4.88it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:58,308 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 86% 215/250 [00:43<00:07, 4.85it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:58,516 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 86% 216/250 [00:44<00:06, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:58,715 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 87% 217/250 [00:44<00:06, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:58,917 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 87% 218/250 [00:44<00:06, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:59,125 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 88% 219/250 [00:44<00:06, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:59,329 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 88% 220/250 [00:44<00:06, 4.93it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:59,528 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 88% 221/250 [00:45<00:05, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:59,733 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 89% 222/250 [00:45<00:05, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:21:59,936 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 89% 223/250 [00:45<00:05, 4.94it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:00,137 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 90% 224/250 [00:45<00:05, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:00,345 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 90% 225/250 [00:45<00:05, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:00,549 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 90% 226/250 [00:46<00:04, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:00,758 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 91% 227/250 [00:46<00:04, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:00,956 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 91% 228/250 [00:46<00:04, 4.96it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:01,154 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 92% 229/250 [00:46<00:04, 4.98it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:01,353 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 92% 230/250 [00:46<00:04, 4.92it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:01,561 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 92% 231/250 [00:47<00:03, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:01,767 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 93% 232/250 [00:47<00:03, 4.86it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:01,977 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 93% 233/250 [00:47<00:03, 4.83it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:02,188 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 94% 234/250 [00:47<00:03, 4.88it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:02,387 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 94% 235/250 [00:47<00:03, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:02,591 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 94% 236/250 [00:48<00:02, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:02,792 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 95% 237/250 [00:48<00:02, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:02,999 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 95% 238/250 [00:48<00:02, 4.84it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:03,211 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 96% 239/250 [00:48<00:02, 4.87it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:03,413 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 96% 240/250 [00:48<00:02, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:03,614 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 96% 241/250 [00:49<00:01, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:03,816 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 97% 242/250 [00:49<00:01, 4.89it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:04,023 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 97% 243/250 [00:49<00:01, 4.88it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:04,229 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 98% 244/250 [00:49<00:01, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:04,429 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 98% 245/250 [00:50<00:01, 4.84it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:04,644 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 98% 246/250 [00:50<00:00, 4.82it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:04,853 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 99% 247/250 [00:50<00:00, 4.85it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:05,056 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 99% 248/250 [00:50<00:00, 4.91it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:05,254 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 100% 249/250 [00:50<00:00, 4.90it/s][A[INFO|configuration_utils.py:543] 2023-02-15 18:22:05,460 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 100% 250/250 [00:51<00:00, 4.93it/s][A {'eval_loss': 0.43915173411369324, 'eval_bleu': 0.0, 'eval_accuracy': 1.0, 'eval_gen_len': 2.0, 'eval_runtime': 51.3996, 'eval_samples_per_second': 38.911, 'eval_steps_per_second': 4.864, 'epoch': 1.25} 100% 2500/2500 [44:34<00:00, 3.50it/s] [A[INFO|trainer.py:2709] 2023-02-15 18:22:05,804 >> Saving model checkpoint to t5_large_out/t5_v1_1/checkpoint-2500 [INFO|configuration_utils.py:453] 2023-02-15 18:22:05,805 >> Configuration saved in t5_large_out/t5_v1_1/checkpoint-2500/config.json [INFO|configuration_utils.py:336] 2023-02-15 18:22:05,810 >> Configuration saved in t5_large_out/t5_v1_1/checkpoint-2500/generation_config.json [INFO|modeling_utils.py:1704] 2023-02-15 18:22:11,133 >> Model weights saved in t5_large_out/t5_v1_1/checkpoint-2500/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 18:22:11,134 >> tokenizer config file saved in t5_large_out/t5_v1_1/checkpoint-2500/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 18:22:11,134 >> Special tokens file saved in t5_large_out/t5_v1_1/checkpoint-2500/special_tokens_map.json [INFO|tokenization_t5_fast.py:186] 2023-02-15 18:22:11,171 >> Copy vocab file to t5_large_out/t5_v1_1/checkpoint-2500/spiece.model [INFO|trainer.py:1901] 2023-02-15 18:22:22,537 >> Training completed. Do not forget to share your model on huggingface.co/models =) [INFO|trainer.py:2025] 2023-02-15 18:22:22,537 >> Loading best model from t5_large_out/t5_v1_1/checkpoint-500 (score: 1.0). {'train_runtime': 2694.094, 'train_samples_per_second': 7.424, 'train_steps_per_second': 0.928, 'train_loss': 4.872698245239258, 'epoch': 1.25} 100% 2500/2500 [44:54<00:00, 1.08s/it] [INFO|trainer.py:2709] 2023-02-15 18:22:25,217 >> Saving model checkpoint to t5_large_out/t5_v1_1 [INFO|configuration_utils.py:453] 2023-02-15 18:22:25,218 >> Configuration saved in t5_large_out/t5_v1_1/config.json [INFO|configuration_utils.py:336] 2023-02-15 18:22:25,223 >> Configuration saved in t5_large_out/t5_v1_1/generation_config.json [INFO|modeling_utils.py:1704] 2023-02-15 18:22:30,856 >> Model weights saved in t5_large_out/t5_v1_1/pytorch_model.bin [INFO|tokenization_utils_base.py:2160] 2023-02-15 18:22:30,857 >> tokenizer config file saved in t5_large_out/t5_v1_1/tokenizer_config.json [INFO|tokenization_utils_base.py:2167] 2023-02-15 18:22:30,857 >> Special tokens file saved in t5_large_out/t5_v1_1/special_tokens_map.json [INFO|tokenization_t5_fast.py:186] 2023-02-15 18:22:30,902 >> Copy vocab file to t5_large_out/t5_v1_1/spiece.model ***** train metrics ***** epoch = 1.25 train_loss = 4.8727 train_runtime = 0:44:54.09 train_samples = 16000 train_samples_per_second = 7.424 train_steps_per_second = 0.928 INFO:__main__:*** Evaluate *** [INFO|trainer.py:2964] 2023-02-15 18:22:30,914 >> ***** Running Evaluation ***** [INFO|trainer.py:2966] 2023-02-15 18:22:30,914 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 18:22:30,914 >> Batch size = 8 [INFO|configuration_utils.py:543] 2023-02-15 18:22:30,926 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 0% 0/250 [00:00<?, ?it/s][INFO|configuration_utils.py:543] 2023-02-15 18:22:36,220 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 1% 2/250 [00:00<01:25, 2.90it/s][INFO|configuration_utils.py:543] 2023-02-15 18:22:36,909 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 1% 3/250 [00:00<01:15, 3.29it/s][INFO|configuration_utils.py:543] 2023-02-15 18:22:37,156 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 2% 4/250 [00:01<01:45, 2.33it/s][INFO|configuration_utils.py:543] 2023-02-15 18:22:37,797 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 2% 5/250 [00:04<05:17, 1.30s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:40,717 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 2% 6/250 [00:05<05:31, 1.36s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:42,206 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 3% 7/250 [00:06<04:20, 1.07s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:42,672 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 3% 8/250 [00:10<08:07, 2.02s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:46,742 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 4% 9/250 [00:11<06:45, 1.68s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:47,686 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 4% 10/250 [00:13<07:45, 1.94s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:50,208 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 4% 11/250 [00:14<05:55, 1.49s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:50,667 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 5% 12/250 [00:16<06:21, 1.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:52,536 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 5% 13/250 [00:20<09:09, 2.32s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:56,496 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 6% 14/250 [00:22<09:27, 2.41s/it][INFO|configuration_utils.py:543] 2023-02-15 18:22:59,106 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 6% 15/250 [00:25<09:24, 2.40s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:01,497 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 6% 16/250 [00:25<07:02, 1.81s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:01,918 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 7% 17/250 [00:30<10:35, 2.73s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:06,794 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 7% 18/250 [00:30<07:48, 2.02s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:07,168 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 8% 19/250 [00:35<10:58, 2.85s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:11,952 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 8% 20/250 [00:36<08:22, 2.19s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:12,587 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 8% 21/250 [00:38<08:28, 2.22s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:14,886 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 9% 22/250 [00:38<06:14, 1.64s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:15,182 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 9% 23/250 [00:39<04:43, 1.25s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:15,513 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 10% 24/250 [00:40<05:01, 1.33s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:17,039 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 10% 25/250 [00:42<05:32, 1.48s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:18,858 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 10% 26/250 [00:46<08:20, 2.23s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:22,857 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 11% 27/250 [00:51<11:32, 3.10s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:27,989 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 11% 28/250 [00:55<12:37, 3.41s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:32,115 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 12% 29/250 [00:57<10:01, 2.72s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:33,234 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 12% 30/250 [00:58<08:49, 2.41s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:34,908 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 12% 31/250 [00:59<07:34, 2.08s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:36,212 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 13% 32/250 [01:01<07:14, 1.99s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:38,011 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 13% 33/250 [01:02<05:31, 1.53s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:38,446 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 14% 34/250 [01:05<07:51, 2.18s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:42,155 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 14% 35/250 [01:07<06:51, 1.91s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:43,445 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 14% 36/250 [01:11<09:13, 2.59s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:47,599 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 15% 37/250 [01:15<10:52, 3.06s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:51,779 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 15% 38/250 [01:17<10:04, 2.85s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:54,135 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 16% 39/250 [01:20<09:51, 2.81s/it][INFO|configuration_utils.py:543] 2023-02-15 18:23:56,832 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 16% 40/250 [01:25<11:40, 3.33s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:01,398 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 16% 41/250 [01:30<13:29, 3.87s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:06,531 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 17% 42/250 [01:31<10:46, 3.11s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:07,861 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 17% 43/250 [01:33<09:24, 2.72s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:09,686 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 18% 44/250 [01:38<11:42, 3.41s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:14,702 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 18% 45/250 [01:42<12:02, 3.52s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:18,482 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 18% 46/250 [01:44<10:51, 3.19s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:20,904 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 19% 47/250 [01:47<09:54, 2.93s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:23,224 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 19% 48/250 [01:47<07:22, 2.19s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:23,686 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 20% 49/250 [01:52<10:20, 3.09s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:28,871 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 20% 50/250 [01:57<12:28, 3.74s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:34,133 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 20% 51/250 [02:03<13:51, 4.18s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:39,329 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 21% 52/250 [02:06<12:35, 3.81s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:42,296 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 21% 53/250 [02:07<09:57, 3.03s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:43,508 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 22% 54/250 [02:09<09:05, 2.78s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:45,704 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 22% 55/250 [02:12<09:29, 2.92s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:48,946 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 22% 56/250 [02:13<07:11, 2.23s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:49,551 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 23% 57/250 [02:18<09:32, 2.96s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:54,238 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 23% 58/250 [02:18<07:25, 2.32s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:55,051 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 24% 59/250 [02:22<08:17, 2.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:58,315 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 24% 60/250 [02:23<06:37, 2.09s/it][INFO|configuration_utils.py:543] 2023-02-15 18:24:59,222 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 24% 61/250 [02:24<05:42, 1.81s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:00,379 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 25% 62/250 [02:24<04:38, 1.48s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:01,089 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 25% 63/250 [02:26<04:53, 1.57s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:02,859 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 26% 64/250 [02:31<08:05, 2.61s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:07,893 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 26% 65/250 [02:32<06:06, 1.98s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:08,420 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 26% 66/250 [02:32<04:53, 1.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:09,109 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 27% 67/250 [02:33<03:37, 1.19s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:09,356 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 27% 68/250 [02:38<07:15, 2.39s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:14,560 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 28% 69/250 [02:39<06:30, 2.16s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:16,158 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 28% 70/250 [02:40<04:44, 1.58s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:16,401 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 28% 71/250 [02:41<04:53, 1.64s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:18,172 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 29% 72/250 [02:43<05:08, 1.74s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:20,134 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 29% 73/250 [02:48<07:21, 2.50s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:24,404 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 30% 74/250 [02:52<09:09, 3.12s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:28,993 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 30% 75/250 [02:53<07:20, 2.52s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:30,090 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 30% 76/250 [02:54<06:03, 2.09s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:31,176 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 31% 77/250 [02:56<05:15, 1.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:32,380 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 31% 78/250 [02:57<04:52, 1.70s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:33,802 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 32% 79/250 [02:58<04:33, 1.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:35,160 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 32% 80/250 [03:03<07:13, 2.55s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:39,933 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 32% 81/250 [03:07<08:34, 3.04s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:44,122 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 33% 82/250 [03:08<06:20, 2.27s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:44,579 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 33% 83/250 [03:09<05:18, 1.91s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:45,646 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 34% 84/250 [03:13<07:01, 2.54s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:49,662 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 34% 85/250 [03:17<08:16, 3.01s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:53,774 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 34% 86/250 [03:17<05:57, 2.18s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:54,017 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 35% 87/250 [03:18<04:55, 1.81s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:54,962 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 35% 88/250 [03:20<05:14, 1.94s/it][INFO|configuration_utils.py:543] 2023-02-15 18:25:57,200 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 36% 89/250 [03:26<07:46, 2.90s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:02,331 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 36% 90/250 [03:26<05:57, 2.23s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:03,018 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 36% 91/250 [03:29<06:17, 2.38s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:05,729 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 37% 92/250 [03:33<07:35, 2.88s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:09,784 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 37% 93/250 [03:33<05:27, 2.09s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:10,026 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 38% 94/250 [03:39<07:53, 3.04s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:15,277 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 38% 95/250 [03:42<08:02, 3.11s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:18,573 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 38% 96/250 [03:43<06:09, 2.40s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:19,299 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 39% 97/250 [03:47<08:02, 3.15s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:24,217 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 39% 98/250 [03:53<09:37, 3.80s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:29,516 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 40% 99/250 [03:53<07:03, 2.80s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:30,002 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 40% 100/250 [03:57<07:56, 3.17s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:34,038 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 40% 101/250 [04:03<09:23, 3.78s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:39,229 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 41% 102/250 [04:07<09:49, 3.98s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:43,687 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 41% 103/250 [04:09<08:39, 3.54s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:46,179 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 42% 104/250 [04:12<07:49, 3.21s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:48,645 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 42% 105/250 [04:15<07:18, 3.03s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:51,228 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 42% 106/250 [04:16<05:55, 2.47s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:52,392 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 43% 107/250 [04:21<07:49, 3.28s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:57,574 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 43% 108/250 [04:22<06:17, 2.66s/it][INFO|configuration_utils.py:543] 2023-02-15 18:26:58,783 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 44% 109/250 [04:24<05:59, 2.55s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:01,073 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 44% 110/250 [04:25<04:47, 2.05s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:01,973 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 44% 111/250 [04:26<03:36, 1.56s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:02,368 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 45% 112/250 [04:29<05:06, 2.22s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:06,136 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 45% 113/250 [04:33<06:17, 2.76s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:10,142 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 46% 114/250 [04:36<06:12, 2.74s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:12,850 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 46% 115/250 [04:38<05:16, 2.35s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:14,270 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 46% 116/250 [04:42<06:46, 3.03s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:18,913 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 47% 117/250 [04:45<06:29, 2.93s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:21,602 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 47% 118/250 [04:48<06:25, 2.92s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:24,489 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 48% 119/250 [04:52<07:20, 3.37s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:28,902 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 48% 120/250 [04:55<06:40, 3.08s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:31,325 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 48% 121/250 [04:55<04:56, 2.30s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:31,798 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 49% 122/250 [04:58<05:18, 2.49s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:34,738 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 49% 123/250 [04:59<04:10, 1.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:35,503 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 50% 124/250 [05:00<03:35, 1.71s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:36,595 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 50% 125/250 [05:00<02:49, 1.35s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:37,116 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 50% 126/250 [05:06<05:07, 2.48s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:42,228 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 51% 127/250 [05:06<04:03, 1.98s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:43,036 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 51% 128/250 [05:07<03:02, 1.50s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:43,415 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 52% 129/250 [05:08<02:44, 1.36s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:44,444 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 52% 130/250 [05:12<04:31, 2.26s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:48,816 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 52% 131/250 [05:13<03:36, 1.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:49,601 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 53% 132/250 [05:18<05:30, 2.80s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:54,706 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 53% 133/250 [05:19<04:38, 2.38s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:56,101 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 54% 134/250 [05:21<04:16, 2.21s/it][INFO|configuration_utils.py:543] 2023-02-15 18:27:57,912 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 54% 135/250 [05:23<04:13, 2.20s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:00,089 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 54% 136/250 [05:27<04:45, 2.51s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:03,308 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 55% 137/250 [05:30<05:00, 2.66s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:06,329 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 55% 138/250 [05:30<03:51, 2.07s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:07,020 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 56% 139/250 [05:32<03:33, 1.92s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:08,608 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 56% 140/250 [05:34<03:34, 1.95s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:10,610 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 56% 141/250 [05:37<03:59, 2.19s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:13,374 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 57% 142/250 [05:37<03:00, 1.67s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:13,835 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 57% 143/250 [05:38<02:34, 1.44s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:14,746 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 58% 144/250 [05:40<02:54, 1.64s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:16,856 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 58% 145/250 [05:41<02:19, 1.33s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:17,459 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 58% 146/250 [05:42<02:29, 1.44s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:19,147 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 59% 147/250 [05:44<02:35, 1.51s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:20,814 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 59% 148/250 [05:45<02:04, 1.22s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:21,370 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 60% 149/250 [05:47<02:22, 1.41s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:23,235 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 60% 150/250 [05:48<02:32, 1.53s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:25,026 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 60% 151/250 [05:49<02:13, 1.35s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:25,968 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 61% 152/250 [05:51<02:26, 1.49s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:27,789 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 61% 153/250 [05:55<03:41, 2.29s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:31,933 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 62% 154/250 [06:00<05:02, 3.16s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:37,113 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 62% 155/250 [06:03<04:30, 2.85s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:39,243 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 62% 156/250 [06:07<05:03, 3.23s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:43,365 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 63% 157/250 [06:09<04:27, 2.88s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:45,431 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 63% 158/250 [06:10<03:44, 2.44s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:46,858 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 64% 159/250 [06:11<03:10, 2.10s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:48,148 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 64% 160/250 [06:16<04:15, 2.84s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:52,721 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 64% 161/250 [06:18<03:57, 2.67s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:54,988 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 65% 162/250 [06:21<03:56, 2.69s/it][INFO|configuration_utils.py:543] 2023-02-15 18:28:57,728 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 65% 163/250 [06:24<04:08, 2.86s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:00,980 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 66% 164/250 [06:29<05:03, 3.52s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:06,057 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 66% 165/250 [06:34<05:39, 4.00s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:11,166 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 66% 166/250 [06:37<05:02, 3.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:13,819 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 67% 167/250 [06:38<03:43, 2.70s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:14,422 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 67% 168/250 [06:39<03:09, 2.31s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:15,844 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 68% 169/250 [06:44<04:12, 3.12s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:20,840 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 68% 170/250 [06:45<03:15, 2.44s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:21,702 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 68% 171/250 [06:49<03:57, 3.00s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:26,012 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 69% 172/250 [06:50<03:07, 2.41s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:27,032 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 69% 173/250 [06:51<02:33, 1.99s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:28,062 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 70% 174/250 [06:52<01:51, 1.47s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:28,314 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 70% 175/250 [06:53<01:39, 1.33s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:29,299 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 70% 176/250 [06:53<01:23, 1.13s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:29,982 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 71% 177/250 [06:54<01:22, 1.14s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:31,125 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 71% 178/250 [06:56<01:32, 1.29s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:32,766 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 72% 179/250 [06:57<01:17, 1.09s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:33,406 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 72% 180/250 [07:01<02:23, 2.05s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:37,682 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 72% 181/250 [07:01<01:48, 1.57s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:38,149 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 73% 182/250 [07:05<02:33, 2.25s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:41,988 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 73% 183/250 [07:07<02:29, 2.23s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:44,150 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 74% 184/250 [07:11<03:02, 2.77s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:48,183 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 74% 185/250 [07:12<02:11, 2.02s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:48,470 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 74% 186/250 [07:17<03:09, 2.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:53,638 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 75% 187/250 [07:19<02:55, 2.78s/it][INFO|configuration_utils.py:543] 2023-02-15 18:29:55,982 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 75% 188/250 [07:24<03:35, 3.47s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:01,063 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 76% 189/250 [07:26<02:58, 2.92s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:02,704 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 76% 190/250 [07:29<02:57, 2.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:05,770 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 76% 191/250 [07:34<03:30, 3.58s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:10,772 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 77% 192/250 [07:38<03:40, 3.81s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:15,128 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 77% 193/250 [07:39<02:40, 2.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:15,639 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 78% 194/250 [07:44<03:16, 3.51s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:20,763 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 78% 195/250 [07:48<03:13, 3.52s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:24,302 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 78% 196/250 [07:53<03:37, 4.03s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:29,513 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 79% 197/250 [07:57<03:43, 4.22s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:34,181 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 79% 198/250 [07:59<02:50, 3.28s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:35,260 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 80% 199/250 [07:59<02:06, 2.49s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:35,904 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 80% 200/250 [08:03<02:20, 2.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:39,490 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 80% 201/250 [08:04<01:48, 2.21s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:40,295 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 81% 202/250 [08:07<02:02, 2.56s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:43,653 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 81% 203/250 [08:09<01:59, 2.53s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:46,129 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 82% 204/250 [08:11<01:38, 2.15s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:47,390 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 82% 205/250 [08:11<01:14, 1.65s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:47,868 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 82% 206/250 [08:13<01:20, 1.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:50,103 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 83% 207/250 [08:14<01:08, 1.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:51,179 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 83% 208/250 [08:15<00:56, 1.34s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:51,929 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 84% 209/250 [08:18<01:17, 1.89s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:55,081 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 84% 210/250 [08:19<00:56, 1.41s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:55,376 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 84% 211/250 [08:20<00:57, 1.48s/it][INFO|configuration_utils.py:543] 2023-02-15 18:30:57,033 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 85% 212/250 [08:25<01:28, 2.32s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:01,313 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 85% 213/250 [08:29<01:43, 2.81s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:05,249 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 86% 214/250 [08:32<01:48, 3.01s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:08,727 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 86% 215/250 [08:33<01:19, 2.29s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:09,326 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 86% 216/250 [08:34<01:04, 1.89s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:10,289 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 87% 217/250 [08:37<01:14, 2.25s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:13,369 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 87% 218/250 [08:40<01:24, 2.65s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:16,951 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 88% 219/250 [08:44<01:36, 3.11s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:21,158 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 88% 220/250 [08:49<01:50, 3.69s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:26,181 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 88% 221/250 [08:50<01:19, 2.75s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:26,734 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 89% 222/250 [08:52<01:09, 2.47s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:28,574 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 89% 223/250 [08:56<01:20, 2.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:32,694 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 90% 224/250 [08:56<00:56, 2.19s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:33,072 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 90% 225/250 [08:57<00:45, 1.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:34,033 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 90% 226/250 [09:03<01:08, 2.85s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:39,296 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 91% 227/250 [09:06<01:12, 3.15s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:43,139 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 91% 228/250 [09:09<01:06, 3.00s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:45,794 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 92% 229/250 [09:14<01:16, 3.62s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:50,871 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 92% 230/250 [09:17<01:07, 3.37s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:53,638 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 92% 231/250 [09:22<01:11, 3.75s/it][INFO|configuration_utils.py:543] 2023-02-15 18:31:58,280 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 93% 232/250 [09:26<01:12, 4.04s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:02,988 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 93% 233/250 [09:29<01:00, 3.53s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:05,334 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 94% 234/250 [09:29<00:43, 2.71s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:06,141 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 94% 235/250 [09:34<00:51, 3.41s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:11,186 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 94% 236/250 [09:35<00:35, 2.50s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:11,569 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 95% 237/250 [09:39<00:39, 3.04s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:15,852 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 95% 238/250 [09:42<00:35, 2.93s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:18,518 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 96% 239/250 [09:47<00:38, 3.53s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:23,449 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 96% 240/250 [09:47<00:26, 2.64s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:24,023 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 96% 241/250 [09:49<00:21, 2.43s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:25,976 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 97% 242/250 [09:53<00:23, 2.92s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:30,032 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 97% 243/250 [09:54<00:16, 2.32s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:30,936 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 98% 244/250 [09:55<00:10, 1.80s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:31,533 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 98% 245/250 [09:55<00:06, 1.33s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:31,779 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 98% 246/250 [09:56<00:04, 1.16s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:32,539 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 99% 247/250 [09:59<00:05, 1.70s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:35,501 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 99% 248/250 [10:03<00:04, 2.41s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:39,570 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 100% 249/250 [10:06<00:02, 2.62s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:42,691 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 100% 250/250 [10:08<00:00, 2.43s/it] ***** eval metrics ***** epoch = 1.25 eval_accuracy = 1.0 eval_bleu = 0.0076 eval_gen_len = 12.767 eval_loss = 9.8625 eval_runtime = 0:10:14.04 eval_samples = 2000 eval_samples_per_second = 3.257 eval_steps_per_second = 0.407 INFO:__main__:*** Predict *** [INFO|trainer.py:2964] 2023-02-15 18:32:44,958 >> ***** Running Prediction ***** [INFO|trainer.py:2966] 2023-02-15 18:32:44,959 >> Num examples = 2000 [INFO|trainer.py:2969] 2023-02-15 18:32:44,959 >> Batch size = 8 [INFO|configuration_utils.py:543] 2023-02-15 18:32:44,969 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 0% 0/250 [00:00<?, ?it/s][INFO|configuration_utils.py:543] 2023-02-15 18:32:45,349 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 1% 2/250 [00:02<05:01, 1.22s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:47,784 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 1% 3/250 [00:04<06:16, 1.52s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:49,736 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 2% 4/250 [00:05<06:00, 1.47s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:51,107 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 2% 5/250 [00:10<10:41, 2.62s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:55,887 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 2% 6/250 [00:11<07:57, 1.96s/it][INFO|configuration_utils.py:543] 2023-02-15 18:32:56,510 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 3% 7/250 [00:16<12:04, 2.98s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:01,659 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 3% 8/250 [00:16<09:05, 2.25s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:02,323 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 4% 9/250 [00:17<07:00, 1.75s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:02,943 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 4% 10/250 [00:19<07:15, 1.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:04,913 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 4% 11/250 [00:24<11:14, 2.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:10,025 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 5% 12/250 [00:24<08:05, 2.04s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:10,273 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 5% 13/250 [00:26<07:38, 1.93s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:11,958 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 6% 14/250 [00:30<10:23, 2.64s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:16,237 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 6% 15/250 [00:32<08:57, 2.29s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:17,700 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 6% 16/250 [00:35<09:32, 2.45s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:20,516 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 7% 17/250 [00:37<09:17, 2.39s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:22,780 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 7% 18/250 [00:41<10:44, 2.78s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:26,465 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 8% 19/250 [00:43<10:11, 2.65s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:28,807 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 8% 20/250 [00:48<13:11, 3.44s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:34,100 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 8% 21/250 [00:51<12:39, 3.32s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:37,127 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 9% 22/250 [00:55<13:03, 3.44s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:40,836 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 9% 23/250 [01:00<14:51, 3.93s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:45,911 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 10% 24/250 [01:02<12:41, 3.37s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:47,985 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 10% 25/250 [01:05<11:55, 3.18s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:50,717 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 10% 26/250 [01:08<11:29, 3.08s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:53,563 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 11% 27/250 [01:10<10:53, 2.93s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:56,150 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 11% 28/250 [01:11<08:09, 2.20s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:56,653 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 12% 29/250 [01:12<06:28, 1.76s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:57,366 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 12% 30/250 [01:12<05:34, 1.52s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:58,335 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 12% 31/250 [01:13<04:20, 1.19s/it][INFO|configuration_utils.py:543] 2023-02-15 18:33:58,757 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 13% 32/250 [01:13<03:29, 1.04it/s][INFO|configuration_utils.py:543] 2023-02-15 18:33:59,185 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 13% 33/250 [01:14<03:02, 1.19it/s][INFO|configuration_utils.py:543] 2023-02-15 18:33:59,743 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 14% 34/250 [01:18<06:32, 1.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:03,836 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 14% 35/250 [01:19<05:12, 1.45s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:04,435 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 14% 36/250 [01:20<05:28, 1.54s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:06,173 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 15% 37/250 [01:25<08:27, 2.38s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:10,521 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 15% 38/250 [01:26<06:57, 1.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:11,524 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 16% 39/250 [01:30<09:27, 2.69s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:15,891 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 16% 40/250 [01:31<07:22, 2.11s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:16,639 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 16% 41/250 [01:31<05:34, 1.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:17,067 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 17% 42/250 [01:36<09:11, 2.65s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:22,170 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 17% 43/250 [01:37<06:47, 1.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:22,549 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 18% 44/250 [01:41<08:55, 2.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:26,611 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 18% 45/250 [01:46<11:20, 3.32s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:31,609 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 18% 46/250 [01:48<10:05, 2.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:33,755 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 19% 47/250 [01:52<11:10, 3.30s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:37,840 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 19% 48/250 [01:54<10:06, 3.00s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:40,138 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 20% 49/250 [01:55<08:13, 2.45s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:41,317 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 20% 50/250 [01:56<05:58, 1.79s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:41,565 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 20% 51/250 [02:01<09:17, 2.80s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:46,720 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 21% 52/250 [02:03<08:26, 2.56s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:48,708 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 21% 53/250 [02:05<08:21, 2.55s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:51,233 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 22% 54/250 [02:07<07:21, 2.25s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:52,791 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 22% 55/250 [02:11<08:43, 2.68s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:56,485 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 22% 56/250 [02:11<06:27, 2.00s/it][INFO|configuration_utils.py:543] 2023-02-15 18:34:56,876 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 23% 57/250 [02:15<08:06, 2.52s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:00,623 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 23% 58/250 [02:17<07:25, 2.32s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:02,477 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 24% 59/250 [02:19<07:09, 2.25s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:04,552 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 24% 60/250 [02:23<08:50, 2.79s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:08,623 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 24% 61/250 [02:24<07:08, 2.27s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:09,665 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 25% 62/250 [02:25<05:42, 1.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:10,444 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 25% 63/250 [02:27<06:32, 2.10s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:13,190 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 26% 64/250 [02:31<08:17, 2.68s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:17,210 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 26% 65/250 [02:36<10:00, 3.24s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:21,783 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 26% 66/250 [02:38<08:44, 2.85s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:23,707 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 27% 67/250 [02:38<06:31, 2.14s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:24,185 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 27% 68/250 [02:39<05:12, 1.72s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:24,916 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 28% 69/250 [02:44<07:55, 2.62s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:29,661 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 28% 70/250 [02:46<07:21, 2.45s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:31,708 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 28% 71/250 [02:51<09:45, 3.27s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:36,884 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 29% 72/250 [02:52<07:26, 2.51s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:37,621 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 29% 73/250 [02:53<05:51, 1.99s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:38,392 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 30% 74/250 [02:54<05:10, 1.77s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:39,639 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 30% 75/250 [02:57<06:16, 2.15s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:42,698 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 30% 76/250 [02:58<05:18, 1.83s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:43,774 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 31% 77/250 [02:58<04:06, 1.42s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:44,251 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 31% 78/250 [03:00<03:58, 1.39s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:45,555 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 32% 79/250 [03:02<05:05, 1.79s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:48,277 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 32% 80/250 [03:03<04:05, 1.45s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:48,921 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 32% 81/250 [03:04<03:30, 1.25s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:49,702 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 33% 82/250 [03:07<04:54, 1.75s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:52,641 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 33% 83/250 [03:07<03:43, 1.34s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:53,019 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 34% 84/250 [03:08<03:23, 1.23s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:53,982 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 34% 85/250 [03:12<05:20, 1.94s/it][INFO|configuration_utils.py:543] 2023-02-15 18:35:57,590 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 34% 86/250 [03:17<07:38, 2.80s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:02,377 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 35% 87/250 [03:19<07:43, 2.84s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:05,336 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 35% 88/250 [03:22<07:01, 2.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:07,378 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 36% 89/250 [03:25<07:45, 2.89s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:10,941 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 36% 90/250 [03:26<05:52, 2.20s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:11,542 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 36% 91/250 [03:30<07:20, 2.77s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:15,628 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 37% 92/250 [03:32<07:04, 2.69s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:18,125 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 37% 93/250 [03:34<06:02, 2.31s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:19,543 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 38% 94/250 [03:35<04:53, 1.88s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:20,435 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 38% 95/250 [03:39<07:04, 2.74s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:25,178 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 38% 96/250 [03:44<08:40, 3.38s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:30,052 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 39% 97/250 [03:47<07:59, 3.14s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:32,615 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 39% 98/250 [03:49<07:37, 3.01s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:35,322 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 40% 99/250 [03:50<05:36, 2.23s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:35,742 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 40% 100/250 [03:50<04:20, 1.74s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:36,336 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 40% 101/250 [03:56<06:49, 2.75s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:41,429 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 41% 102/250 [04:00<08:03, 3.26s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:45,905 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 41% 103/250 [04:00<05:53, 2.40s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:46,292 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 42% 104/250 [04:01<04:19, 1.78s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:46,622 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 42% 105/250 [04:04<05:08, 2.13s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:49,563 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 42% 106/250 [04:07<05:36, 2.34s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:52,394 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 43% 107/250 [04:09<05:19, 2.23s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:54,376 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 43% 108/250 [04:14<07:29, 3.16s/it][INFO|configuration_utils.py:543] 2023-02-15 18:36:59,715 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 44% 109/250 [04:16<06:40, 2.84s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:01,799 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 44% 110/250 [04:18<05:49, 2.50s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:03,499 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 44% 111/250 [04:19<04:56, 2.14s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:04,791 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 45% 112/250 [04:19<03:43, 1.62s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:05,212 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 45% 113/250 [04:24<06:05, 2.67s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:10,328 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 46% 114/250 [04:26<05:34, 2.46s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:12,296 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 46% 115/250 [04:28<04:36, 2.05s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:13,389 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 46% 116/250 [04:28<03:27, 1.55s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:13,762 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 47% 117/250 [04:31<04:14, 1.92s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:16,540 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 47% 118/250 [04:35<05:39, 2.57s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:20,649 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 48% 119/250 [04:37<05:12, 2.39s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:22,601 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 48% 120/250 [04:40<05:54, 2.73s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:26,132 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 48% 121/250 [04:42<05:31, 2.57s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:28,320 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 49% 122/250 [04:44<04:54, 2.30s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:30,008 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 49% 123/250 [04:48<05:38, 2.66s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:33,509 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 50% 124/250 [04:48<04:11, 1.99s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:33,942 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 50% 125/250 [04:49<03:12, 1.54s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:34,422 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 50% 126/250 [04:51<03:55, 1.90s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:37,172 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 51% 127/250 [04:54<04:07, 2.01s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:39,434 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 51% 128/250 [04:55<03:58, 1.96s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:41,269 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 52% 129/250 [04:59<05:12, 2.58s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:45,310 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 52% 130/250 [05:00<04:12, 2.10s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:46,299 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 52% 131/250 [05:06<05:58, 3.01s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:51,433 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 53% 132/250 [05:08<05:30, 2.80s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:53,746 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 53% 133/250 [05:13<06:49, 3.50s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:58,884 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 54% 134/250 [05:14<05:06, 2.64s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:59,507 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 54% 135/250 [05:14<03:46, 1.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:37:59,924 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 54% 136/250 [05:17<04:11, 2.21s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:02,673 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 55% 137/250 [05:18<03:50, 2.04s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:04,316 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 55% 138/250 [05:21<03:59, 2.14s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:06,699 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 56% 139/250 [05:23<04:05, 2.21s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:09,077 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 56% 140/250 [05:26<04:32, 2.48s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:12,172 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 56% 141/250 [05:30<05:10, 2.85s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:15,883 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 57% 142/250 [05:34<05:42, 3.17s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:19,806 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 57% 143/250 [05:38<06:16, 3.52s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:24,145 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 58% 144/250 [05:40<05:13, 2.96s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:25,783 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 58% 145/250 [05:41<04:13, 2.42s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:26,937 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 58% 146/250 [05:44<04:36, 2.66s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:30,165 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 59% 147/250 [05:45<03:35, 2.09s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:30,935 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 59% 148/250 [05:50<05:04, 2.99s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:36,007 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 60% 149/250 [05:52<04:25, 2.63s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:37,806 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 60% 150/250 [05:53<03:40, 2.21s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:39,025 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 60% 151/250 [05:58<05:04, 3.08s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:44,143 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 61% 152/250 [05:59<03:46, 2.31s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:44,650 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 61% 153/250 [06:03<04:51, 3.00s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:49,274 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 62% 154/250 [06:04<03:37, 2.27s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:49,834 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 62% 155/250 [06:09<04:56, 3.12s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:54,925 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 62% 156/250 [06:09<03:35, 2.30s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:55,312 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 63% 157/250 [06:10<02:40, 1.72s/it][INFO|configuration_utils.py:543] 2023-02-15 18:38:55,695 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 63% 158/250 [06:15<04:11, 2.74s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:00,794 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 64% 159/250 [06:19<04:53, 3.22s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:05,153 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 64% 160/250 [06:20<03:33, 2.37s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:05,528 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 64% 161/250 [06:20<02:37, 1.77s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:05,906 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 65% 162/250 [06:21<02:18, 1.58s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:07,035 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 65% 163/250 [06:24<02:51, 1.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:09,927 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 66% 164/250 [06:24<02:08, 1.50s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:10,311 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 66% 165/250 [06:28<02:50, 2.00s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:13,498 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 66% 166/250 [06:30<02:45, 1.97s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:15,390 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 67% 167/250 [06:31<02:23, 1.73s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:16,571 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 67% 168/250 [06:36<03:41, 2.70s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:21,515 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 68% 169/250 [06:38<03:40, 2.73s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:24,309 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 68% 170/250 [06:43<04:11, 3.14s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:28,427 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 68% 171/250 [06:45<03:57, 3.01s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:31,112 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 69% 172/250 [06:50<04:23, 3.38s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:35,352 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 69% 173/250 [06:50<03:15, 2.54s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:35,949 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 70% 174/250 [06:50<02:22, 1.87s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:36,251 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 70% 175/250 [06:56<03:33, 2.85s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:41,391 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 70% 176/250 [06:57<03:09, 2.56s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:43,286 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 71% 177/250 [07:00<03:07, 2.57s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:45,872 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 71% 178/250 [07:01<02:24, 2.01s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:46,561 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 72% 179/250 [07:06<03:22, 2.85s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:51,366 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 72% 180/250 [07:09<03:35, 3.08s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:54,985 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 72% 181/250 [07:11<03:08, 2.73s/it][INFO|configuration_utils.py:543] 2023-02-15 18:39:56,909 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 73% 182/250 [07:16<03:55, 3.47s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:02,094 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 73% 183/250 [07:18<03:14, 2.90s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:03,655 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 74% 184/250 [07:19<02:34, 2.34s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:04,690 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 74% 185/250 [07:24<03:18, 3.06s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:09,424 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 74% 186/250 [07:28<03:49, 3.58s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:14,235 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 75% 187/250 [07:34<04:15, 4.06s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:19,404 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 75% 188/250 [07:35<03:28, 3.37s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:21,160 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 76% 189/250 [07:36<02:45, 2.71s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:22,329 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 76% 190/250 [07:37<02:06, 2.10s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:23,011 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 76% 191/250 [07:37<01:31, 1.56s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:23,302 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 77% 192/250 [07:39<01:35, 1.65s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:25,170 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 77% 193/250 [07:43<02:06, 2.23s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:28,744 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 78% 194/250 [07:45<01:58, 2.11s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:30,573 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 78% 195/250 [07:49<02:30, 2.73s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:34,755 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 78% 196/250 [07:50<02:01, 2.25s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:35,871 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 79% 197/250 [07:55<02:44, 3.11s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:40,994 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 79% 198/250 [07:59<02:51, 3.30s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:44,750 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 80% 199/250 [08:02<02:39, 3.14s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:47,493 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 80% 200/250 [08:07<03:06, 3.72s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:52,589 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 80% 201/250 [08:11<03:03, 3.74s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:56,374 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 81% 202/250 [08:11<02:09, 2.70s/it][INFO|configuration_utils.py:543] 2023-02-15 18:40:56,643 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 81% 203/250 [08:15<02:33, 3.27s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:01,243 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 82% 204/250 [08:19<02:34, 3.36s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:04,797 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 82% 205/250 [08:21<02:14, 2.99s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:06,925 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 82% 206/250 [08:25<02:27, 3.36s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:11,148 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 83% 207/250 [08:26<01:50, 2.58s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:11,912 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 83% 208/250 [08:31<02:14, 3.21s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:16,587 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 84% 209/250 [08:33<01:55, 2.82s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:18,489 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 84% 210/250 [08:34<01:35, 2.39s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:19,893 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 84% 211/250 [08:38<01:54, 2.93s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:24,068 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 85% 212/250 [08:40<01:41, 2.66s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:26,105 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 85% 213/250 [08:45<01:57, 3.17s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:30,455 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 86% 214/250 [08:45<01:24, 2.36s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:30,918 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 86% 215/250 [08:50<01:44, 3.00s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:35,410 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 86% 216/250 [08:52<01:31, 2.69s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:37,400 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 87% 217/250 [08:52<01:08, 2.06s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:37,988 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 87% 218/250 [08:53<00:50, 1.57s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:38,416 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 88% 219/250 [08:54<00:42, 1.38s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:39,362 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 88% 220/250 [08:54<00:35, 1.17s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:40,044 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 88% 221/250 [08:55<00:27, 1.07it/s][INFO|configuration_utils.py:543] 2023-02-15 18:41:40,422 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 89% 222/250 [08:56<00:31, 1.11s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:41,937 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 89% 223/250 [08:57<00:31, 1.18s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:43,278 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 90% 224/250 [08:59<00:34, 1.33s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:44,963 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 90% 225/250 [09:00<00:26, 1.07s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:45,422 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 90% 226/250 [09:02<00:34, 1.45s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:47,759 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 91% 227/250 [09:06<00:49, 2.16s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:51,583 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 91% 228/250 [09:06<00:35, 1.63s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:51,959 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 92% 229/250 [09:10<00:50, 2.38s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:56,113 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 92% 230/250 [09:12<00:42, 2.12s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:57,622 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 92% 231/250 [09:12<00:31, 1.65s/it][INFO|configuration_utils.py:543] 2023-02-15 18:41:58,182 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 93% 232/250 [09:17<00:46, 2.60s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:02,986 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 93% 233/250 [09:20<00:47, 2.81s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:06,300 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 94% 234/250 [09:22<00:39, 2.49s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:08,046 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 94% 235/250 [09:23<00:32, 2.14s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:09,348 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 94% 236/250 [09:29<00:42, 3.03s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:14,456 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 95% 237/250 [09:30<00:32, 2.50s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:15,741 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 95% 238/250 [09:30<00:22, 1.87s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:16,133 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 96% 239/250 [09:31<00:16, 1.51s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:16,810 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 96% 240/250 [09:34<00:18, 1.85s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:19,460 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 96% 241/250 [09:34<00:13, 1.48s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:20,072 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 97% 242/250 [09:37<00:14, 1.75s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:22,455 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 97% 243/250 [09:40<00:15, 2.16s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:25,569 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 98% 244/250 [09:43<00:14, 2.35s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:28,357 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 98% 245/250 [09:45<00:11, 2.26s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:30,426 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 98% 246/250 [09:47<00:08, 2.24s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:32,609 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 99% 247/250 [09:49<00:07, 2.34s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:35,183 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 99% 248/250 [09:51<00:04, 2.21s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:37,079 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 100% 249/250 [09:53<00:02, 2.18s/it][INFO|configuration_utils.py:543] 2023-02-15 18:42:39,206 >> Generate config GenerationConfig { "decoder_start_token_id": 0, "eos_token_id": 1, "pad_token_id": 0, "transformers_version": "4.26.1" } 100% 250/250 [09:56<00:00, 2.39s/it] ***** predict metrics ***** predict_accuracy = 1.0 predict_bleu = 0.0081 predict_gen_len = 12.2785 predict_loss = 9.8709 predict_runtime = 0:09:57.12 predict_samples = 2000 predict_samples_per_second = 3.349 predict_steps_per_second = 0.419 [INFO|modelcard.py:449] 2023-02-15 18:42:42,406 >> Dropping the following result as it does not have all the necessary fields: {'task': {'name': 'Translation', 'type': 'translation'}, 'metrics': [{'name': 'Bleu', 'type': 'bleu', 'value': 0.0076}, {'name': 'Accuracy', 'type': 'accuracy', 'value': 1.0}]}
FLAN T5
from transformers import pipeline, AutoTokenizer, AutoModelForSeq2SeqLM
import json
if torch.cuda.is_available():
device = 0
else:
device = -1
def perform_shot_learning(pipeline_type, model_name, test_file):
class_type = AutoModelForSeq2SeqLM
model = class_type.from_pretrained(model_name, torch_dtype=torch.float32)
tokenizer = AutoTokenizer.from_pretrained(model_name)
our_pipeline = pipeline(pipeline_type, model=model, tokenizer=tokenizer, device=device)
correct = 0
labels = "possible labels: sadness, joy, love, anger, fear, surprise"
with open(test_file) as f:
f_lines = f.readlines()
for line in f_lines:
ex = json.loads(line)
prompt = ex['text']
tmp = labels + '\n' + f'text: {prompt}' + '\n' + 'label: '
predict = our_pipeline(tmp, do_sample=False)[0]['generated_text']
if predict == ex['label']:
correct += 1
print(f'Accuracy: {correct/len(f_lines)}')
test_ds = 'data/s2s-test.json'
perform_shot_learning('text2text-generation', 'google/flan-t5-large', test_ds)
/usr/local/lib/python3.8/dist-packages/transformers/pipelines/base.py:1043: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset warnings.warn(
Accuracy: 0.647
!zip -r /content/training_cache_roberta_gpt.zip /content/roberta_gpt_cache
!zip -r /content/roberta_gpt_out.zip /content/roberta_gpt_out
adding: content/roberta_gpt_cache/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/roberta_gpt_cache_roberta_custom_json_default-2fc7d0d25bce81a9_0.0.0_0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51.lock (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/tokenizer.json (deflated 59%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/config.json (deflated 47%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/merges.txt (deflated 53%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/vocab.json (deflated 63%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/snapshots/ff46155979338ff8063cdad90908b498ab91b181/pytorch_model.bin (deflated 41%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/blobs/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/blobs/8db5e7ac5bfc9ec8b613b776009300fe3685d957 (deflated 47%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/blobs/ad0bcbeb288f0d1373d88e0762e66357f55b8311 (deflated 59%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/blobs/278b7a95739c4392fae9b818bb5343dde20be1b89318f37a6d939e1e1b9e461b (deflated 41%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/blobs/226b0752cac7789c48f0cb3ec53eda48b7be36cc (deflated 53%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/blobs/5606f48548d99a9829d10a96cd364b816b02cd21 (deflated 63%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/refs/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/refs/main (deflated 3%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/.no_exist/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/.no_exist/ff46155979338ff8063cdad90908b498ab91b181/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/.no_exist/ff46155979338ff8063cdad90908b498ab91b181/special_tokens_map.json (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/.no_exist/ff46155979338ff8063cdad90908b498ab91b181/added_tokens.json (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/models--roberta-base/.no_exist/ff46155979338ff8063cdad90908b498ab91b181/tokenizer_config.json (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/json/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51.incomplete_info.lock (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/ (stored 0%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-7f5564bde85bb8c0.arrow (deflated 88%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/json-train.arrow (deflated 64%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-de6cef99ad203f9d.arrow (deflated 88%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/json-test.arrow (deflated 64%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/json-validation.arrow (deflated 64%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/dataset_info.json (deflated 57%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-138d491b60344550.arrow (deflated 88%) adding: content/roberta_gpt_cache/roberta_custom/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51_builder.lock (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/tokenizer.json (deflated 59%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/config.json (deflated 50%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/merges.txt (deflated 53%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/vocab.json (deflated 67%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/snapshots/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/pytorch_model.bin (deflated 16%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/blobs/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/blobs/1f1d9aaca301414e7f6c9396df506798ff4eb9a6 (deflated 67%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/blobs/4b988bccc9dc5adacd403c00b4704976196548f8 (deflated 59%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/blobs/10c66461e4c109db5a2196bff4bb59be30396ed8 (deflated 50%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/blobs/226b0752cac7789c48f0cb3ec53eda48b7be36cc (deflated 53%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/blobs/7c5d3f4b8b76583b422fcb9189ad6c89d5d97a094541ce8932dce3ecabde1421 (deflated 16%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/refs/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/refs/main (deflated 3%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/.no_exist/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/.no_exist/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/.no_exist/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/special_tokens_map.json (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/.no_exist/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/added_tokens.json (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/models--gpt2/.no_exist/e7da7f221d5bf496a48136c0cd264e630fe9fcc8/tokenizer_config.json (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/roberta_gpt_cache_custom_gpt2_json_default-2fc7d0d25bce81a9_0.0.0_0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51.lock (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/json/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51.incomplete_info.lock (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/ (stored 0%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/json-train.arrow (deflated 64%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-d96eda4fa0456cad.arrow (deflated 88%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/json-test.arrow (deflated 64%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-70f9aa9f07dd2d49.arrow (deflated 88%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/json-validation.arrow (deflated 64%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/dataset_info.json (deflated 57%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-bea742f4a59644e1.arrow (deflated 88%) adding: content/roberta_gpt_cache/custom_gpt2/json/default-2fc7d0d25bce81a9/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51_builder.lock (stored 0%) adding: content/roberta_gpt_out/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/train_results.json (deflated 40%) adding: content/roberta_gpt_out/gpt2_custom/trainer_state.json (deflated 81%) adding: content/roberta_gpt_out/gpt2_custom/tokenizer.json (deflated 72%) adding: content/roberta_gpt_out/gpt2_custom/README.md (deflated 54%) adding: content/roberta_gpt_out/gpt2_custom/all_results.json (deflated 57%) adding: content/roberta_gpt_out/gpt2_custom/special_tokens_map.json (deflated 60%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/trainer_state.json (deflated 81%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/tokenizer.json (deflated 72%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/special_tokens_map.json (deflated 60%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/config.json (deflated 56%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/scheduler.pt (deflated 50%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/merges.txt (deflated 53%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/vocab.json (deflated 59%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/optimizer.pt (deflated 30%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/tokenizer_config.json (deflated 41%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/pytorch_model.bin (deflated 9%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/rng_state.pth (deflated 28%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2500/training_args.bin (deflated 49%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/trainer_state.json (deflated 75%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/tokenizer.json (deflated 72%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/special_tokens_map.json (deflated 60%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/config.json (deflated 56%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/scheduler.pt (deflated 49%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/merges.txt (deflated 53%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/vocab.json (deflated 59%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/optimizer.pt (deflated 30%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/tokenizer_config.json (deflated 41%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/pytorch_model.bin (deflated 9%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/rng_state.pth (deflated 28%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1000/training_args.bin (deflated 49%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/trainer_state.json (deflated 77%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/tokenizer.json (deflated 72%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/special_tokens_map.json (deflated 60%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/config.json (deflated 56%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/scheduler.pt (deflated 49%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/merges.txt (deflated 53%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/vocab.json (deflated 59%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/optimizer.pt (deflated 30%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/tokenizer_config.json (deflated 41%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/pytorch_model.bin (deflated 9%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/rng_state.pth (deflated 28%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-1500/training_args.bin (deflated 49%) adding: content/roberta_gpt_out/gpt2_custom/runs/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/runs/Feb15_16-43-51_b7fb20e65b38/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/runs/Feb15_16-43-51_b7fb20e65b38/events.out.tfevents.1676479446.b7fb20e65b38.18625.0 (deflated 63%) adding: content/roberta_gpt_out/gpt2_custom/runs/Feb15_16-43-51_b7fb20e65b38/1676479446.8678486/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/runs/Feb15_16-43-51_b7fb20e65b38/1676479446.8678486/events.out.tfevents.1676479446.b7fb20e65b38.18625.1 (deflated 63%) adding: content/roberta_gpt_out/gpt2_custom/runs/Feb15_16-43-51_b7fb20e65b38/events.out.tfevents.1676479939.b7fb20e65b38.18625.2 (deflated 28%) adding: content/roberta_gpt_out/gpt2_custom/config.json (deflated 56%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/trainer_state.json (deflated 79%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/tokenizer.json (deflated 72%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/special_tokens_map.json (deflated 60%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/config.json (deflated 56%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/scheduler.pt (deflated 49%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/merges.txt (deflated 53%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/vocab.json (deflated 59%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/optimizer.pt (deflated 30%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/tokenizer_config.json (deflated 41%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/pytorch_model.bin (deflated 9%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/rng_state.pth (deflated 28%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-2000/training_args.bin (deflated 49%) adding: content/roberta_gpt_out/gpt2_custom/merges.txt (deflated 53%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/ (stored 0%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/trainer_state.json (deflated 67%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/tokenizer.json (deflated 72%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/special_tokens_map.json (deflated 60%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/config.json (deflated 56%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/scheduler.pt (deflated 49%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/merges.txt (deflated 53%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/vocab.json (deflated 59%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/optimizer.pt (deflated 31%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/tokenizer_config.json (deflated 41%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/pytorch_model.bin (deflated 9%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/rng_state.pth (deflated 28%) adding: content/roberta_gpt_out/gpt2_custom/checkpoint-500/training_args.bin (deflated 49%) adding: content/roberta_gpt_out/gpt2_custom/vocab.json (deflated 59%) adding: content/roberta_gpt_out/gpt2_custom/tokenizer_config.json (deflated 41%) adding: content/roberta_gpt_out/gpt2_custom/predict_results_None.txt (deflated 62%) adding: content/roberta_gpt_out/gpt2_custom/eval_results.json (deflated 42%) adding: content/roberta_gpt_out/gpt2_custom/pytorch_model.bin (deflated 9%) adding: content/roberta_gpt_out/gpt2_custom/training_args.bin (deflated 49%) adding: content/roberta_gpt_out/roberta_custom/ (stored 0%) adding: content/roberta_gpt_out/roberta_custom/train_results.json (deflated 41%) adding: content/roberta_gpt_out/roberta_custom/trainer_state.json (deflated 54%) adding: content/roberta_gpt_out/roberta_custom/tokenizer.json (deflated 72%) adding: content/roberta_gpt_out/roberta_custom/README.md (deflated 46%) adding: content/roberta_gpt_out/roberta_custom/all_results.json (deflated 56%) adding: content/roberta_gpt_out/roberta_custom/special_tokens_map.json (deflated 52%) adding: content/roberta_gpt_out/roberta_custom/runs/ (stored 0%) adding: content/roberta_gpt_out/roberta_custom/runs/Feb15_16-41-51_b7fb20e65b38/ (stored 0%) adding: content/roberta_gpt_out/roberta_custom/runs/Feb15_16-41-51_b7fb20e65b38/1676479330.978654/ (stored 0%) adding: content/roberta_gpt_out/roberta_custom/runs/Feb15_16-41-51_b7fb20e65b38/1676479330.978654/events.out.tfevents.1676479330.b7fb20e65b38.18052.1 (deflated 63%) adding: content/roberta_gpt_out/roberta_custom/runs/Feb15_16-41-51_b7fb20e65b38/events.out.tfevents.1676479330.b7fb20e65b38.18052.0 (deflated 58%) adding: content/roberta_gpt_out/roberta_custom/runs/Feb15_16-41-51_b7fb20e65b38/events.out.tfevents.1676479421.b7fb20e65b38.18052.2 (deflated 28%) adding: content/roberta_gpt_out/roberta_custom/config.json (deflated 55%) adding: content/roberta_gpt_out/roberta_custom/merges.txt (deflated 53%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/ (stored 0%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/trainer_state.json (deflated 46%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/tokenizer.json (deflated 72%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/special_tokens_map.json (deflated 52%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/config.json (deflated 55%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/scheduler.pt (deflated 49%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/merges.txt (deflated 53%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/vocab.json (deflated 59%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/optimizer.pt (deflated 23%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/tokenizer_config.json (deflated 48%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/pytorch_model.bin (deflated 17%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/rng_state.pth (deflated 28%) adding: content/roberta_gpt_out/roberta_custom/checkpoint-500/training_args.bin (deflated 49%) adding: content/roberta_gpt_out/roberta_custom/vocab.json (deflated 59%) adding: content/roberta_gpt_out/roberta_custom/tokenizer_config.json (deflated 48%) adding: content/roberta_gpt_out/roberta_custom/predict_results_None.txt (deflated 62%) adding: content/roberta_gpt_out/roberta_custom/eval_results.json (deflated 42%) adding: content/roberta_gpt_out/roberta_custom/pytorch_model.bin (deflated 17%) adding: content/roberta_gpt_out/roberta_custom/training_args.bin (deflated 49%)
!zip -r /content/training_cache_t5_large.zip /content/t5_large_cache
!zip -r /content/t5_large_out.zip /content/t5_large_out
adding: content/t5_large_cache/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/t5_large_cache_t5_v1_1_json_default-5ed1982b30e343c8_0.0.0_0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51.lock (stored 0%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/snapshots/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/snapshots/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/snapshots/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/generation_config.json (deflated 27%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/snapshots/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/special_tokens_map.json (deflated 83%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/snapshots/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/config.json (deflated 43%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/snapshots/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/tokenizer_config.json (deflated 82%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/snapshots/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/pytorch_model.bin (deflated 53%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/snapshots/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/spiece.model (deflated 48%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/blobs/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/blobs/329243624cf70001991b9f0410d222a618bd33188eadc9890259b60cbc78f944 (deflated 53%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/blobs/4e28ff6ebdf584f5372d9de68867399142435d9a (deflated 48%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/blobs/d52815623b46b7db1c4b957b5a83a8ad30b0146a (deflated 27%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/blobs/b7a0675689c9da2e2e4a6ab3ada2886fa907d56f (deflated 43%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/blobs/b114c318caf72f6e89ea92e0755c41327a453198 (deflated 82%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/blobs/881bdbffc06e471924ecea57f962bc5f8e2a9f21 (deflated 83%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/refs/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/refs/main (deflated 3%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/.no_exist/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/.no_exist/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/.no_exist/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/tokenizer.json (stored 0%) adding: content/t5_large_cache/t5_v1_1/models--google--t5-v1_1-large/.no_exist/a98b0fcd0b8137ded40cdf0c0cf0ee884e7c9726/added_tokens.json (stored 0%) adding: content/t5_large_cache/t5_v1_1/json/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51.incomplete_info.lock (stored 0%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/ (stored 0%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/json-train.arrow (deflated 62%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-3ade70e6a4c3e151.arrow (deflated 74%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-85561eabafb886e8.arrow (deflated 74%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/json-test.arrow (deflated 62%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/json-validation.arrow (deflated 62%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/cache-2d36083b35a3e0a9.arrow (deflated 74%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51/dataset_info.json (deflated 58%) adding: content/t5_large_cache/t5_v1_1/json/default-5ed1982b30e343c8/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51_builder.lock (stored 0%) adding: content/t5_large_out/ (stored 0%) adding: content/t5_large_out/t5_v1_1/ (stored 0%) adding: content/t5_large_out/t5_v1_1/generation_config.json (deflated 29%) adding: content/t5_large_out/t5_v1_1/train_results.json (deflated 40%) adding: content/t5_large_out/t5_v1_1/trainer_state.json (deflated 82%) adding: content/t5_large_out/t5_v1_1/predict_results.json (deflated 49%) adding: content/t5_large_out/t5_v1_1/tokenizer.json (deflated 75%) adding: content/t5_large_out/t5_v1_1/generated_predictions.txt (deflated 82%) adding: content/t5_large_out/t5_v1_1/README.md (deflated 57%) adding: content/t5_large_out/t5_v1_1/all_results.json (deflated 65%) adding: content/t5_large_out/t5_v1_1/special_tokens_map.json (deflated 86%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/ (stored 0%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/generation_config.json (deflated 29%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/trainer_state.json (deflated 82%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/tokenizer.json (deflated 75%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/special_tokens_map.json (deflated 86%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/config.json (deflated 47%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/scheduler.pt (deflated 50%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/optimizer.pt (deflated 12%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/tokenizer_config.json (deflated 80%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/pytorch_model.bin (deflated 26%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/rng_state.pth (deflated 28%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/spiece.model (deflated 48%) adding: content/t5_large_out/t5_v1_1/checkpoint-2500/training_args.bin (deflated 49%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/ (stored 0%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/generation_config.json (deflated 29%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/trainer_state.json (deflated 76%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/tokenizer.json (deflated 75%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/special_tokens_map.json (deflated 86%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/config.json (deflated 47%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/scheduler.pt (deflated 49%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/optimizer.pt (deflated 12%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/tokenizer_config.json (deflated 80%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/pytorch_model.bin (deflated 26%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/rng_state.pth (deflated 28%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/spiece.model (deflated 48%) adding: content/t5_large_out/t5_v1_1/checkpoint-1000/training_args.bin (deflated 49%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/ (stored 0%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/generation_config.json (deflated 29%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/trainer_state.json (deflated 79%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/tokenizer.json (deflated 75%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/special_tokens_map.json (deflated 86%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/config.json (deflated 47%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/scheduler.pt (deflated 49%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/optimizer.pt (deflated 12%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/tokenizer_config.json (deflated 80%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/pytorch_model.bin (deflated 26%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/rng_state.pth (deflated 28%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/spiece.model (deflated 48%) adding: content/t5_large_out/t5_v1_1/checkpoint-1500/training_args.bin (deflated 49%) adding: content/t5_large_out/t5_v1_1/runs/ (stored 0%) adding: content/t5_large_out/t5_v1_1/runs/Feb15_17-36-52_b7fb20e65b38/ (stored 0%) adding: content/t5_large_out/t5_v1_1/runs/Feb15_17-36-52_b7fb20e65b38/1676482651.1311803/ (stored 0%) adding: content/t5_large_out/t5_v1_1/runs/Feb15_17-36-52_b7fb20e65b38/1676482651.1311803/events.out.tfevents.1676482651.b7fb20e65b38.31799.1 (deflated 63%) adding: content/t5_large_out/t5_v1_1/runs/Feb15_17-36-52_b7fb20e65b38/events.out.tfevents.1676485964.b7fb20e65b38.31799.2 (deflated 34%) adding: content/t5_large_out/t5_v1_1/runs/Feb15_17-36-52_b7fb20e65b38/events.out.tfevents.1676482651.b7fb20e65b38.31799.0 (deflated 64%) adding: content/t5_large_out/t5_v1_1/config.json (deflated 47%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/ (stored 0%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/generation_config.json (deflated 29%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/trainer_state.json (deflated 81%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/tokenizer.json (deflated 75%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/special_tokens_map.json (deflated 86%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/config.json (deflated 47%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/scheduler.pt (deflated 49%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/optimizer.pt (deflated 12%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/tokenizer_config.json (deflated 80%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/pytorch_model.bin (deflated 26%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/rng_state.pth (deflated 28%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/spiece.model (deflated 48%) adding: content/t5_large_out/t5_v1_1/checkpoint-2000/training_args.bin (deflated 49%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/ (stored 0%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/generation_config.json (deflated 29%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/trainer_state.json (deflated 68%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/tokenizer.json (deflated 75%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/special_tokens_map.json (deflated 86%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/config.json (deflated 47%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/scheduler.pt (deflated 49%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/optimizer.pt (deflated 12%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/tokenizer_config.json (deflated 80%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/pytorch_model.bin (deflated 26%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/rng_state.pth (deflated 28%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/spiece.model (deflated 48%) adding: content/t5_large_out/t5_v1_1/checkpoint-500/training_args.bin (deflated 49%) adding: content/t5_large_out/t5_v1_1/tokenizer_config.json (deflated 80%) adding: content/t5_large_out/t5_v1_1/eval_results.json (deflated 45%) adding: content/t5_large_out/t5_v1_1/pytorch_model.bin (deflated 26%) adding: content/t5_large_out/t5_v1_1/spiece.model (deflated 48%) adding: content/t5_large_out/t5_v1_1/training_args.bin (deflated 49%)
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive