82 KiB
82 KiB
! python3 -m pip install transformersLooking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Collecting transformers Downloading transformers-4.20.1-py3-none-any.whl (4.4 MB) [K |████████████████████████████████| 4.4 MB 25.8 MB/s [?25hCollecting huggingface-hub<1.0,>=0.1.0 Downloading huggingface_hub-0.8.1-py3-none-any.whl (101 kB) [K |████████████████████████████████| 101 kB 14.1 MB/s [?25hRequirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2022.6.2) Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers) (4.64.0) Collecting tokenizers!=0.11.3,<0.13,>=0.11.1 Downloading tokenizers-0.12.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB) [K |████████████████████████████████| 6.6 MB 56.5 MB/s [?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (1.21.6) Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers) (4.11.4) Collecting pyyaml>=5.1 Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB) [K |████████████████████████████████| 596 kB 41.7 MB/s [?25hRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (21.3) Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.7.1) Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0,>=0.1.0->transformers) (4.1.1) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->transformers) (3.0.9) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers) (3.8.0) Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (3.0.4) Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2.10) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2022.6.15) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (1.24.3) Installing collected packages: pyyaml, tokenizers, huggingface-hub, transformers Attempting uninstall: pyyaml Found existing installation: PyYAML 3.13 Uninstalling PyYAML-3.13: Successfully uninstalled PyYAML-3.13 Successfully installed huggingface-hub-0.8.1 pyyaml-6.0 tokenizers-0.12.1 transformers-4.20.1
import csv, lzma
# Reads input from directory and returns a list
def io_read(dir):
X = []
if 'xz' in dir:
with lzma.open(dir) as f:
for line in f:
text = line.decode('utf-8')
text = text.split('\t')
X.append(text)
else:
with open(dir) as f:
for line in f:
X.append(line.replace('\n', ''))
return X
# Takes the output (list) and writes it into directory
def io_write(output, dir):
with open(dir, 'w', newline='') as f:
writer = csv.writer(f)
writer.writerows(output)x_train = [x[0] for x in io_read('/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/train/in.tsv.xz')]
y_train = [int(y) for y in io_read('/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/train/expected.tsv')]
# x_dev = [x[0] for x in io_read('/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/dev-0/in.tsv.xz')]
# y_dev = [int(y) for y in io_read('/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/dev-0/expected.tsv')]
# x_test = [x[0] for x in io_read('/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/test-A/in.tsv.xz')]from transformers import BertTokenizerFast
model_name = "google/bert_uncased_L-4_H-512_A-8"
tokenizer = BertTokenizerFast.from_pretrained(model_name)
train_encodings = tokenizer(x_train, truncation=True, padding='max_length', max_length=128)import torch
class Dataset(torch.utils.data.Dataset):
def __init__(self, encodings, labels=None):
self.encodings = encodings
self.labels = labels
def __getitem__(self, idx):
item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
if self.labels:
item["labels"] = torch.tensor(self.labels[idx])
return item
def __len__(self):
return len(self.encodings["input_ids"])train_dataset = Dataset(train_encodings, y_train)from transformers import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained(model_name)Downloading: 0%| | 0.00/111M [00:00<?, ?B/s]
Some weights of the model checkpoint at google/bert_uncased_L-4_H-512_A-8 were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.decoder.bias', 'cls.seq_relationship.weight', 'cls.predictions.bias', 'cls.predictions.decoder.weight'] - This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). - This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Some weights of BertForSequenceClassification were not initialized from the model checkpoint at google/bert_uncased_L-4_H-512_A-8 and are newly initialized: ['classifier.bias', 'classifier.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
for param in model.base_model.parameters():
param.requires_grad = Falsefrom transformers import TrainingArguments, Trainer
training_args = TrainingArguments(
output_dir="./results",
learning_rate=2e-5,
per_device_train_batch_size=16,
num_train_epochs=3,
weight_decay=0.01,
warmup_steps=500,
save_total_limit=1
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset
)PyTorch: setting up devices The default value for the training argument `--report_to` will change in v5 (from all installed integrations to none). In v5, you will need to use `--report_to all` to get the same behavior as now. You should start updating your code and make this info disappear :-).
trainer.train()/usr/local/lib/python3.7/dist-packages/transformers/optimization.py:310: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning FutureWarning, ***** Running training ***** Num examples = 289579 Num Epochs = 3 Instantaneous batch size per device = 16 Total train batch size (w. parallel, distributed & accumulation) = 16 Gradient Accumulation steps = 1 Total optimization steps = 54297
[54297/54297 21:16, Epoch 3/3]
| Step | Training Loss |
|---|---|
| 500 | 0.552400 |
| 1000 | 0.542700 |
| 1500 | 0.552900 |
| 2000 | 0.549600 |
| 2500 | 0.548200 |
| 3000 | 0.546700 |
| 3500 | 0.546100 |
| 4000 | 0.545600 |
| 4500 | 0.544800 |
| 5000 | 0.546000 |
| 5500 | 0.545800 |
| 6000 | 0.541200 |
| 6500 | 0.536100 |
| 7000 | 0.537200 |
| 7500 | 0.545700 |
| 8000 | 0.543100 |
| 8500 | 0.548600 |
| 9000 | 0.538600 |
| 9500 | 0.540100 |
| 10000 | 0.543100 |
| 10500 | 0.540700 |
| 11000 | 0.541200 |
| 11500 | 0.542100 |
| 12000 | 0.538600 |
| 12500 | 0.535000 |
| 13000 | 0.538300 |
| 13500 | 0.546500 |
| 14000 | 0.541200 |
| 14500 | 0.530200 |
| 15000 | 0.532000 |
| 15500 | 0.535000 |
| 16000 | 0.540900 |
| 16500 | 0.543100 |
| 17000 | 0.538300 |
| 17500 | 0.544900 |
| 18000 | 0.538400 |
| 18500 | 0.539000 |
| 19000 | 0.544100 |
| 19500 | 0.541900 |
| 20000 | 0.537700 |
| 20500 | 0.527200 |
| 21000 | 0.535400 |
| 21500 | 0.542500 |
| 22000 | 0.533400 |
| 22500 | 0.534400 |
| 23000 | 0.534100 |
| 23500 | 0.538500 |
| 24000 | 0.533100 |
| 24500 | 0.532800 |
| 25000 | 0.542600 |
| 25500 | 0.531200 |
| 26000 | 0.532100 |
| 26500 | 0.535500 |
| 27000 | 0.543600 |
| 27500 | 0.527600 |
| 28000 | 0.529800 |
| 28500 | 0.537800 |
| 29000 | 0.537900 |
| 29500 | 0.538700 |
| 30000 | 0.535400 |
| 30500 | 0.544600 |
| 31000 | 0.537100 |
| 31500 | 0.529200 |
| 32000 | 0.539300 |
| 32500 | 0.534800 |
| 33000 | 0.538100 |
| 33500 | 0.536100 |
| 34000 | 0.540800 |
| 34500 | 0.544600 |
| 35000 | 0.541200 |
| 35500 | 0.536500 |
| 36000 | 0.531600 |
| 36500 | 0.541100 |
| 37000 | 0.535600 |
| 37500 | 0.533100 |
| 38000 | 0.530700 |
| 38500 | 0.526200 |
| 39000 | 0.539200 |
| 39500 | 0.530500 |
| 40000 | 0.535000 |
| 40500 | 0.532100 |
| 41000 | 0.523400 |
| 41500 | 0.533900 |
| 42000 | 0.534600 |
| 42500 | 0.530300 |
| 43000 | 0.543600 |
| 43500 | 0.531700 |
| 44000 | 0.535400 |
| 44500 | 0.524000 |
| 45000 | 0.534900 |
| 45500 | 0.537900 |
| 46000 | 0.539000 |
| 46500 | 0.537600 |
| 47000 | 0.535000 |
| 47500 | 0.540500 |
| 48000 | 0.535000 |
| 48500 | 0.540900 |
| 49000 | 0.535700 |
| 49500 | 0.531900 |
| 50000 | 0.535700 |
| 50500 | 0.531900 |
| 51000 | 0.535600 |
| 51500 | 0.538700 |
| 52000 | 0.536400 |
| 52500 | 0.536500 |
| 53000 | 0.539100 |
| 53500 | 0.526300 |
| 54000 | 0.525300 |
Saving model checkpoint to ./results/checkpoint-500 Configuration saved in ./results/checkpoint-500/config.json Model weights saved in ./results/checkpoint-500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-18000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-1000 Configuration saved in ./results/checkpoint-1000/config.json Model weights saved in ./results/checkpoint-1000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-1500 Configuration saved in ./results/checkpoint-1500/config.json Model weights saved in ./results/checkpoint-1500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-1000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-2000 Configuration saved in ./results/checkpoint-2000/config.json Model weights saved in ./results/checkpoint-2000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-1500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-2500 Configuration saved in ./results/checkpoint-2500/config.json Model weights saved in ./results/checkpoint-2500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-2000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-3000 Configuration saved in ./results/checkpoint-3000/config.json Model weights saved in ./results/checkpoint-3000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-2500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-3500 Configuration saved in ./results/checkpoint-3500/config.json Model weights saved in ./results/checkpoint-3500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-3000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-4000 Configuration saved in ./results/checkpoint-4000/config.json Model weights saved in ./results/checkpoint-4000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-3500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-4500 Configuration saved in ./results/checkpoint-4500/config.json Model weights saved in ./results/checkpoint-4500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-4000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-5000 Configuration saved in ./results/checkpoint-5000/config.json Model weights saved in ./results/checkpoint-5000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-4500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-5500 Configuration saved in ./results/checkpoint-5500/config.json Model weights saved in ./results/checkpoint-5500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-5000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-6000 Configuration saved in ./results/checkpoint-6000/config.json Model weights saved in ./results/checkpoint-6000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-5500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-6500 Configuration saved in ./results/checkpoint-6500/config.json Model weights saved in ./results/checkpoint-6500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-6000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-7000 Configuration saved in ./results/checkpoint-7000/config.json Model weights saved in ./results/checkpoint-7000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-6500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-7500 Configuration saved in ./results/checkpoint-7500/config.json Model weights saved in ./results/checkpoint-7500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-7000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-8000 Configuration saved in ./results/checkpoint-8000/config.json Model weights saved in ./results/checkpoint-8000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-7500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-8500 Configuration saved in ./results/checkpoint-8500/config.json Model weights saved in ./results/checkpoint-8500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-8000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-9000 Configuration saved in ./results/checkpoint-9000/config.json Model weights saved in ./results/checkpoint-9000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-8500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-9500 Configuration saved in ./results/checkpoint-9500/config.json Model weights saved in ./results/checkpoint-9500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-9000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-10000 Configuration saved in ./results/checkpoint-10000/config.json Model weights saved in ./results/checkpoint-10000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-9500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-10500 Configuration saved in ./results/checkpoint-10500/config.json Model weights saved in ./results/checkpoint-10500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-10000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-11000 Configuration saved in ./results/checkpoint-11000/config.json Model weights saved in ./results/checkpoint-11000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-10500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-11500 Configuration saved in ./results/checkpoint-11500/config.json Model weights saved in ./results/checkpoint-11500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-11000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-12000 Configuration saved in ./results/checkpoint-12000/config.json Model weights saved in ./results/checkpoint-12000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-11500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-12500 Configuration saved in ./results/checkpoint-12500/config.json Model weights saved in ./results/checkpoint-12500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-12000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-13000 Configuration saved in ./results/checkpoint-13000/config.json Model weights saved in ./results/checkpoint-13000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-12500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-13500 Configuration saved in ./results/checkpoint-13500/config.json Model weights saved in ./results/checkpoint-13500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-13000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-14000 Configuration saved in ./results/checkpoint-14000/config.json Model weights saved in ./results/checkpoint-14000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-13500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-14500 Configuration saved in ./results/checkpoint-14500/config.json Model weights saved in ./results/checkpoint-14500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-14000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-15000 Configuration saved in ./results/checkpoint-15000/config.json Model weights saved in ./results/checkpoint-15000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-14500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-15500 Configuration saved in ./results/checkpoint-15500/config.json Model weights saved in ./results/checkpoint-15500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-15000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-16000 Configuration saved in ./results/checkpoint-16000/config.json Model weights saved in ./results/checkpoint-16000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-15500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-16500 Configuration saved in ./results/checkpoint-16500/config.json Model weights saved in ./results/checkpoint-16500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-16000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-17000 Configuration saved in ./results/checkpoint-17000/config.json Model weights saved in ./results/checkpoint-17000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-16500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-17500 Configuration saved in ./results/checkpoint-17500/config.json Model weights saved in ./results/checkpoint-17500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-17000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-18000 Configuration saved in ./results/checkpoint-18000/config.json Model weights saved in ./results/checkpoint-18000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-17500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-18500 Configuration saved in ./results/checkpoint-18500/config.json Model weights saved in ./results/checkpoint-18500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-18000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-19000 Configuration saved in ./results/checkpoint-19000/config.json Model weights saved in ./results/checkpoint-19000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-18500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-19500 Configuration saved in ./results/checkpoint-19500/config.json Model weights saved in ./results/checkpoint-19500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-19000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-20000 Configuration saved in ./results/checkpoint-20000/config.json Model weights saved in ./results/checkpoint-20000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-19500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-20500 Configuration saved in ./results/checkpoint-20500/config.json Model weights saved in ./results/checkpoint-20500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-20000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-21000 Configuration saved in ./results/checkpoint-21000/config.json Model weights saved in ./results/checkpoint-21000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-20500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-21500 Configuration saved in ./results/checkpoint-21500/config.json Model weights saved in ./results/checkpoint-21500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-21000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-22000 Configuration saved in ./results/checkpoint-22000/config.json Model weights saved in ./results/checkpoint-22000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-21500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-22500 Configuration saved in ./results/checkpoint-22500/config.json Model weights saved in ./results/checkpoint-22500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-22000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-23000 Configuration saved in ./results/checkpoint-23000/config.json Model weights saved in ./results/checkpoint-23000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-22500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-23500 Configuration saved in ./results/checkpoint-23500/config.json Model weights saved in ./results/checkpoint-23500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-23000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-24000 Configuration saved in ./results/checkpoint-24000/config.json Model weights saved in ./results/checkpoint-24000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-23500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-24500 Configuration saved in ./results/checkpoint-24500/config.json Model weights saved in ./results/checkpoint-24500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-24000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-25000 Configuration saved in ./results/checkpoint-25000/config.json Model weights saved in ./results/checkpoint-25000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-24500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-25500 Configuration saved in ./results/checkpoint-25500/config.json Model weights saved in ./results/checkpoint-25500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-25000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-26000 Configuration saved in ./results/checkpoint-26000/config.json Model weights saved in ./results/checkpoint-26000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-25500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-26500 Configuration saved in ./results/checkpoint-26500/config.json Model weights saved in ./results/checkpoint-26500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-26000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-27000 Configuration saved in ./results/checkpoint-27000/config.json Model weights saved in ./results/checkpoint-27000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-26500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-27500 Configuration saved in ./results/checkpoint-27500/config.json Model weights saved in ./results/checkpoint-27500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-27000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-28000 Configuration saved in ./results/checkpoint-28000/config.json Model weights saved in ./results/checkpoint-28000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-27500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-28500 Configuration saved in ./results/checkpoint-28500/config.json Model weights saved in ./results/checkpoint-28500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-28000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-29000 Configuration saved in ./results/checkpoint-29000/config.json Model weights saved in ./results/checkpoint-29000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-28500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-29500 Configuration saved in ./results/checkpoint-29500/config.json Model weights saved in ./results/checkpoint-29500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-29000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-30000 Configuration saved in ./results/checkpoint-30000/config.json Model weights saved in ./results/checkpoint-30000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-29500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-30500 Configuration saved in ./results/checkpoint-30500/config.json Model weights saved in ./results/checkpoint-30500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-30000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-31000 Configuration saved in ./results/checkpoint-31000/config.json Model weights saved in ./results/checkpoint-31000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-30500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-31500 Configuration saved in ./results/checkpoint-31500/config.json Model weights saved in ./results/checkpoint-31500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-31000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-32000 Configuration saved in ./results/checkpoint-32000/config.json Model weights saved in ./results/checkpoint-32000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-31500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-32500 Configuration saved in ./results/checkpoint-32500/config.json Model weights saved in ./results/checkpoint-32500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-32000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-33000 Configuration saved in ./results/checkpoint-33000/config.json Model weights saved in ./results/checkpoint-33000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-32500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-33500 Configuration saved in ./results/checkpoint-33500/config.json Model weights saved in ./results/checkpoint-33500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-33000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-34000 Configuration saved in ./results/checkpoint-34000/config.json Model weights saved in ./results/checkpoint-34000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-33500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-34500 Configuration saved in ./results/checkpoint-34500/config.json Model weights saved in ./results/checkpoint-34500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-34000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-35000 Configuration saved in ./results/checkpoint-35000/config.json Model weights saved in ./results/checkpoint-35000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-34500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-35500 Configuration saved in ./results/checkpoint-35500/config.json Model weights saved in ./results/checkpoint-35500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-35000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-36000 Configuration saved in ./results/checkpoint-36000/config.json Model weights saved in ./results/checkpoint-36000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-35500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-36500 Configuration saved in ./results/checkpoint-36500/config.json Model weights saved in ./results/checkpoint-36500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-36000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-37000 Configuration saved in ./results/checkpoint-37000/config.json Model weights saved in ./results/checkpoint-37000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-36500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-37500 Configuration saved in ./results/checkpoint-37500/config.json Model weights saved in ./results/checkpoint-37500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-37000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-38000 Configuration saved in ./results/checkpoint-38000/config.json Model weights saved in ./results/checkpoint-38000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-37500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-38500 Configuration saved in ./results/checkpoint-38500/config.json Model weights saved in ./results/checkpoint-38500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-38000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-39000 Configuration saved in ./results/checkpoint-39000/config.json Model weights saved in ./results/checkpoint-39000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-38500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-39500 Configuration saved in ./results/checkpoint-39500/config.json Model weights saved in ./results/checkpoint-39500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-39000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-40000 Configuration saved in ./results/checkpoint-40000/config.json Model weights saved in ./results/checkpoint-40000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-39500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-40500 Configuration saved in ./results/checkpoint-40500/config.json Model weights saved in ./results/checkpoint-40500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-40000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-41000 Configuration saved in ./results/checkpoint-41000/config.json Model weights saved in ./results/checkpoint-41000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-40500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-41500 Configuration saved in ./results/checkpoint-41500/config.json Model weights saved in ./results/checkpoint-41500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-41000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-42000 Configuration saved in ./results/checkpoint-42000/config.json Model weights saved in ./results/checkpoint-42000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-41500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-42500 Configuration saved in ./results/checkpoint-42500/config.json Model weights saved in ./results/checkpoint-42500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-42000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-43000 Configuration saved in ./results/checkpoint-43000/config.json Model weights saved in ./results/checkpoint-43000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-42500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-43500 Configuration saved in ./results/checkpoint-43500/config.json Model weights saved in ./results/checkpoint-43500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-43000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-44000 Configuration saved in ./results/checkpoint-44000/config.json Model weights saved in ./results/checkpoint-44000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-43500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-44500 Configuration saved in ./results/checkpoint-44500/config.json Model weights saved in ./results/checkpoint-44500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-44000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-45000 Configuration saved in ./results/checkpoint-45000/config.json Model weights saved in ./results/checkpoint-45000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-44500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-45500 Configuration saved in ./results/checkpoint-45500/config.json Model weights saved in ./results/checkpoint-45500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-45000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-46000 Configuration saved in ./results/checkpoint-46000/config.json Model weights saved in ./results/checkpoint-46000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-45500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-46500 Configuration saved in ./results/checkpoint-46500/config.json Model weights saved in ./results/checkpoint-46500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-46000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-47000 Configuration saved in ./results/checkpoint-47000/config.json Model weights saved in ./results/checkpoint-47000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-46500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-47500 Configuration saved in ./results/checkpoint-47500/config.json Model weights saved in ./results/checkpoint-47500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-47000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-48000 Configuration saved in ./results/checkpoint-48000/config.json Model weights saved in ./results/checkpoint-48000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-47500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-48500 Configuration saved in ./results/checkpoint-48500/config.json Model weights saved in ./results/checkpoint-48500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-48000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-49000 Configuration saved in ./results/checkpoint-49000/config.json Model weights saved in ./results/checkpoint-49000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-48500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-49500 Configuration saved in ./results/checkpoint-49500/config.json Model weights saved in ./results/checkpoint-49500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-49000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-50000 Configuration saved in ./results/checkpoint-50000/config.json Model weights saved in ./results/checkpoint-50000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-49500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-50500 Configuration saved in ./results/checkpoint-50500/config.json Model weights saved in ./results/checkpoint-50500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-50000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-51000 Configuration saved in ./results/checkpoint-51000/config.json Model weights saved in ./results/checkpoint-51000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-50500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-51500 Configuration saved in ./results/checkpoint-51500/config.json Model weights saved in ./results/checkpoint-51500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-51000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-52000 Configuration saved in ./results/checkpoint-52000/config.json Model weights saved in ./results/checkpoint-52000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-51500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-52500 Configuration saved in ./results/checkpoint-52500/config.json Model weights saved in ./results/checkpoint-52500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-52000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-53000 Configuration saved in ./results/checkpoint-53000/config.json Model weights saved in ./results/checkpoint-53000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-52500] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-53500 Configuration saved in ./results/checkpoint-53500/config.json Model weights saved in ./results/checkpoint-53500/pytorch_model.bin Deleting older checkpoint [results/checkpoint-53000] due to args.save_total_limit Saving model checkpoint to ./results/checkpoint-54000 Configuration saved in ./results/checkpoint-54000/config.json Model weights saved in ./results/checkpoint-54000/pytorch_model.bin Deleting older checkpoint [results/checkpoint-53500] due to args.save_total_limit Training completed. Do not forget to share your model on huggingface.co/models =)
TrainOutput(global_step=54297, training_loss=0.537743896358476, metrics={'train_runtime': 1276.4291, 'train_samples_per_second': 680.599, 'train_steps_per_second': 42.538, 'total_flos': 8589565725967872.0, 'train_loss': 0.537743896358476, 'epoch': 3.0})x_dev = [x[0] for x in io_read('/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/dev-0/in.tsv.xz')]
y_dev = [int(y) for y in io_read('/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/dev-0/expected.tsv')]
# x_test = [x[0] for x in io_read('/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/test-A/in.tsv.xz')]
dev_encodings = tokenizer(x_dev, truncation=True, padding='max_length', max_length=128)
dev_dataset = Dataset(dev_encodings, y_dev)dev_predicted = trainer.predict(dev_dataset).predictions***** Running Prediction ***** Num examples = 5272 Batch size = 8
[659/659 02:38]
dev_output = ['0' if x > y else '1' for x, y in dev_predicted]io_write(dev_output, '/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/dev-0/out.tsv')x_test = [x[0] for x in io_read('/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/test-A/in.tsv.xz')]
y_test = [0 for x in x_test]
test_encodings = tokenizer(x_test, truncation=True, padding='max_length', max_length=128)
test_dataset = Dataset(test_encodings, y_test)
test_predicted = trainer.predict(test_dataset).predictions
test_output = ['0' if x > y else '1' for x, y in test_predicted]
io_write(test_output, '/content/drive/MyDrive/ai-tech/eki/paranormal-or-skeptic/test-A/out.tsv')***** Running Prediction ***** Num examples = 5152 Batch size = 8
[659/659 04:54]