s444415/uczenie_glebokie_projekt
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
e457ab5cee
uczenie_glebokie_projekt/T5Classification.ipynb
Michał Kozłowski e457ab5cee init
2023-02-13 14:22:50 +01:00

288 KiB
Raw Blame History

T5 model classification training

Links:

!pip install datasets transformers sentencepiece
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting datasets
  Downloading datasets-2.9.0-py3-none-any.whl (462 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 462.8/462.8 KB 8.3 MB/s eta 0:00:00
[?25hCollecting transformers
  Downloading transformers-4.26.1-py3-none-any.whl (6.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 66.9 MB/s eta 0:00:00
[?25hCollecting sentencepiece
  Downloading sentencepiece-0.1.97-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 49.0 MB/s eta 0:00:00
[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from datasets) (1.21.6)
Collecting huggingface-hub<1.0.0,>=0.2.0
  Downloading huggingface_hub-0.12.0-py3-none-any.whl (190 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 190.3/190.3 KB 14.6 MB/s eta 0:00:00
[?25hRequirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (2.25.1)
Requirement already satisfied: pandas in /usr/local/lib/python3.8/dist-packages (from datasets) (1.3.5)
Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (9.0.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from datasets) (23.0)
Collecting responses<0.19
  Downloading responses-0.18.0-py3-none-any.whl (38 kB)
Collecting xxhash
  Downloading xxhash-3.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (213 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 213.0/213.0 KB 10.7 MB/s eta 0:00:00
[?25hRequirement already satisfied: dill<0.3.7 in /usr/local/lib/python3.8/dist-packages (from datasets) (0.3.6)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.8/dist-packages (from datasets) (3.8.3)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (2023.1.0)
Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (4.64.1)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (6.0)
Collecting multiprocess
  Downloading multiprocess-0.70.14-py38-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.0/132.0 KB 10.5 MB/s eta 0:00:00
[?25hCollecting tokenizers!=0.11.3,<0.14,>=0.11.1
  Downloading tokenizers-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.6/7.6 MB 72.5 MB/s eta 0:00:00
[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.9.0)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (2.1.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (22.2.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (6.0.4)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.3)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.1)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (4.0.2)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.8.2)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.8/dist-packages (from huggingface-hub<1.0.0,>=0.2.0->datasets) (4.4.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests>=2.19.0->datasets) (1.24.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests>=2.19.0->datasets) (2022.12.7)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests>=2.19.0->datasets) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests>=2.19.0->datasets) (4.0.0)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 KB 10.8 MB/s eta 0:00:00
[?25hRequirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas->datasets) (2022.7.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.7.3->pandas->datasets) (1.15.0)
Installing collected packages: tokenizers, sentencepiece, xxhash, urllib3, multiprocess, responses, huggingface-hub, transformers, datasets
  Attempting uninstall: urllib3
    Found existing installation: urllib3 1.24.3
    Uninstalling urllib3-1.24.3:
      Successfully uninstalled urllib3-1.24.3
Successfully installed datasets-2.9.0 huggingface-hub-0.12.0 multiprocess-0.70.14 responses-0.18.0 sentencepiece-0.1.97 tokenizers-0.13.2 transformers-4.26.1 urllib3-1.26.14 xxhash-3.2.0

from datasets import load_dataset
import torch
from transformers import T5ForConditionalGeneration, T5Tokenizer, TrainingArguments, Trainer
from sklearn.metrics import accuracy_score, precision_recall_fscore_support
import tensorflow as tf
from torch.utils.data import DataLoader, RandomSampler, SequentialSampler, TensorDataset
import random
import time
import numpy as np
import datetime
import sklearn
from tqdm.notebook import tqdm
import os

Load data and transform dataset

def load_and_process_dataset():
    dataset = load_dataset("sst2")
    dataset.remove_columns('idx')
    del dataset['test']
    dataset['test'] = dataset['validation']
    del dataset['validation']
    split_dataset = dataset['train'].train_test_split(test_size=1600)
    dataset['train'] = split_dataset['train']
    dataset['validation'] = split_dataset['test']
    return dataset
dataset = load_and_process_dataset()
dataset
WARNING:datasets.builder:Found cached dataset sst2 (/root/.cache/huggingface/datasets/sst2/default/2.0.0/9896208a8d85db057ac50c72282bcb8fe755accc671a57dd8059d4e130961ed5)

  0%|          | 0/3 [00:00<?, ?it/s]
DatasetDict({
    train: Dataset({
        features: ['idx', 'sentence', 'label'],
        num_rows: 65749
    })
    test: Dataset({
        features: ['idx', 'sentence', 'label'],
        num_rows: 872
    })
    validation: Dataset({
        features: ['idx', 'sentence', 'label'],
        num_rows: 1600
    })
})
for key in dataset.keys():
  dataset[key] = dataset[key].map(lambda x: {'source_text': f"sentiment-analysis: {x['sentence']}", 'target_text': 'positive' if x['label'] else 'negative'}).remove_columns(['idx', 'sentence', 'label'])
  0%|          | 0/65749 [00:00<?, ?ex/s]
  0%|          | 0/872 [00:00<?, ?ex/s]
  0%|          | 0/1600 [00:00<?, ?ex/s]
train = dataset['train']
validation = dataset['validation']
test = dataset['test']
dataset['train'][0]
{'source_text': 'sentiment-analysis: smart , sassy and exceptionally charming ',
 'target_text': 'positive'}
tokenizer = T5Tokenizer.from_pretrained("t5-small")
model = T5ForConditionalGeneration.from_pretrained("t5-small")
Downloading (…)ve/main/spiece.model:   0%|          | 0.00/792k [00:00<?, ?B/s]
Downloading (…)lve/main/config.json:   0%|          | 0.00/1.21k [00:00<?, ?B/s]
/usr/local/lib/python3.8/dist-packages/transformers/models/t5/tokenization_t5.py:163: FutureWarning: This tokenizer was incorrectly instantiated with a model max length of 512 which will be corrected in Transformers v5.
For now, this behavior is kept to avoid breaking backwards compatibility when padding/encoding with `truncation is True`.
- Be aware that you SHOULD NOT rely on t5-small automatically truncating your input to 512 when padding/encoding.
- If you want to encode/pad to sequences longer than 512 you can either instantiate this tokenizer with `model_max_length` or pass `max_length` when encoding/padding.
- To avoid this warning, please instantiate this tokenizer with `model_max_length` set to your preferred value.
  warnings.warn(

Downloading (…)"pytorch_model.bin";:   0%|          | 0.00/242M [00:00<?, ?B/s]
Downloading (…)neration_config.json:   0%|          | 0.00/147 [00:00<?, ?B/s]
model.cuda();
def tokenize(dataset):
  input_ids = []
  target_ids = []
  attention_masks = []
  for row in dataset:
    encoding = tokenizer.encode_plus(
      row['source_text'],
      padding="max_length",
      max_length=86,
      truncation=True,
      add_special_tokens = True,
      return_attention_mask = True,
      return_tensors="pt"
    )
    target_encoding = tokenizer.encode_plus(
      row['target_text'],
      padding="max_length",
      max_length=2,
      truncation=True,
      add_special_tokens = True,
      return_attention_mask = True,
      return_tensors="pt",
    )
    input_ids.append(encoding['input_ids'])
    target_ids.append(target_encoding['input_ids'])
    attention_masks.append(encoding['attention_mask'])
  return TensorDataset(torch.cat(input_ids, dim=0), torch.cat(attention_masks, dim=0), torch.cat(target_ids, dim=0))

train_tokenized = tokenize(train)
validation_tokenized = tokenize(validation)
test_tokenized = tokenize(test)
batch_size = 32

train_dataloader = DataLoader(
            train_tokenized,
            sampler = RandomSampler(train_tokenized),
            batch_size = batch_size
        )

validation_dataloader = DataLoader(
            validation_tokenized,
            sampler = SequentialSampler(validation_tokenized),
            batch_size = batch_size
        )

test_dataloader = DataLoader(
            test_tokenized,
            sampler = SequentialSampler(test_tokenized),
            batch_size = batch_size
        )

Main model training class

class T5Model:
  def __init__(self, model, tokenizer, train_dataloader, val_dataloader, logs_dir_path, seed=42, epochs=3, lr=1e-4, eps=1e-8):
    self.model = model
    self.tokenizer = tokenizer
    self.train_dataloader = train_dataloader
    self.val_dataloader = val_dataloader
    self.seed=seed
    self.epochs = epochs
    self.learning_rate = lr
    self.eps = eps
    self.logs_dir_path = logs_dir_path
    self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    self.optimizer = torch.optim.AdamW(model.parameters(), lr = lr, eps = eps)
    self.tensorboard_writter()
    self.init_seed()

  def init_seed(self):
    random.seed(self.seed)
    np.random.seed(self.seed)
    torch.manual_seed(self.seed)
    torch.cuda.manual_seed_all(self.seed)

  def tensorboard_writter(self):
    if not os.path.exists(self.logs_dir_path):
      os.mkdir(self.logs_dir_path)
    self.summary_writer = tf.summary.create_file_writer(self.logs_dir_path)

  def compute_metrics(self, target, preds):
      return sklearn.metrics.accuracy_score(target, preds)

  def _train(self, epoch_number):
    print(f"Training epoch: {epoch_number}")
    self.model.train()
    train_accuracy_total = 0
    pbar = tqdm(self.train_dataloader, total=len(self.train_dataloader))
    for step, batch in enumerate(pbar):
        b_input_ids = batch[0].to(self.device)
        b_input_mask = batch[1].to(self.device)
        b_labels = batch[2].to(self.device)

        outputs = self.model(
            input_ids=b_input_ids,
            attention_mask=b_input_mask,
            labels=b_labels
        )

        gen_output = self.model.generate(
            input_ids = b_input_ids,
            attention_mask = b_input_mask, 
            max_length=3, 
            num_beams=2,
            repetition_penalty=2.5, 
            length_penalty=1.0, 
            early_stopping=True
        )

        preds = [self.tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True) for g in gen_output]
        target = [self.tokenizer.decode(t, skip_special_tokens=True, clean_up_tokenization_spaces=True) for t in b_labels]

        loss = outputs[0]
        self.optimizer.zero_grad()
        loss.backward()
        self.optimizer.step()

        accuracy = self.compute_metrics(target, preds)
        train_accuracy_total += accuracy
        pbar.set_description(f"Epoch {epoch_number+1}, step {step}, train loss {loss}, accuracy: {accuracy}")

        with self.summary_writer.as_default():
          tf.summary.scalar(name=f"Epoch: {epoch_number+1}, step/train_accuracy", data=accuracy ,step=step)
          tf.summary.scalar(name=f"Epoch: {epoch_number+1}, step/train_loss", data=loss.cpu().detach().numpy(), step=step)
        self.summary_writer.flush()
    print(f"Epoch: {epoch_number+1}, Average train accuracy: {train_accuracy_total/len(self.train_dataloader)}")

  def train(self):
    print(f"Starting training, epochs: {self.epochs}")
    for i in range(0, self.epochs):
      self._train(i)
      self.validate(i)
      
  def validate(self, epoch_number):
    print(f"Validation epoch: {epoch_number}")
    self.model.eval()
    val_accuracy_total = 0
    pbar_val = tqdm(self.val_dataloader, total=len(self.val_dataloader))
    for step, batch in enumerate(pbar_val):
        b_input_ids = batch[0].to(self.device)
        b_input_mask = batch[1].to(self.device)
        b_labels = batch[2].to(self.device)

        with torch.no_grad():        

          outputs = self.model(
            input_ids=b_input_ids,
            attention_mask=b_input_mask,
            labels=b_labels
          )

          loss = outputs[0]

          gen_output = self.model.generate(
            input_ids = b_input_ids,
            attention_mask = b_input_mask, 
            max_length=3, 
            num_beams=2,
            repetition_penalty=2.5, 
            length_penalty=1.0, 
            early_stopping=True
          )

          preds = [self.tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True) for g in gen_output]
          target = [self.tokenizer.decode(t, skip_special_tokens=True, clean_up_tokenization_spaces=True) for t in b_labels]

          accuracy_val = self.compute_metrics(target, preds)
          val_accuracy_total += accuracy_val
          pbar_val.set_description(f"Epoch {epoch_number+1}, val loss {loss}, accuracy: {accuracy_val}")

          with self.summary_writer.as_default():
            tf.summary.scalar(name=f"Epoch: {epoch_number+1}, step/val_accuracy", data=accuracy_val ,step=step)
            tf.summary.scalar(name=f"Epoch: {epoch_number+1}, step/val_loss", data=loss.cpu().detach().numpy(), step=step)
          self.summary_writer.flush()

    print(f"Epoch: {epoch_number+1}, Average validation accuracy: {val_accuracy_total/len(self.val_dataloader)}")

  def evaluate(self, test_dataloader):
    self.model.eval()
    total_test_acc = 0
    for batch in tqdm(test_dataloader, total=len(test_dataloader)):
        b_input_ids = batch[0].to(self.device)
        b_input_mask = batch[1].to(self.device)
        b_labels = batch[2].to(self.device)
        
        with torch.no_grad():        

          gen_output = self.model.generate(
            input_ids = b_input_ids,
            attention_mask = b_input_mask, 
            max_length=3, 
            num_beams=2,
            repetition_penalty=2.5, 
            length_penalty=1.0, 
            early_stopping=True
          )
            
        preds = [self.tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True) for g in gen_output]
        target = [self.tokenizer.decode(t, skip_special_tokens=True, clean_up_tokenization_spaces=True) for t in b_labels]
        total_test_acc += self.compute_metrics(preds, target)  
    print("Accuracy: ",total_test_acc / len(test_dataloader))
    
t5model = T5Model(model, tokenizer, train_dataloader, validation_dataloader, "./logs")
t5model.train()
Starting training, epochs: 3
Training epoch: 0

  0%|          | 0/2055 [00:00<?, ?it/s]
Epoch: 1, Average train accuracy: 0.8825947167188043
Validation epoch: 0

  0%|          | 0/50 [00:00<?, ?it/s]
Epoch: 1, Average validation accuracy: 0.926875
Training epoch: 1

  0%|          | 0/2055 [00:00<?, ?it/s]
Epoch: 2, Average train accuracy: 0.9252577916811494
Validation epoch: 1

  0%|          | 0/50 [00:00<?, ?it/s]
Epoch: 2, Average validation accuracy: 0.936875
Training epoch: 2

  0%|          | 0/2055 [00:00<?, ?it/s]
Epoch: 3, Average train accuracy: 0.9391720252577916
Validation epoch: 2

  0%|          | 0/50 [00:00<?, ?it/s]
Epoch: 3, Average validation accuracy: 0.93875

t5model.evaluate(test_dataloader)
  0%|          | 0/28 [00:00<?, ?it/s]
Accuracy:  0.9129464285714286

!tensorboard dev upload --logdir logs --name t5-sst2
2023-02-13 11:49:46.367397: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia
2023-02-13 11:49:46.367509: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia
2023-02-13 11:49:46.367529: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

***** TensorBoard Uploader *****

This will upload your TensorBoard logs to https://tensorboard.dev/ from
the following directory:

logs

This TensorBoard will be visible to everyone. Do not upload sensitive
data.

Your use of this service is subject to Google's Terms of Service
<https://policies.google.com/terms> and Privacy Policy
<https://policies.google.com/privacy>, and TensorBoard.dev's Terms of Service
<https://tensorboard.dev/policy/terms/>.

This notice will not be shown again while you are logged into the uploader.
To log out, run `tensorboard dev auth revoke`.

Continue? (yes/NO) yes

To sign in with the TensorBoard uploader:

1. On your computer or phone, visit:

   https://www.google.com/device

2. Sign in with your Google account, then enter:

   ZBRH-SMMW


Upload started and will continue reading any new data as it's added to the logdir.

To stop uploading, press Ctrl-C.

New experiment created. View your TensorBoard at: https://tensorboard.dev/experiment/CgaWd9pATZeuquRT7TZp7w/

[2023-02-13T11:50:01] Started scanning logdir.
[2023-02-13T11:50:04] Total uploaded: 12630 scalars, 0 tensors, 0 binary objects


Interrupted. View your TensorBoard at https://tensorboard.dev/experiment/CgaWd9pATZeuquRT7TZp7w/
Traceback (most recent call last):
  File "/usr/local/bin/tensorboard", line 8, in <module>
    sys.exit(run_main())
  File "/usr/local/lib/python3.8/dist-packages/tensorboard/main.py", line 46, in run_main
    app.run(tensorboard.main, flags_parser=tensorboard.configure)
  File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "/usr/local/lib/python3.8/dist-packages/tensorboard/program.py", line 276, in main
    return runner(self.flags) or 0
  File "/usr/local/lib/python3.8/dist-packages/tensorboard/uploader/uploader_subcommand.py", line 691, in run
    return _run(flags, self._experiment_url_callback)
  File "/usr/local/lib/python3.8/dist-packages/tensorboard/uploader/uploader_subcommand.py", line 124, in _run
    intent.execute(server_info, channel)
  File "/usr/local/lib/python3.8/dist-packages/grpc/_channel.py", line 1564, in __exit__
    self._close()
  File "/usr/local/lib/python3.8/dist-packages/grpc/_channel.py", line 1550, in _close
    self._channel.close(cygrpc.StatusCode.cancelled, 'Channel closed!')
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 513, in grpc._cython.cygrpc.Channel.close
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 399, in grpc._cython.cygrpc._close
  File "src/python/grpcio/grpc/_cython/_cygrpc/channel.pyx.pxi", line 429, in grpc._cython.cygrpc._close
  File "/usr/lib/python3.8/threading.py", line 364, in notify_all
    def notify_all(self):
KeyboardInterrupt
^C

t5model.model.save_pretrained("./model")
t5model.tokenizer.save_pretrained("./model")

('./model/tokenizer_config.json',
 './model/special_tokens_map.json',
 './model/spiece.model',
 './model/added_tokens.json')
!zip -r /content/model model
  adding: model/ (stored 0%)
  adding: model/tokenizer_config.json (deflated 82%)
  adding: model/config.json (deflated 62%)
  adding: model/generation_config.json (deflated 29%)
  adding: model/pytorch_model.bin (deflated 8%)
  adding: model/special_tokens_map.json (deflated 86%)
  adding: model/spiece.model (deflated 48%)

!huggingface-cli login
    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|
    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|
    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|
    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|
    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|
    
    To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .
Token: 
Add token as git credential? (Y/n) y
Token is valid.
Cannot authenticate through git-credential as no helper is defined on your machine.
You might have to re-authenticate when pushing to the Hugging Face Hub.
Run the following command in your terminal in case you want to set the 'store' credential helper as default.

git config --global credential.helper store

Read https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage for more details.
Token has not been saved to git credential helper.
Your token has been saved to /root/.cache/huggingface/token
Login successful

t5model.model.push_to_hub("Zombely/t5-model")
Upload 1 LFS files:   0%|          | 0/1 [00:00<?, ?it/s]
pytorch_model.bin:   0%|          | 0.00/242M [00:00<?, ?B/s]
CommitInfo(commit_url='https://huggingface.co/Zombely/t5-model/commit/caab3e8d5523bc7309774f13f522fd6f06666b29', commit_message='Upload T5ForConditionalGeneration', commit_description='', oid='caab3e8d5523bc7309774f13f522fd6f06666b29', pr_url=None, pr_revision=None, pr_num=None)
t5model.tokenizer.push_to_hub("Zombely/t5-tokenizer")

spiece.model:   0%|          | 0.00/792k [00:00<?, ?B/s]
Upload 1 LFS files:   0%|          | 0/1 [00:00<?, ?it/s]
CommitInfo(commit_url='https://huggingface.co/Zombely/t5-tokenizer/commit/640998bd3d496cbd5266140fbeff873d07c0158e', commit_message='Upload tokenizer', commit_description='', oid='640998bd3d496cbd5266140fbeff873d07c0158e', pr_url=None, pr_revision=None, pr_num=None)