14 KiB
14 KiB
!pip install transformers datasets torch sentencepiece
Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.35.2) Requirement already satisfied: datasets in /usr/local/lib/python3.10/dist-packages (2.16.1) Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (2.1.0+cu121) Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (0.1.99) Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers) (3.13.1) Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.20.2) Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (1.23.5) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers) (23.2) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (6.0.1) Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (2023.6.3) Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers) (2.31.0) Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.15.0) Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.4.1) Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers) (4.66.1) Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (10.0.1) Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets) (0.6) Requirement already satisfied: dill<0.3.8,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.3.7) Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (1.5.3) Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.4.1) Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from datasets) (0.70.15) Requirement already satisfied: fsspec[http]<=2023.10.0,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.6.0) Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.9.1) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch) (4.5.0) Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch) (1.12) Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch) (3.2.1) Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch) (3.1.2) Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch) (2.1.0) Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (23.2.0) Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.4) Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.4) Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.4.1) Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1) Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.6) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2.0.7) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2023.11.17) Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch) (2.1.3) Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2) Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2023.3.post1) Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch) (1.3.0) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.16.0)
from datasets import load_dataset
from transformers import AutoModelForSeq2SeqLM
from transformers import AutoTokenizer
from transformers import pipeline
import torch
from tqdm import tqdm
from sklearn.metrics import accuracy_scoredef load_and_process_dataset():
dataset = load_dataset("sst2")
dataset.remove_columns('idx')
del dataset['test']
dataset['test'] = dataset['validation']
del dataset['validation']
split_dataset = dataset['train'].train_test_split(test_size=1600)
dataset['train'] = split_dataset['train']
dataset['validation'] = split_dataset['test']
return datasetdataset = load_and_process_dataset()
dataset/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: The secret `HF_TOKEN` does not exist in your Colab secrets. To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session. You will be able to reuse this secret in all of your notebooks. Please note that authentication is recommended but still optional to access public models or datasets. warnings.warn(
DatasetDict({
train: Dataset({
features: ['idx', 'sentence', 'label'],
num_rows: 65749
})
test: Dataset({
features: ['idx', 'sentence', 'label'],
num_rows: 872
})
validation: Dataset({
features: ['idx', 'sentence', 'label'],
num_rows: 1600
})
})tokenizer = AutoTokenizer.from_pretrained('google/flan-t5-base')def transform_dataset(dataset):
new_dataset = []
for row in dataset['test']:
text = row['sentence'].replace("\n", "")
new_row = {'sentence': text, 'label': "negative" if row['label'] == 0 else "positive"}
new_dataset.append(new_row)
return new_dataset
new_dataset = transform_dataset(dataset)
dataset_copy = new_dataset.copy()
model = AutoModelForSeq2SeqLM.from_pretrained('google/flan-t5-base')
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
sentiment_classifier = pipeline("sentiment-analysis")
def create_predictions(test_data):
predictions = []
expected = []
for row in tqdm(test_data):
input_text = row['sentence']
result = sentiment_classifier(input_text)
label = result[0]['label'].lower()
# Zamiana etykiet na format używany w zbiorze danych
if label == 'positive':
label = 'positive'
else:
label = 'negative'
predictions.append(label)
expected.append(row['label'])
return predictions, expected
No model was supplied, defaulted to distilbert-base-uncased-finetuned-sst-2-english and revision af0f99b (https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english). Using a pipeline without specifying a model name and revision in production is not recommended.
pred = create_predictions(dataset_copy)
predictions = pred[0]
expected = pred[1]
100%|██████████| 872/872 [01:08<00:00, 12.65it/s]
accuracy = accuracy_score(expected, [p if p in ["positive", "negative"] else "negative" for p in predictions])
print("Accuracy:", accuracy)
Accuracy: 0.9105504587155964
example_sentences = [
"This movie was an amazing journey.",
"I really did not like the new web design.",
"The team did a great job with this project.",
"I am not happy with the service.",
"This is the best book I have ever read!",
]
for sentence in example_sentences:
result = sentiment_classifier(sentence)
print(f"Sentence: '{sentence}'\nSentiment: {result[0]['label']}, Score: {result[0]['score']:.2f}\n")
Sentence: 'This movie was an amazing journey.' Sentiment: POSITIVE, Score: 1.00 Sentence: 'I really did not like the new web design.' Sentiment: NEGATIVE, Score: 1.00 Sentence: 'The team did a great job with this project.' Sentiment: POSITIVE, Score: 1.00 Sentence: 'I am not happy with the service.' Sentiment: NEGATIVE, Score: 1.00 Sentence: 'This is the best book I have ever read!' Sentiment: POSITIVE, Score: 1.00
example_sentences = [
"The cat is sitting on the mat.",
"There are clouds in the sky.",
"The book is on the table.",
"A car is parked outside.",
"The door is closed.",
]
for sentence in example_sentences:
result = sentiment_classifier(sentence)
print(f"Sentence: '{sentence}'\nSentiment: {result[0]['label']}, Score: {result[0]['score']:.2f}\n")Sentence: 'The cat is sitting on the mat.' Sentiment: NEGATIVE, Score: 0.98 Sentence: 'There are clouds in the sky.' Sentiment: POSITIVE, Score: 1.00 Sentence: 'The book is on the table.' Sentiment: POSITIVE, Score: 0.99 Sentence: 'A car is parked outside.' Sentiment: POSITIVE, Score: 0.90 Sentence: 'The door is closed.' Sentiment: NEGATIVE, Score: 0.98