2024-05-10 14:56:49 +02:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Uczenie głębokie – –
"# 3. RNN"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Podejście softmax z embeddingami na przykładzie NER"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 11,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Defaulting to user installation because normal site-packages is not writeable\n",
"Requirement already satisfied: torch in /home/pawel/.local/lib/python3.10/site-packages (2.3.0)\n",
2024-05-13 10:07:22 +02:00
"Requirement already satisfied: torchtext in /home/pawel/.local/lib/python3.10/site-packages (0.18.0)\n",
"Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (2.20.5)\n",
"Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (12.1.105)\n",
2024-05-10 14:56:49 +02:00
"Requirement already satisfied: sympy in /home/pawel/.local/lib/python3.10/site-packages (from torch) (1.12)\n",
2024-05-13 10:07:22 +02:00
"Requirement already satisfied: jinja2 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (3.1.3)\n",
"Requirement already satisfied: filelock in /home/pawel/.local/lib/python3.10/site-packages (from torch) (3.13.1)\n",
"Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (11.0.2.54)\n",
"Requirement already satisfied: triton==2.3.0 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (2.3.0)\n",
"Requirement already satisfied: fsspec in /home/pawel/.local/lib/python3.10/site-packages (from torch) (2024.2.0)\n",
2024-05-10 14:56:49 +02:00
"Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (12.1.105)\n",
2024-05-13 10:07:22 +02:00
"Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (12.1.0.106)\n",
"Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (11.4.5.107)\n",
2024-05-10 14:56:49 +02:00
"Requirement already satisfied: typing-extensions>=4.8.0 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (4.10.0)\n",
"Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (8.9.2.26)\n",
"Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (12.1.105)\n",
"Requirement already satisfied: networkx in /home/pawel/.local/lib/python3.10/site-packages (from torch) (3.3)\n",
2024-05-13 10:07:22 +02:00
"Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (12.1.105)\n",
"Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (12.1.3.1)\n",
"Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /home/pawel/.local/lib/python3.10/site-packages (from torch) (10.3.2.106)\n",
2024-05-10 14:56:49 +02:00
"Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/pawel/.local/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch) (12.4.127)\n",
2024-05-13 10:07:22 +02:00
"Requirement already satisfied: tqdm in /home/pawel/.local/lib/python3.10/site-packages (from torchtext) (4.66.2)\n",
2024-05-10 14:56:49 +02:00
"Requirement already satisfied: requests in /home/pawel/.local/lib/python3.10/site-packages (from torchtext) (2.31.0)\n",
"Requirement already satisfied: numpy in /home/pawel/.local/lib/python3.10/site-packages (from torchtext) (1.26.4)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in /home/pawel/.local/lib/python3.10/site-packages (from jinja2->torch) (2.1.5)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /home/pawel/.local/lib/python3.10/site-packages (from requests->torchtext) (2024.2.2)\n",
"Requirement already satisfied: idna<4,>=2.5 in /home/pawel/.local/lib/python3.10/site-packages (from requests->torchtext) (3.6)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /home/pawel/.local/lib/python3.10/site-packages (from requests->torchtext) (2.2.1)\n",
2024-05-13 10:07:22 +02:00
"Requirement already satisfied: charset-normalizer<4,>=2 in /home/pawel/.local/lib/python3.10/site-packages (from requests->torchtext) (3.3.2)\n",
"Requirement already satisfied: mpmath>=0.19 in /home/pawel/.local/lib/python3.10/site-packages (from sympy->torch) (1.3.0)\n"
2024-05-10 14:56:49 +02:00
]
}
],
"source": [
"!pip install torch torchtext"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 12,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"from collections import Counter\n",
"\n",
"import torch\n",
"from datasets import load_dataset\n",
"from torchtext.vocab import vocab\n",
"from tqdm.notebook import tqdm"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Wczytujemy zbiór danych `conll2003` (https://huggingface.co/datasets/conll2003), który zawiera teksty oznaczone znacznikami części mowy (*POS tags*): "
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 13,
2024-05-10 14:56:49 +02:00
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"dataset = load_dataset(\"conll2003\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Poiżej funkcja, która tworzy słownik (https://pytorch.org/text/stable/vocab.html).\n",
"\n",
"Parametr `special` określa symbole specjalne:\n",
"* `<unk>` –
"* `<pad>` –
"* `<bos>` –
"* `<eos>` –
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 14,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"def build_vocab(dataset):\n",
" counter = Counter()\n",
" for document in dataset:\n",
" counter.update(document)\n",
2024-05-13 10:07:22 +02:00
" return vocab(counter, specials=[\"<unk>\", \"<pad>\", \"<bos>\", \"<eos>\"])"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 15,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
2024-05-13 10:07:22 +02:00
"v = build_vocab(dataset[\"train\"][\"tokens\"])"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 16,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"itos = v.get_itos() # mapowanie indeksów na tokeny"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['<unk>', '<pad>', '<bos>', '<eos>', 'EU', 'rejects', 'German', 'call', 'to', 'boycott', 'British', 'lamb', '.', 'Peter', 'Blackburn', 'BRUSSELS', '1996-08-22', 'The', 'European', 'Commission', 'said', 'on', 'Thursday', 'it', 'disagreed', 'with', 'advice', 'consumers', 'shun', 'until', 'scientists', 'determine', 'whether', 'mad', 'cow', 'disease', 'can', 'be', 'transmitted', 'sheep', 'Germany', \"'s\", 'representative', 'the', 'Union', 'veterinary', 'committee', 'Werner', 'Zwingmann', 'Wednesday', 'should', 'buy', 'sheepmeat', 'from', 'countries', 'other', 'than', 'Britain', 'scientific', 'was', 'clearer', '\"', 'We', 'do', \"n't\", 'support', 'any', 'such', 'recommendation', 'because', 'we', 'see', 'grounds', 'for', ',', 'chief', 'spokesman', 'Nikolaus', 'van', 'der', 'Pas', 'told', 'a', 'news', 'briefing', 'He', 'further', 'study', 'required', 'and', 'if', 'found', 'that', 'action', 'needed', 'taken', 'by', 'proposal', 'last', 'month', 'Farm', 'Commissioner', 'Franz', 'Fischler', 'ban', 'brains', 'spleens', 'spinal', 'cords', 'human', 'animal', 'food', 'chains', 'highly', 'specific', 'precautionary', 'move', 'protect', 'health', 'proposed', 'EU-wide', 'measures', 'after', 'reports', 'France', 'under', 'laboratory', 'conditions', 'could', 'contract', 'Bovine', 'Spongiform', 'Encephalopathy', '(', 'BSE', ')', '--', 'But', 'agreed', 'review', 'his', 'standing', 'mational', 'officials', 'questioned', 'justified', 'as', 'there', 'only', 'slight', 'risk', 'Spanish', 'Minister', 'Loyola', 'de', 'Palacio', 'had', 'earlier', 'accused', 'at', 'an', 'farm', 'ministers', \"'\", 'meeting', 'of', 'causing', 'unjustified', 'alarm', 'through', 'dangerous', 'generalisation', 'Only', 'backed', 'multidisciplinary', 'committees', 'are', 'due', 're-examine', 'issue', 'early', 'next', 'make', 'recommendations', 'senior', 'Sheep', 'have', 'long', 'been', 'known', 'scrapie', 'brain-wasting', 'similar', 'which', 'is', 'believed', 'transferred', 'cattle', 'feed', 'containing', 'waste', 'farmers', 'denied', 'danger', 'their', 'but', 'expressed', 'concern', 'government', 'avoid', 'might', 'influence', 'across', 'Europe', 'What', 'extremely', 'careful', 'how', 'going', 'take', 'lead', 'Welsh', 'National', 'Farmers', 'NFU', 'chairman', 'John', 'Lloyd', 'Jones', 'BBC', 'radio', 'Bonn', 'has', 'led', 'efforts', 'public', 'consumer', 'confidence', 'collapsed', 'in', 'March', 'report', 'suggested', 'humans', 'illness', 'eating', 'contaminated', 'beef', 'imported', '47,600', 'year', 'nearly', 'half', 'total', 'imports', 'It', 'brought', '4,275', 'tonnes', 'mutton', 'some', '10', 'percent', 'overall', 'Rare', 'Hendrix', 'song', 'draft', 'sells', 'almost', '$', '17,000', 'LONDON', 'A', 'rare', 'handwritten', 'U.S.', 'guitar', 'legend', 'Jimi', 'sold', 'auction', 'late', 'musician', 'favourite', 'possessions', 'Florida', 'restaurant', 'paid', '10,925', 'pounds', '16,935', 'Ai', 'no', 'telling', 'penned', 'piece', 'London', 'hotel', 'stationery', '1966', 'At', 'end', 'January', '1967', 'concert', 'English', 'city', 'Nottingham', 'he', 'threw', 'sheet', 'paper', 'into', 'audience', 'where', 'retrieved', 'fan', 'Buyers', 'also', 'snapped', 'up', '16', 'items', 'were', 'put', 'former', 'girlfriend', 'Kathy', 'Etchingham', 'who', 'lived', 'him', '1969', 'They', 'included', 'black', 'lacquer', 'mother', 'pearl', 'inlaid', 'box', 'used', 'store', 'drugs', 'anonymous', 'Australian', 'purchaser', 'bought', '5,060', '7,845', 'guitarist', 'died', 'overdose', '1970', 'aged', '27', 'China', 'says', 'Taiwan', 'spoils', 'atmosphere', 'talks', 'BEIJING', 'Taipei', 'spoiling', 'resumption', 'Strait', 'visit', 'Ukraine', 'Taiwanese', 'Vice', 'President', 'Lien', 'Chan', 'this', 'week', 'infuriated', 'Beijing', 'Speaking', 'hours', 'Chinese', 'state', 'media', 'time', 'right', 'engage', 'political', 'Foreign', 'Ministry', 'Shen', 'Guofang', 'Reuters', ':', 'necessary', 'opening', 'disrupted', 'authorities', 'State', 'quoted', 'top', 'negotiator', 'Tang', 'Shubei', 'visiting', 'group', 'rivals', 'hold', 'Now', 'two', 'sides', '...', 'hostility', 'overseas', 'edition',
]
}
],
"source": [
"print(itos)"
]
},
{
"cell_type": "code",
"execution_count": 18,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"23627"
]
},
2024-05-13 10:07:22 +02:00
"execution_count": 18,
2024-05-10 14:56:49 +02:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(itos) # liczba różnych tokenów w słowniku"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 19,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
2024-05-13 10:07:22 +02:00
"5"
2024-05-10 14:56:49 +02:00
]
},
2024-05-13 10:07:22 +02:00
"execution_count": 19,
2024-05-10 14:56:49 +02:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2024-05-13 10:07:22 +02:00
"v[\"rejects\"] # indeks tokenu `on`"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 20,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0"
]
},
2024-05-13 10:07:22 +02:00
"execution_count": 20,
2024-05-10 14:56:49 +02:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"v[\"<unk>\"] # indeks nieznanego tokenu"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"W przypadku, gdy w analizowanym tekście znajdzie się token, którego nie ma w słowniku, będzie reprezentowany przez indeks domyślny (*default index*). Ustawiamy, żeby był taki sam, jak indeks „nieznanego tokenu”:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 21,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"v.set_default_index(v[\"<unk>\"])"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 22,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"def data_process(dt):\n",
" # Wektoryzacja dokumentów tekstowych.\n",
2024-05-13 10:07:22 +02:00
" return [\n",
" torch.tensor(\n",
" [v[\"<bos>\"]] + [v[token] for token in document] + [v[\"<eos>\"]],\n",
" dtype=torch.long,\n",
" )\n",
" for document in dt\n",
" ]"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 23,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"def labels_process(dt):\n",
2024-05-13 08:23:16 +02:00
" # Wektoryzacja etykiet (NER)\n",
2024-05-13 10:07:22 +02:00
" return [torch.tensor([0] + document + [0], dtype=torch.long) for document in dt]"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Teraz wektoryzujemy wszystkie dane:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 24,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
2024-05-13 10:07:22 +02:00
"train_tokens_ids = data_process(dataset[\"train\"][\"tokens\"])"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 25,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
2024-05-13 10:07:22 +02:00
"test_tokens_ids = data_process(dataset[\"test\"][\"tokens\"])"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 26,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
2024-05-13 10:07:22 +02:00
"validation_tokens_ids = data_process(dataset[\"validation\"][\"tokens\"])"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 27,
2024-05-10 14:56:49 +02:00
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
2024-05-13 10:07:22 +02:00
"train_labels = labels_process(dataset[\"train\"][\"ner_tags\"])"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 28,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
2024-05-13 10:07:22 +02:00
"validation_labels = labels_process(dataset[\"validation\"][\"ner_tags\"])"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 29,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
2024-05-13 10:07:22 +02:00
"test_labels = labels_process(dataset[\"test\"][\"ner_tags\"])"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Przykład, jak wyglądają dane po zwektoryzowaniu:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 30,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([ 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 3])"
]
},
2024-05-13 10:07:22 +02:00
"execution_count": 30,
2024-05-10 14:56:49 +02:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_tokens_ids[0]"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 31,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'id': '0',\n",
" 'tokens': ['EU',\n",
" 'rejects',\n",
" 'German',\n",
" 'call',\n",
" 'to',\n",
" 'boycott',\n",
" 'British',\n",
" 'lamb',\n",
" '.'],\n",
" 'pos_tags': [22, 42, 16, 21, 35, 37, 16, 21, 7],\n",
" 'chunk_tags': [11, 21, 11, 12, 21, 22, 11, 12, 0],\n",
" 'ner_tags': [3, 0, 7, 0, 0, 0, 7, 0, 0]}"
]
},
2024-05-13 10:07:22 +02:00
"execution_count": 31,
2024-05-10 14:56:49 +02:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
2024-05-13 10:07:22 +02:00
"dataset[\"train\"][0]"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 32,
2024-05-10 14:56:49 +02:00
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"tensor([0, 3, 0, 7, 0, 0, 0, 7, 0, 0, 0])"
]
},
2024-05-13 10:07:22 +02:00
"execution_count": 32,
2024-05-10 14:56:49 +02:00
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_labels[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Funkcja, której użyjemy do ewaluacji:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 33,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"def get_scores(y_true, y_pred):\n",
" # Funkcja zwraca precyzję, pokrycie i F1\n",
" acc_score = 0\n",
" tp = 0\n",
" fp = 0\n",
" selected_items = 0\n",
2024-05-13 10:07:22 +02:00
" relevant_items = 0\n",
2024-05-10 14:56:49 +02:00
"\n",
2024-05-13 10:07:22 +02:00
" for p, t in zip(y_pred, y_true):\n",
2024-05-10 14:56:49 +02:00
" if p == t:\n",
2024-05-13 10:07:22 +02:00
" acc_score += 1\n",
2024-05-10 14:56:49 +02:00
"\n",
" if p > 0 and p == t:\n",
2024-05-13 10:07:22 +02:00
" tp += 1\n",
2024-05-10 14:56:49 +02:00
"\n",
" if p > 0:\n",
" selected_items += 1\n",
"\n",
2024-05-13 10:07:22 +02:00
" if t > 0:\n",
" relevant_items += 1\n",
2024-05-10 14:56:49 +02:00
"\n",
" if selected_items == 0:\n",
" precision = 1.0\n",
" else:\n",
" precision = tp / selected_items\n",
2024-05-13 10:07:22 +02:00
"\n",
2024-05-10 14:56:49 +02:00
" if relevant_items == 0:\n",
" recall = 1.0\n",
" else:\n",
" recall = tp / relevant_items\n",
2024-05-13 10:07:22 +02:00
"\n",
" if precision + recall == 0.0:\n",
2024-05-10 14:56:49 +02:00
" f1 = 0.0\n",
" else:\n",
2024-05-13 10:07:22 +02:00
" f1 = 2 * precision * recall / (precision + recall)\n",
2024-05-10 14:56:49 +02:00
"\n",
" return precision, recall, f1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
2024-05-13 08:23:16 +02:00
"Ile mamy różnych tagów NER?"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 34,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"9\n"
]
}
],
"source": [
2024-05-13 10:07:22 +02:00
"num_tags = max([max(x) for x in dataset[\"train\"][\"ner_tags\"]]) + 1\n",
2024-05-10 14:56:49 +02:00
"print(num_tags)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Implementacja rekurencyjnej sieci neuronowej LSTM:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 35,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"class LSTM(torch.nn.Module):\n",
"\n",
" def __init__(self):\n",
" super(LSTM, self).__init__()\n",
2024-05-13 10:07:22 +02:00
" self.emb = torch.nn.Embedding(len(v.get_itos()), 100)\n",
" self.rec = torch.nn.LSTM(100, 256, 1, batch_first=True)\n",
" self.fc1 = torch.nn.Linear(256, num_tags)\n",
2024-05-10 14:56:49 +02:00
"\n",
" def forward(self, x):\n",
" emb = torch.relu(self.emb(x))\n",
" lstm_output, (h_n, c_n) = self.rec(emb)\n",
" out_weights = self.fc1(lstm_output)\n",
" return out_weights"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Stworzenie modelu:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 36,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"lstm = LSTM()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Definicja funkcji kosztu:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 37,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"criterion = torch.nn.CrossEntropyLoss()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Definicja optymalizatora:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 38,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"optimizer = torch.optim.Adam(lstm.parameters())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Funkcja do ewaluacji modelu:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 39,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"def eval_model(dataset_tokens, dataset_labels, model):\n",
" Y_true = []\n",
" Y_pred = []\n",
" for i in tqdm(range(len(dataset_labels))):\n",
" batch_tokens = dataset_tokens[i].unsqueeze(0)\n",
" tags = list(dataset_labels[i].numpy())\n",
" Y_true += tags\n",
2024-05-13 10:07:22 +02:00
"\n",
2024-05-10 14:56:49 +02:00
" Y_batch_pred_weights = model(batch_tokens).squeeze(0)\n",
2024-05-13 10:07:22 +02:00
" Y_batch_pred = torch.argmax(Y_batch_pred_weights, 1)\n",
2024-05-10 14:56:49 +02:00
" Y_pred += list(Y_batch_pred.numpy())\n",
"\n",
2024-05-13 10:07:22 +02:00
" return get_scores(Y_true, Y_pred)"
2024-05-10 14:56:49 +02:00
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Uczenie modelu:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 40,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [],
"source": [
"NUM_EPOCHS = 5"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": 41,
2024-05-10 14:56:49 +02:00
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
2024-05-13 10:07:22 +02:00
"model_id": "6b09e994ea71476e8e8600f708aca2ea",
2024-05-10 14:56:49 +02:00
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0/14041 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
2024-05-13 10:07:22 +02:00
"ename": "KeyboardInterrupt",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[41], line 16\u001b[0m\n\u001b[1;32m 13\u001b[0m loss \u001b[38;5;241m=\u001b[39m criterion(predicted_tags\u001b[38;5;241m.\u001b[39msqueeze(\u001b[38;5;241m0\u001b[39m),tags\u001b[38;5;241m.\u001b[39msqueeze(\u001b[38;5;241m1\u001b[39m))\n\u001b[1;32m 15\u001b[0m loss\u001b[38;5;241m.\u001b[39mbackward()\n\u001b[0;32m---> 16\u001b[0m \u001b[43moptimizer\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mstep\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 18\u001b[0m lstm\u001b[38;5;241m.\u001b[39meval()\n\u001b[1;32m 19\u001b[0m \u001b[38;5;28mprint\u001b[39m(eval_model(validation_tokens_ids, validation_labels, lstm))\n",
"File \u001b[0;32m~/.local/lib/python3.10/site-packages/torch/optim/optimizer.py:391\u001b[0m, in \u001b[0;36mOptimizer.profile_hook_step.<locals>.wrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m 386\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 387\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mRuntimeError\u001b[39;00m(\n\u001b[1;32m 388\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mfunc\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m must return None or a tuple of (new_args, new_kwargs), but got \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mresult\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 389\u001b[0m )\n\u001b[0;32m--> 391\u001b[0m out \u001b[38;5;241m=\u001b[39m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 392\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_optimizer_step_code()\n\u001b[1;32m 394\u001b[0m \u001b[38;5;66;03m# call optimizer step post hooks\u001b[39;00m\n",
"File \u001b[0;32m~/.local/lib/python3.10/site-packages/torch/optim/optimizer.py:76\u001b[0m, in \u001b[0;36m_use_grad_for_differentiable.<locals>._use_grad\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m 74\u001b[0m torch\u001b[38;5;241m.\u001b[39mset_grad_enabled(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mdefaults[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mdifferentiable\u001b[39m\u001b[38;5;124m'\u001b[39m])\n\u001b[1;32m 75\u001b[0m torch\u001b[38;5;241m.\u001b[39m_dynamo\u001b[38;5;241m.\u001b[39mgraph_break()\n\u001b[0;32m---> 76\u001b[0m ret \u001b[38;5;241m=\u001b[39m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 77\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m 78\u001b[0m torch\u001b[38;5;241m.\u001b[39m_dynamo\u001b[38;5;241m.\u001b[39mgraph_break()\n",
"File \u001b[0;32m~/.local/lib/python3.10/site-packages/torch/optim/adam.py:168\u001b[0m, in \u001b[0;36mAdam.step\u001b[0;34m(self, closure)\u001b[0m\n\u001b[1;32m 157\u001b[0m beta1, beta2 \u001b[38;5;241m=\u001b[39m group[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mbetas\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[1;32m 159\u001b[0m has_complex \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_init_group(\n\u001b[1;32m 160\u001b[0m group,\n\u001b[1;32m 161\u001b[0m params_with_grad,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 165\u001b[0m max_exp_avg_sqs,\n\u001b[1;32m 166\u001b[0m state_steps)\n\u001b[0;32m--> 168\u001b[0m \u001b[43madam\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 169\u001b[0m \u001b[43m \u001b[49m\u001b[43mparams_with_grad\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 170\u001b[0m \u001b[43m \u001b[49m\u001b[43mgrads\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 171\u001b[0m \u001b[43m \u001b[49m\u001b[43mexp_avgs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 172\u001b[0m \u001b[43m \u001b[49m\u001b[43mexp_avg_sqs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 173\u001b[0m \u001b[43m \u001b[49m\u001b[43mmax_exp_avg_sqs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 174\u001b[0m \u001b[43m \u001b[49m\u001b[43mstate_steps\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 175\u001b[0m \u001b[43m \u001b[49m\u001b[43mamsgrad\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgroup\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mamsgrad\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 176\u001b[0m \u001b[43m \u001b[49m\u001b[43mhas_complex\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mhas_complex\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 177\u001b[0m \u001b[43m \u001b[49m\u001b[43mbeta1\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mbeta1\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 178\u001b[0m \u001b[43m \u001b[49m\u001b[43mbeta2\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mbeta2\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 179\u001b[0m \u001b[43m \u001b[49m\u001b[43mlr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgroup\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mlr\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 180\u001b[0m \u001b[43m \u001b[49m\u001b[43mweight_decay\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgroup\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mweight_decay\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 181\u001b[0m \u001b[43m \u001b[49m\u001b[43meps\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgroup\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43meps\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 182\u001b[0m \u001b[43m \u001b[49m\u001b[43mmaximize\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgroup\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mmaximize\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 183\u001b[0m \u001b[43m \u001b[49m\u001b[43mforeach\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgroup\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mforeach\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 184\u001b[0m \u001b[43m \u001b[49m\u001b[43mcapturable\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgroup\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mcapturable\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\
"File \u001b[0;32m~/.local/lib/python3.10/site-packages/torch/optim/adam.py:318\u001b[0m, in \u001b[0;36madam\u001b[0;34m(params, grads, exp_avgs, exp_avg_sqs, max_exp_avg_sqs, state_steps, foreach, capturable, differentiable, fused, grad_scale, found_inf, has_complex, amsgrad, beta1, beta2, lr, weight_decay, eps, maximize)\u001b[0m\n\u001b[1;32m 315\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 316\u001b[0m func \u001b[38;5;241m=\u001b[39m _single_tensor_adam\n\u001b[0;32m--> 318\u001b[0m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[43mparams\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 319\u001b[0m \u001b[43m \u001b[49m\u001b[43mgrads\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 320\u001b[0m \u001b[43m \u001b[49m\u001b[43mexp_avgs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 321\u001b[0m \u001b[43m \u001b[49m\u001b[43mexp_avg_sqs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 322\u001b[0m \u001b[43m \u001b[49m\u001b[43mmax_exp_avg_sqs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 323\u001b[0m \u001b[43m \u001b[49m\u001b[43mstate_steps\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 324\u001b[0m \u001b[43m \u001b[49m\u001b[43mamsgrad\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mamsgrad\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 325\u001b[0m \u001b[43m \u001b[49m\u001b[43mhas_complex\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mhas_complex\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 326\u001b[0m \u001b[43m \u001b[49m\u001b[43mbeta1\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mbeta1\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 327\u001b[0m \u001b[43m \u001b[49m\u001b[43mbeta2\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mbeta2\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 328\u001b[0m \u001b[43m \u001b[49m\u001b[43mlr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mlr\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 329\u001b[0m \u001b[43m \u001b[49m\u001b[43mweight_decay\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mweight_decay\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 330\u001b[0m \u001b[43m \u001b[49m\u001b[43meps\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43meps\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 331\u001b[0m \u001b[43m \u001b[49m\u001b[43mmaximize\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmaximize\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 332\u001b[0m \u001b[43m \u001b[49m\u001b[43mcapturable\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcapturable\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 333\u001b[0m \u001b[43m \u001b[49m\u001b[43mdifferentiable\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mdifferentiable\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 334\u001b[0m \u001b[43m \u001b[49m\u001b[43mgrad_scale\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mgrad_scale\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 335\u001b[0m \u001b[43m \u001b[49m\u001b[43mfound_inf\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfound_inf\u001b[49m\u001b[43m)\u001b[49m\n",
"File \u001b[0;32m~/.local/lib/python3.10/site-packages/torch/optim/adam.py:393\u001b[0m, in \u001b[0;36m_single_tensor_adam\u001b[0;34m(params, grads, exp_avgs, exp_avg_sqs, max_exp_avg_sqs, state_steps, grad_scale, found_inf, amsgrad, has_complex, beta1, beta2, lr, weight_decay, eps, maximize, capturable, differentiable)\u001b[0m\n\u001b[1;32m 390\u001b[0m param \u001b[38;5;241m=\u001b[39m torch\u001b[38;5;241m.\u001b[39mview_as_real(param)\n\u001b[1;32m 392\u001b[0m \u001b[38;5;66;03m# Decay the first and second moment running average coefficient\u001b[39;00m\n\u001b[0;32m--> 393\u001b[0m \u001b[43mexp_avg\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mlerp_\u001b[49m\u001b[43m(\u001b[49m\u001b[43mgrad\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m1\u001b[39;49m\u001b[43m \u001b[49m\u001b[38;5;241;43m-\u001b[39;49m\u001b[43m \u001b[49m\u001b[43mbeta1\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 394\u001b[0m exp_avg_sq\u001b[38;5;241m.\u001b[39mmul_(beta2)\u001b[38;5;241m.\u001b[39maddcmul_(grad, grad\u001b[38;5;241m.\u001b[39mconj(), value\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m1\u001b[39m \u001b[38;5;241m-\u001b[39m beta2)\n\u001b[1;32m 396\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m capturable \u001b[38;5;129;01mor\u001b[39;00m differentiable:\n",
"\u001b[0;31mKeyboardInterrupt\u001b[0m: "
2024-05-10 14:56:49 +02:00
]
}
],
"source": [
"for i in range(NUM_EPOCHS):\n",
" lstm.train()\n",
2024-05-13 10:07:22 +02:00
" # for i in tqdm(range(500)):\n",
2024-05-10 14:56:49 +02:00
" for i in tqdm(range(len(train_labels))):\n",
" batch_tokens = train_tokens_ids[i].unsqueeze(0)\n",
" tags = train_labels[i].unsqueeze(1)\n",
2024-05-13 10:07:22 +02:00
"\n",
2024-05-10 14:56:49 +02:00
" predicted_tags = lstm(batch_tokens)\n",
"\n",
" optimizer.zero_grad()\n",
2024-05-13 10:07:22 +02:00
" loss = criterion(predicted_tags.squeeze(0), tags.squeeze(1))\n",
"\n",
2024-05-10 14:56:49 +02:00
" loss.backward()\n",
" optimizer.step()\n",
2024-05-13 10:07:22 +02:00
"\n",
2024-05-10 14:56:49 +02:00
" lstm.eval()\n",
" print(eval_model(validation_tokens_ids, validation_labels, lstm))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Ewaluacja:"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": null,
2024-05-10 14:56:49 +02:00
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "d494e8de77cc4597b07eb4bbaff1d241",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0/3250 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"(0.7134837896666285, 0.7239335115657329, 0.7186706669743826)"
]
},
"execution_count": 61,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"eval_model(validation_tokens_ids, validation_labels, lstm)"
]
},
{
"cell_type": "code",
2024-05-13 10:07:22 +02:00
"execution_count": null,
2024-05-10 14:56:49 +02:00
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "a221459483784b94bc2251b8dae6bbba",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0/3453 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"(0.6529463280370325, 0.6433678500986193, 0.6481217013349891)"
]
},
"execution_count": 62,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"eval_model(test_tokens_ids, test_labels, lstm)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Zadanie 3\n",
"\n",
"Sklonuj repozytorium https://git.wmi.amu.edu.pl/kubapok/en-ner-conll-2003\n",
"\n",
2024-05-13 10:07:22 +02:00
"Stwórz model *sequence labelling* realizujący zadanie NER, oparty o dowolną rekurencyjną sieć neuronową (możesz wzorować się na przykładzie z zajęć).\n",
2024-05-10 14:56:49 +02:00
"\n",
"W plikach dev-0/out.tsv oraz test-A/out.tsv umieść wyniki predykcji dla dev-0/in.tsv i test-A/in.tsv odpowiednio.\n",
"Do ewaluacji wykorzystaj narzędzie GEval (https://gitlab.com/filipg/geval):\n",
"\n",
" wget https://gonito.net/get/bin/geval\n",
" chmod u+x geval\n",
" ./geval --help\n",
"\n",
"Liczba punktów uzyskanych za zadanie zależy od uzyskanej wartości accuracy na zbiorze `test-A` (wynik zaokrąglony w górę):\n",
"\n",
" points = math.ceil(accuracy * 7.0)\n",
"\n",
"⚠️ W systemie Moodle proszę załączyć plik `test-A/out.tsv` oraz link do repozytorium z rozwiązaniem zadania.\n",
" "
]
}
],
"metadata": {
"author": "Jakub Pokrywka",
"email": "kubapok@wmi.amu.edu.pl",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"lang": "pl",
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
},
"subtitle": "11.NER RNN[ćwiczenia]",
"title": "Ekstrakcja informacji",
"year": "2021"
},
"nbformat": 4,
"nbformat_minor": 4
}
