kdudzic/aitech-sd-lab
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

add evaluation

Browse Source
This commit is contained in:
Kacper 2022-05-06 14:49:03 +02:00
parent 4de73a4a11
commit 273f20e2a5
2 changed files with 50 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
NLU_lab_7-8/evaluation.txt Normal file
View File

@ -0,0 +1,38 @@
*** This evaluation file was generated automatically by the training script ***
Results:
- F-score (micro) 0.2609
- F-score (macro) 0.1509
- Accuracy 0.1648
By class:
precision recall f1-score support
quantity 0.3846 0.8333 0.5263 6
time 0.3333 0.4286 0.3750 7
title 0.3333 0.2222 0.2667 9
goal 0.0000 0.0000 0.0000 10
area 0.0000 0.0000 0.0000 3
name 0.7500 0.6000 0.6667 5
date 0.3333 0.3333 0.3333 3
interval 0.0000 0.0000 0.0000 1
seat 0.0000 0.0000 0.0000 3
ticketnumber 0.0000 0.0000 0.0000 3
e-mail 0.0000 0.0000 0.0000 3
phone 1.0000 1.0000 1.0000 1
row 0.0000 0.0000 0.0000 2
movie 0.0000 0.0000 0.0000 2
reducedQuantity 0.0000 0.0000 0.0000 2
seats 0.0000 0.0000 0.0000 0
purchaseType 0.0000 0.0000 0.0000 1
bankAccountNumber 0.0000 0.0000 0.0000 1
email 0.0000 0.0000 0.0000 1
hour 0.0000 0.0000 0.0000 1
seatPlacement 0.0000 0.0000 0.0000 1
micro avg 0.3000 0.2308 0.2609 65
macro avg 0.1493 0.1627 0.1509 65
weighted avg 0.2060 0.2308 0.2079 65
samples avg 0.1648 0.1648 0.1648 65
2022-05-02 19:47:10,324 ----------------------------------------------------------------------------------------------------

12
NLU_lab_7-8/main.py
View File

@ -81,5 +81,17 @@ try:
model = SequenceTagger.load('slot-model-pl/best-model.pt')
except:
model = SequenceTagger.load('slot-model-pl/final-model.pt')
log_file = open('slot-model-pl/training.log', encoding='utf-8')
log_lines = log_file.readlines()
log_file.close()
with open('slot-model-pl/training.log', encoding='utf-8') as log_file, open('evaluation.txt', 'w', encoding='utf-8') \
as eval_file:
for num, line in enumerate(log_file):
if line == 'Results:\n':
lines_to_write_start = num
eval_file.write('*** This evaluation file was generated automatically by the training script ***\n\n')
for line in log_lines[lines_to_write_start:]:
eval_file.write(line)
print(tabulate(predict(model, 'Jeden bilet na imię Jan Kowalski na film Batman'.split())))