filipg/geval
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add an example

Browse Source
This commit is contained in:
Filip Gralinski 2018-10-19 14:58:29 +02:00
parent c27a989b3f
commit bbfb9821fc
1 changed files with 106 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

106
README.md
View File

@ -303,6 +303,112 @@ be hard to interpret, so you could try other metrics.
So now you can see that the accuracy is over 78% and the likelihood So now you can see that the accuracy is over 78% and the likelihood
(i.e. geometric mean of probabilities of the correct classes) is 0.62. (i.e. geometric mean of probabilities of the correct classes) is 0.62.
## Yet another example
geval --metric MultiLabel-F1 -e https://gonito.net/gitlist/poleval-2018-ner.git/raw/submission-02284/dev-0/expected.tsv -o https://gonito.net/gitlist/poleval-2018-ner.git/raw/submission-02284/dev-0/out.tsv -i https://gonito.net/gitlist/poleval-2018-ner.git/raw/submission-02284/dev-0/in.tsv -w | head -n 100
exp:persName.addName 40 0.57266043 0.00000000000000045072
exp:persName 529 0.87944043 0.00000000000026284497
out:persName 526 0.89273910 0.00000000000189290814
exp:orgName 259 0.85601779 0.00000000009060752668
exp:1 234 0.81006729 0.00000004388133229664
exp:persName.forename 369 0.89791618 0.00000071680839330093
out:persName.surname 295 0.91783693 0.00000383192943077228
exp:placeName.region 32 0.83566990 0.00000551293116462680
out:5 82 0.85116074 0.00000607788112334637
exp:geogName 73 0.77593244 0.00000632581466839333
exp:placeName.settlement 167 0.87590291 0.00000690938211727142
exp:3 76 0.82971415 0.00000814340048123796
exp:6 75 0.89089104 0.00001275304858586339
out:persName.forename 362 0.92159232 0.00001426230958467042
exp:5 80 0.88315404 0.00002873600974251028
out:6 73 0.88823384 0.00004347998129569157
out:placeName.country 117 0.91174320 0.00005844859302012576
exp:27 14 0.89859509 0.00010111139128096410
out:2 106 0.87870029 0.00012339467984127947
exp:2 106 0.89150352 0.00013927462137254036
out:placeName.settlement 161 0.91193317 0.00015801636090376342
exp:10 55 0.88490168 0.00019500445941971885
out:10 55 0.88952978 0.00020384459146120533
out:27 13 0.83260073 0.00022093811378190520
exp:11 50 0.91544979 0.00022538447932126170
exp:persName.surname 284 0.94568239 0.00029790914546478866
out:geogName 68 0.87991682 0.00033570934160678480
exp:25 14 0.83275422 0.00034911992940182120
exp:20 23 0.86023258 0.00037403771750947510
out:orgName 228 0.93054071 0.00041409255783249570
exp:placeName.bloc 4 0.25000000 0.00058004178654680340
out:placeName 4 0.45288462 0.00079963839942791270
exp:placeName 4 0.55288462 0.00090031630413270230
exp:placeName.district 6 0.54575163 0.00093126444116410190
out:25 13 0.84259978 0.00098291350949343270
exp:18 33 0.90467916 0.00099014945726474700
exp:placeName.country 111 0.92607628 0.00103154555626810890
out:persName.addName 16 0.85999111 0.00103238048710726150
exp:1,2,3 11 0.71733569 0.00104285196244713480
exp:9 70 0.89791862 0.00109937869723650940
out:1,2,3 11 0.75929374 0.00112334313326076900
out:15 30 0.90901990 0.00132066041179418900
exp:15 30 0.91710071 0.00139871001216425860
out:14 48 0.90205283 0.00145838060555712980
out:36 6 0.74672188 0.00146644432521086550
exp:26 14 0.86061091 0.00169416966498835550
out:26 14 0.86434574 0.00172101465871527430
in<1>:Chrystus 6 0.86234615 0.00178789911479861950
out:9 69 0.89843853 0.00182996622711856130
exp:26,27 4 0.86532091 0.00187926423000622310
out:26,27 4 0.86532091 0.00187926423000622310
out:4 87 0.89070069 0.00193025851603233500
out:18 32 0.91509324 0.00208916541118153300
exp:14 47 0.89135689 0.00247634067123241170
exp:8 71 0.91390223 0.00248155467568570200
out:3 67 0.89624455 0.00251005273204463700
exp:13,14,15 7 0.69047619 0.00264339993981820200
out:11 46 0.94453652 0.00300877389223088140
exp:13 39 0.89762050 0.00304040573378035300
exp:25,26 7 0.72969188 0.00305728291769260170
in<1>:ku 3 0.64285714 0.00409664186965377500
exp:24 13 0.84446849 0.00422204049045033550
in<1>:Szkole 3 0.69841270 0.00459053755028235000
in<1>:gmina 3 0.72619048 0.00471502973611559400
out:23,24,25,26,27 3 0.74444444 0.00478548560174827300
exp:35 3 0.73479853 0.00479495029982456600
out:35 3 0.73479853 0.00479495029982456600
out:20 20 0.91318903 0.00505032866577808350
in<1>:SJL 6 0.40000000 0.00510505196247920600
exp:36 5 0.84704664 0.00533176800260401500
exp:23 17 0.88215614 0.00535729183315928400
out:13 38 0.90181485 0.00563103165587168000
in<1>:przykład 12 0.63611111 0.00619614049735634600
in<1>:" 184 0.89698360 0.00671336491979657000
exp:22 18 0.86584897 0.00678536930472158100
exp:5,6 21 0.92398078 0.00701181665145694000
exp:32 11 0.87372682 0.00725144019981003500
in<1>:bycia 4 0.25000000 0.00765937730815748400
exp:4 84 0.90829786 0.00781071034965166500
exp:7 69 0.87580842 0.00825171941550910600
in<1>:11 6 0.68919969 0.00833858334198865600
exp:17 35 0.92766981 0.00901683910684479200
in<1>:Ochlapusem 2 0.00000000 0.00911768813656929300
in<1>:Wydra 2 0.00000000 0.00911768813656929300
in<1>:molo 2 0.00000000 0.00911768813656929300
in<1>:samą 2 0.00000000 0.00911768813656929300
out:placeName.region 23 0.89830894 0.00950994259651506200
out:1 206 0.91410839 0.01028654356654566000
out:25,26 6 0.78464052 0.01052324370840473200
in<1>:wynikiem 2 0.25000000 0.01083031507722793800
in<1>:czci 2 0.28571429 0.01131535182961013700
in<1>:obejrzał 2 0.33333333 0.01146449651732581700
exp:2,3,4,5,6 2 0.36666667 0.01174236718700471900
exp:12 48 0.91708259 0.01199411048538193800
in<1>:przyszedł 4 0.61666667 0.01206312763924867500
in<1>:zachowania 2 0.45000000 0.01231568593500110600
in<1>:Bacha 2 0.41666667 0.01343470684272302300
in<1>:grobu 4 0.74166667 0.01357123871263958600
in<1>:Brytania 2 0.53333333 0.01357876718525224600
in<1>:rewolucja 2 0.53333333 0.01357876718525224600
## Preparing a Gonito challenge ## Preparing a Gonito challenge
### Directory structure of a Gonito challenge ### Directory structure of a Gonito challenge