update and improve README
This commit is contained in:
parent
4b06f59d3a
commit
066d8bbe88
80
README.md
80
README.md
@ -13,6 +13,10 @@ The official repository is `git://gonito.net/geval`, browsable at
|
|||||||
## Installing
|
## Installing
|
||||||
|
|
||||||
You need [Haskell Stack](https://github.com/commercialhaskell/stack).
|
You need [Haskell Stack](https://github.com/commercialhaskell/stack).
|
||||||
|
You could install Stack with your package manager or with:
|
||||||
|
|
||||||
|
curl -sSL https://get.haskellstack.org/ | sh
|
||||||
|
|
||||||
When you've got Haskell Stack, install GEval with:
|
When you've got Haskell Stack, install GEval with:
|
||||||
|
|
||||||
git clone git://gonito.net/geval
|
git clone git://gonito.net/geval
|
||||||
@ -20,12 +24,23 @@ When you've got Haskell Stack, install GEval with:
|
|||||||
stack setup
|
stack setup
|
||||||
stack install
|
stack install
|
||||||
|
|
||||||
|
(Note that when you're running Haskell Stack for the first time it
|
||||||
|
will take some time and a couple of gigabytes on your disk.)
|
||||||
|
|
||||||
By default, `geval` binary is installed in `$HOME/.local/bin`, so in
|
By default, `geval` binary is installed in `$HOME/.local/bin`, so in
|
||||||
order to run `geval` you need to either add `$HOME/.local/bin` to
|
order to run `geval` you need to either add `$HOME/.local/bin` to
|
||||||
`$PATH` or to type:
|
`$PATH` in your configuration or to type:
|
||||||
|
|
||||||
PATH="$HOME/.local/bin" geval ...
|
PATH="$HOME/.local/bin" geval ...
|
||||||
|
|
||||||
|
### Plan B — just download the GEval binary
|
||||||
|
|
||||||
|
(Assuming you have a standard 64-bit Linux.)
|
||||||
|
|
||||||
|
wget https://gonito.net/get/bin/geval
|
||||||
|
chmod u+x geval
|
||||||
|
./geval --help
|
||||||
|
|
||||||
## Preparing a Gonito challenge
|
## Preparing a Gonito challenge
|
||||||
|
|
||||||
### Directory structure of a Gonito challenge
|
### Directory structure of a Gonito challenge
|
||||||
@ -157,29 +172,48 @@ Then let Gonito pull them and evaluate your results.
|
|||||||
|
|
||||||
## `geval` options
|
## `geval` options
|
||||||
|
|
||||||
geval [--init] [--precision PRECISION] [--out-directory OUT-DIRECTORY]
|
Usage: geval ([--init] | [-l|--line-by-line] | [-d|--diff OTHER-OUT])
|
||||||
[--expected-directory EXPECTED-DIRECTORY] [--test-name NAME]
|
([-s|--sort] | [-r|--reverse-sort]) [--out-directory OUT-DIRECTORY]
|
||||||
[--out-file OUT] [--expected-file EXPECTED] [--metric METRIC]
|
[--expected-directory EXPECTED-DIRECTORY] [-t|--test-name NAME]
|
||||||
|
[-o|--out-file OUT] [-e|--expected-file EXPECTED]
|
||||||
|
[-i|--input-file INPUT] [-a|--alt-metric METRIC]
|
||||||
|
[-m|--metric METRIC] [-p|--precision NUMBER-OF-FRACTIONAL-DIGITS]
|
||||||
|
Run evaluation for tests in Gonito platform
|
||||||
|
|
||||||
-h,--help Show this help text
|
Available options:
|
||||||
--init Init a sample Gonito challenge rather than run an
|
-h,--help Show this help text
|
||||||
evaluation
|
--init Init a sample Gonito challenge rather than run an
|
||||||
--precision PRECISION Precision with which the evaluation results should be
|
evaluation
|
||||||
shown
|
-l,--line-by-line Give scores for each line rather than the whole test
|
||||||
--out-directory OUT-DIRECTORY
|
set
|
||||||
Directory with test results to be
|
-d,--diff OTHER-OUT compare results
|
||||||
evaluated (default: ".")
|
-s,--sort When in line-by-line or diff mode, sort the results
|
||||||
--expected-directory EXPECTED-DIRECTORY
|
from the worst to the best
|
||||||
Directory with expected test results (the same as
|
-r,--reverse-sort When in line-by-line or diff mode, sort the results
|
||||||
OUT-DIRECTORY, if not given)
|
from the best to the worst
|
||||||
--test-name NAME Test name (i.e. subdirectory with results or expected
|
--out-directory OUT-DIRECTORY
|
||||||
results) (default: "test-A")
|
Directory with test results to be
|
||||||
--out-file OUT The name of the file to be
|
evaluated (default: ".")
|
||||||
evaluated (default: "out.tsv")
|
--expected-directory EXPECTED-DIRECTORY
|
||||||
--expected-file EXPECTED The name of the file with expected
|
Directory with expected test results (the same as
|
||||||
results (default: "expected.tsv")
|
OUT-DIRECTORY, if not given)
|
||||||
--metric METRIC Metric to be used - RMSE, MSE, Accuracy or
|
-t,--test-name NAME Test name (i.e. subdirectory with results or expected
|
||||||
BLEU (default: RMSE)
|
results) (default: "test-A")
|
||||||
|
-o,--out-file OUT The name of the file to be
|
||||||
|
evaluated (default: "out.tsv")
|
||||||
|
-e,--expected-file EXPECTED
|
||||||
|
The name of the file with expected
|
||||||
|
results (default: "expected.tsv")
|
||||||
|
-i,--input-file INPUT The name of the file with the input (applicable only
|
||||||
|
for some metrics) (default: "in.tsv")
|
||||||
|
-a,--alt-metric METRIC Alternative metric (overrides --metric option)
|
||||||
|
-m,--metric METRIC Metric to be used - RMSE, MSE, Accuracy, LogLoss,
|
||||||
|
Likelihood, F-measure (specify as F1, F2, F0.25,
|
||||||
|
etc.), MAP, BLEU, NMI, ClippEU, LogLossHashed,
|
||||||
|
LikelihoodHashed, BIO-F1, BIO-F1-Labels or CharMatch
|
||||||
|
-p,--precision NUMBER-OF-FRACTIONAL-DIGITS
|
||||||
|
Arithmetic precision, i.e. the number of fractional
|
||||||
|
digits to be shown
|
||||||
|
|
||||||
If you need another metric, let me know, or do it yourself!
|
If you need another metric, let me know, or do it yourself!
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user