update and improve README

This commit is contained in:
Filip Gralinski 2018-06-11 21:29:39 +02:00
parent 4b06f59d3a
commit 066d8bbe88

View File

@ -13,6 +13,10 @@ The official repository is `git://gonito.net/geval`, browsable at
## Installing ## Installing
You need [Haskell Stack](https://github.com/commercialhaskell/stack). You need [Haskell Stack](https://github.com/commercialhaskell/stack).
You could install Stack with your package manager or with:
curl -sSL https://get.haskellstack.org/ | sh
When you've got Haskell Stack, install GEval with: When you've got Haskell Stack, install GEval with:
git clone git://gonito.net/geval git clone git://gonito.net/geval
@ -20,12 +24,23 @@ When you've got Haskell Stack, install GEval with:
stack setup stack setup
stack install stack install
(Note that when you're running Haskell Stack for the first time it
will take some time and a couple of gigabytes on your disk.)
By default, `geval` binary is installed in `$HOME/.local/bin`, so in By default, `geval` binary is installed in `$HOME/.local/bin`, so in
order to run `geval` you need to either add `$HOME/.local/bin` to order to run `geval` you need to either add `$HOME/.local/bin` to
`$PATH` or to type: `$PATH` in your configuration or to type:
PATH="$HOME/.local/bin" geval ... PATH="$HOME/.local/bin" geval ...
### Plan B — just download the GEval binary
(Assuming you have a standard 64-bit Linux.)
wget https://gonito.net/get/bin/geval
chmod u+x geval
./geval --help
## Preparing a Gonito challenge ## Preparing a Gonito challenge
### Directory structure of a Gonito challenge ### Directory structure of a Gonito challenge
@ -157,29 +172,48 @@ Then let Gonito pull them and evaluate your results.
## `geval` options ## `geval` options
geval [--init] [--precision PRECISION] [--out-directory OUT-DIRECTORY] Usage: geval ([--init] | [-l|--line-by-line] | [-d|--diff OTHER-OUT])
[--expected-directory EXPECTED-DIRECTORY] [--test-name NAME] ([-s|--sort] | [-r|--reverse-sort]) [--out-directory OUT-DIRECTORY]
[--out-file OUT] [--expected-file EXPECTED] [--metric METRIC] [--expected-directory EXPECTED-DIRECTORY] [-t|--test-name NAME]
[-o|--out-file OUT] [-e|--expected-file EXPECTED]
[-i|--input-file INPUT] [-a|--alt-metric METRIC]
[-m|--metric METRIC] [-p|--precision NUMBER-OF-FRACTIONAL-DIGITS]
Run evaluation for tests in Gonito platform
-h,--help Show this help text Available options:
--init Init a sample Gonito challenge rather than run an -h,--help Show this help text
evaluation --init Init a sample Gonito challenge rather than run an
--precision PRECISION Precision with which the evaluation results should be evaluation
shown -l,--line-by-line Give scores for each line rather than the whole test
--out-directory OUT-DIRECTORY set
Directory with test results to be -d,--diff OTHER-OUT compare results
evaluated (default: ".") -s,--sort When in line-by-line or diff mode, sort the results
--expected-directory EXPECTED-DIRECTORY from the worst to the best
Directory with expected test results (the same as -r,--reverse-sort When in line-by-line or diff mode, sort the results
OUT-DIRECTORY, if not given) from the best to the worst
--test-name NAME Test name (i.e. subdirectory with results or expected --out-directory OUT-DIRECTORY
results) (default: "test-A") Directory with test results to be
--out-file OUT The name of the file to be evaluated (default: ".")
evaluated (default: "out.tsv") --expected-directory EXPECTED-DIRECTORY
--expected-file EXPECTED The name of the file with expected Directory with expected test results (the same as
results (default: "expected.tsv") OUT-DIRECTORY, if not given)
--metric METRIC Metric to be used - RMSE, MSE, Accuracy or -t,--test-name NAME Test name (i.e. subdirectory with results or expected
BLEU (default: RMSE) results) (default: "test-A")
-o,--out-file OUT The name of the file to be
evaluated (default: "out.tsv")
-e,--expected-file EXPECTED
The name of the file with expected
results (default: "expected.tsv")
-i,--input-file INPUT The name of the file with the input (applicable only
for some metrics) (default: "in.tsv")
-a,--alt-metric METRIC Alternative metric (overrides --metric option)
-m,--metric METRIC Metric to be used - RMSE, MSE, Accuracy, LogLoss,
Likelihood, F-measure (specify as F1, F2, F0.25,
etc.), MAP, BLEU, NMI, ClippEU, LogLossHashed,
LikelihoodHashed, BIO-F1, BIO-F1-Labels or CharMatch
-p,--precision NUMBER-OF-FRACTIONAL-DIGITS
Arithmetic precision, i.e. the number of fractional
digits to be shown
If you need another metric, let me know, or do it yourself! If you need another metric, let me know, or do it yourself!