Improvements in README

This commit is contained in:
Filip Gralinski 2019-11-16 10:27:03 +01:00
parent 9521f0d931
commit fb74f568bb

View File

@ -2,7 +2,7 @@
GEval is a Haskell library and a stand-alone tool for evaluating the GEval is a Haskell library and a stand-alone tool for evaluating the
results of solutions to machine learning challenges as defined in the results of solutions to machine learning challenges as defined in the
[Gonito](https://gonito.net) platform. Also could be used outside the [Gonito](https://gonito.net) platform. Also, could be used outside the
context of Gonito.net challenges, assuming the test data is given in context of Gonito.net challenges, assuming the test data is given in
simple TSV (tab-separated values) files. simple TSV (tab-separated values) files.
@ -14,6 +14,29 @@ The official repository is `git://gonito.net/geval`, browsable at
## Installing ## Installing
### The easy way: just download the fully static GEval binary
(Assuming you have a 64-bit Linux.)
wget https://gonito.net/get/bin/geval
chmod u+x geval
./geval --help
#### On Windows
For Windows, you should use Windows PowerShell.
wget https://gonito.net/get/bin/geval
Next, you should go to the folder where you download `geval` and right-click to `geval` file.
Go to `Properties` and in the section `Security` grant full access to the folder.
Or you should use `icacls "folder path to geval" /grant USER:<username>`
This is a fully static binary, it should work on any 64-bit Linux or 64-bit Windows.
### Build from scratch
You need [Haskell Stack](https://github.com/commercialhaskell/stack). You need [Haskell Stack](https://github.com/commercialhaskell/stack).
You could install Stack with your package manager or with: You could install Stack with your package manager or with:
@ -66,6 +89,8 @@ In case the lzma package is not installed on your Linux, you need to run (assumi
sudo apt-get install pkg-config liblzma-dev libpq-dev libpcre3-dev sudo apt-get install pkg-config liblzma-dev libpq-dev libpcre3-dev
#### Windows issues
If you see this message on Windows during executing `stack test` command: If you see this message on Windows during executing `stack test` command:
In the dependencies for geval-1.21.1.0: In the dependencies for geval-1.21.1.0:
@ -81,8 +106,8 @@ And you should add `lzma-clib-5.2.2` and `unix-compat-0.5.2` to section extra-de
If you see message about missing pkg-config on Windpws you should download two packages from the site: If you see message about missing pkg-config on Windpws you should download two packages from the site:
http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/ http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/
These packages are: These packages are:
- pkg-config (newest version) - pkg-config (the newest version)
- gettext-runtime (newest version) - gettext-runtime (the newest version)
Extract `pkg-config.exe` file in Windows PATH Extract `pkg-config.exe` file in Windows PATH
Extract init.dll file from gettext-runtime Extract init.dll file from gettext-runtime
@ -91,26 +116,6 @@ and extract libglib-2.0-0.dll file.
All files you should put for example in `C:\MinGW\bin` directory. All files you should put for example in `C:\MinGW\bin` directory.
### Plan B — just download the GEval binary
(Assuming you have a 64-bit Linux.)
wget https://gonito.net/get/bin/geval
chmod u+x geval
./geval --help
For Windows you should use Windows Powershell.
wget https://gonito.net/get/bin/geval
Next you should go to the folder where you download `geval` and right-click to `geval` file.
Go to `Properties` and in the section `Security` grant full access to the folder.
Or you should use `icacls "folder path to geval" /grant USER:<username>`
This is a fully static binary, it should work on any 64-bit Linux or 64-bit Windows.
## Quick tour ## Quick tour
Let's use GEval to evaluate machine translation (MT) systems (but keep Let's use GEval to evaluate machine translation (MT) systems (but keep
@ -226,7 +231,7 @@ But why were double quotes so problematic in German-English
translation?! Well, look at the second-worst feature — `&apos;&apos;` translation?! Well, look at the second-worst feature — `&apos;&apos;`
in the _output_! Oops, it seems like a very stupid mistake with in the _output_! Oops, it seems like a very stupid mistake with
post-processing was done and no double quote was correctly generated, post-processing was done and no double quote was correctly generated,
which decreased the score a little bit for each sentence in which the which decreased the score a little for each sentence in which the
quote was expected. quote was expected.
When I fixed this simple bug, the BLUE metric increased from 0.27358 When I fixed this simple bug, the BLUE metric increased from 0.27358
@ -539,9 +544,9 @@ submitted. The suggested way to do this is as follows:
`test-A/expected.tsv` added. This branch should be accessible by `test-A/expected.tsv` added. This branch should be accessible by
Gonito platform, but should be kept “hidden” for regular users (or Gonito platform, but should be kept “hidden” for regular users (or
at least they should be kindly asked not to peek there). It is at least they should be kindly asked not to peek there). It is
recommended (though not obligatory) that this branch contain all recommended (though not obligatory) that this branch contains all
the source codes and data used to generate the train/dev/test sets. the source codes and data used to generate the train/dev/test sets.
(Use [git-annex](https://git-annex.branchable.com/) if you have really big files there.) (Use [git-annex](https://git-annex.branchable.com/) if you have huge files there.)
Branch (1) should be the parent of the branch (2), for instance, the Branch (1) should be the parent of the branch (2), for instance, the
repo (for the toy “planets” challenge) could be created as follows: repo (for the toy “planets” challenge) could be created as follows: