gonito/templates/presentation-4real.hamlet

189 lines
6.1 KiB
Plaintext

<script src="/static/js/sigma.min.js">
<script src="/static/js/sigma.parsers.json.min.js">
<div id="title" class="step" data-x="0" data-y="-1500">
<h1>Gonito.net
<p>open platform for research competition, cooperation and reproducibility
<p class="footnote">Filip Graliński, Rafał Jaworski,<br/>Łukasz Borchmann, Piotr Wierzchoń
<p class="footnote">LREC 2016 / 4REAL Workshop
<div class="step slide" data-x="0" data-y="-500">
<h2>Motivation
<ul>
<li>We needed a place to keep track of our results for:
<ul>
<li>carrying out our research
<li>teaching our students
<li>just playing with our data sets
<li>(but didn't want to save the world or earn $$$)
<div class="step slide" data-x="0" data-y="500">
<h2>Alternatives?
<ul>
<li><a href="https://www.kaggle.com">Kaggle</a>
<ul>
<li>you need to <em>ask</em> Kaggle people
<li>Kaggle in class only for teaching
<li>not fully open ☹
<li><a href="https://www.drivendata.org">DrivenData</a>
<ul>
<li>more or less similar to Kaggle</li>
<li><a href="https://codalab.org">CodaLab</a>
<ul>
<li>trying hard but hadn't found it ☹
<li>open!
<li>no git
<div class="step slide" data-x="1000" data-y="500">
<p>That's why we created
<h1>Gonito.net
<p>open platform for machine learning competition
<ul>
<li>Be open
<ul>
<li>GNU Affero General Public License
<li>Anyone can set up their own instance
<li>Users are encouraged to share source codes
<li>Users are free to use whatever programming language and tools
<li>Use git
<ul>
<li>Challenges are created as Git repositories
<li>Solutions uploaded with Git
<li>Solutions can be forked and reused
<li>Even if a Gonito.net ceases to exist, Git repos may live
<div class="step slide" data-x="2000" data-y="500">
<p>… and
<h1>GEval
<p>companion Haskell library and stand-alone tool for machine learning evaluation
<ul>
<li>accuracy
<li>(R)MSE
<li>BLEU
<li>… more to come
<div class="step slide" style="height: 800px;" data-x="1000" data-y="1500">
<h2>A challenge…
<div style="font-size: 50%" class="readme">
^{readme}
<div class="step slide" data-x="2000" data-y="2000">
<h2>Install GEval
<pre>
git clone git://gonito.net/geval
cd geval
stack setup
stack install
# add ~/.local/bin to $PATH
<div class="step slide" style="height: 800px;" data-x="1000" data-y="2500">
<h2>Current status…
<div style="font-size: 50%">
^{sampleLeaderboard}
<div class="step slide" data-x="2000" data-y="3000">
<h2>How to replicate the best result?
<ul>
<li>Let's assume you know Git commit hash
<ul>
<li>(e.g. you found it in the paper)
<li>see the submission number at http://gonito.net/q/db43f2
<pre>
git clone git://gonito.net/petite-difference-challenge -b submission-00115
cd petite-difference-challenge
make clean
make
geval --test-name dev-0
<div class="step slide" data-x="2000" data-y="4000">
<h2>I wanna do something better
<p>Sign up, add SSH key etc.
<p>Clone the repo:
<pre>
git clone ssh://gitolite@gonito.net/john/petite-difference-challenge
cd petite-difference-challenge
git pull ssh://gitolite@gonito.net/petite-difference-challenge
git push origin master
cd petite-difference-challenge
<p>Work on your solution and check locally:
<pre>
geval --test-name dev-0
<p>Push your brilliant solution:
<pre>
git add dev-0/out.tsv test-A/out.tsv # ...
git commit -m 'my brilliant solution'
git push origin master
<p>… and let Gonito.net know about your submission (or configure a Git hook)
<div class="step slide" style="height: 800px;" data-x="1000" data-y="4500">
<h2>… and see your results
<div style="font-size: 50%">
^{Table.buildBootstrap (submissionsTable Nothing (challengeName challenge) scheme challengeRepo tests) evaluationMaps}
<div class="step slide" data-x="2000" data-y="5000">
<h2>Be open!</h2>
<p>You can open <span class="glyphicon glyphicon-share"></span>&nbsp; your submission so that everybody could see it
<p>… and you fork somebody else's submission <span class="glyphicon glyphicon-folder-open"></span>&nbsp; and build upon it
<pre>
git clone ssh://gitolite@gonito.net/john/petite-difference-challenge
cd petite-difference-challenge
git fetch ssh://gitolite@gonito.net/petite-difference-challenge submission-00115
git reset --hard FETCH_HEAD
<p>… or you can browse the code at <a href="http://gonito.net/gitlist/petite-difference-challenge.git/submission-00115">http://gonito.net/gitlist/petite-difference-challenge.git/submission-00115</a>
<div class="step slide" data-x="1000" data-y="5500">
<h2>Who forked and submitted what and when?
<p><a href="http://gonito.net/challenge/petite-difference-challenge">Click here to see the graph</a>
<!-- <div id="graph-container"> -->
<div class="step slide" style="height: 500px;" data-x="1000" data-y="6500">
<h2>Yet another challenge
<div style="font-size: 50%">
^{sampleLeaderboard'}
<div class="step slide" data-x="0" data-y="7000">
<h2>Future?
<ul>
<li>works for us!
<ul>
<li>as an auxiliary teaching tool
<li>for keeping track of our research
<li>when writing a paper in a team
<li>why not for other people?
<ul>
<li>for organizing shared tasks
<li>for tracking effort of a research community in a longer-term perspective
<li>TODOs
<ul>
<li>Git annex for storing models
<li>?
<div class="step slide" data-x="0" data-y="8000">
<h1>Thanks!
<p>And check it out at <a href="http://gonito.net">gonito.net</a>:
<ul>
<li><a href="http://gonito.net/petite-difference-challenge">gender classification challenge</a>
<li><a href="http://gonito.net/retroc">temporal classification challenge</a>