5 Commits

Author	SHA1	Message	Date
rjawor	bd73749388	new tokenizer	2017-04-26 17:02:18 +02:00
rjawor	b3d7c993aa	tokenize only option - no word map	2016-01-01 20:45:07 +01:00
rjawor	5a57406875	finished original word positions	2015-06-27 12:40:24 +02:00
rjawor	724bf0d080	new responsibilities of tokenized sentence	2015-06-26 15:38:24 +02:00
rjawor	8432dd321f	tokenizer in progress	2015-06-25 10:12:51 +02:00