Commit Graph

69 Commits

Author SHA1 Message Date
b3d7c993aa tokenize only option - no word map 2016-01-01 20:45:07 +01:00
bbf3853d2a added lowercasing when tokenizing by space 2015-12-29 21:44:46 +01:00
0a8d2fdd39 tokenize by whitespace option 2015-12-27 20:54:40 +01:00
873d7c300c added parameterless constructor for concordia 2015-10-19 15:38:10 +02:00
1adabf4833 add index path as required argument to concordia constructor 2015-10-16 22:14:11 +02:00
fa3138df29 count occurences feature 2015-10-01 13:36:54 +02:00
bd62420cd5 updated tutorial 2015-08-24 14:30:20 +02:00
0a3fd8a04e added an extremely important improvement to the concordia search algorithm - gapped overlays cut-off 2015-08-24 13:10:06 +02:00
209e374226 repaired concordia test 2015-08-19 20:53:40 +02:00
68fecaddf8 adding all tokenized examples 2015-08-19 20:49:26 +02:00
a765443a01 simple search returns matched pattern fragments 2015-08-07 12:54:57 +02:00
28704c2f43 separated tokenization and adding to index 2015-08-01 17:03:39 +02:00
5a57406875 finished original word positions 2015-06-27 12:40:24 +02:00
a8c5fa0c75 original word positions 2015-06-27 10:09:49 +02:00
dba70b4e24 done word positions 2015-06-26 22:50:53 +02:00
724bf0d080 new responsibilities of tokenized sentence 2015-06-26 15:38:24 +02:00
9b1735516c working sentence tokenizer 2015-06-25 20:49:22 +02:00
8432dd321f tokenizer in progress 2015-06-25 10:12:51 +02:00
0baf3e4ef2 character intervals in progress 2015-06-22 13:52:56 +02:00
07d5d4438b clear index, examples 2015-05-04 20:40:44 +02:00
abbd5b1ae8 finished documentation 2015-05-01 14:52:53 +02:00
9e550ca1cf more doc 2015-04-30 22:22:54 +02:00
87a26bfa3b cleaned configuration, doc 2015-04-30 21:15:18 +02:00
b790c6898f stable release 2015-04-30 09:29:10 +02:00
db63cf776e doc 2015-04-28 21:34:07 +02:00
bb7608d05e anubis searcher -> concordia searcher
Former-commit-id: 8afe194adf3163ee62caa30732d9c9dd095df66b
2015-04-24 11:48:32 +02:00
04df67c6f0 100% test in concordia-console. All passed!
Former-commit-id: 6e6186a148d637ba5a0d324d6d68c78708f0942d
2015-04-22 16:50:12 +02:00
f64449311d removed stop words - works slower
Former-commit-id: 97ce33b0a6ea3c89aaa5a4c69cad248c7b2c8203
2015-04-21 21:33:08 +02:00
5c2ae86097 output concordia score
Former-commit-id: fa7db09fe9319fa844d294ca4e7deb22d1328151
2015-04-21 20:44:49 +02:00
7549703414 best overlay computation
Former-commit-id: 986f3d6b611fd276a7b26073daa0094caf078d1e
2015-04-21 15:14:48 +02:00
024fbf72aa concordia search
Former-commit-id: 609c3a54e930ebae45a2e9a07f63991ec4abc9a6
2015-04-17 14:17:59 +02:00
4e02afc897 anubis search v1 - very slow for some patterns
Former-commit-id: ae327d7d24f4bc959d3749745a8c395093a17a50
2015-04-16 11:39:39 +02:00
0d4bdf12de removed using namespace std
Former-commit-id: dbb5129e1f94d83eca887ada0f89d6bb45250f1e
2015-04-15 14:14:10 +02:00
a09999c130 repaired tm matches
Former-commit-id: ee2e73ab1e37db051b8be36b97bc503241c798c0
2015-04-15 11:50:59 +02:00
3a03b01f42 std vectors
Former-commit-id: 5816e87c856f7edc242cc707851a0e2ad05aeb38
2015-04-15 10:55:26 +02:00
e02bbaa0fa getTmMatches
Former-commit-id: 94aa3db2db88195c61c6ac70006c0e1d743dc854
2015-04-14 20:14:30 +02:00
f03b4ad954 fixed lcp search
Former-commit-id: 18192126d134323569bc43205ccc60788d9e6cb6
2015-04-12 12:06:41 +02:00
2533fd5b44 extended markers - length, bitwise operators
Former-commit-id: 948a7fc68bf0b2284ce631d877fc13fa3eaa4882
2015-04-09 22:17:19 +02:00
f83aaef4ed trimming anonymized sentence
Former-commit-id: 316b76717e4075e466828c628e064076d39481c5
2014-08-15 13:22:04 +02:00
2a3c7eddfe anubis search suspended
Former-commit-id: 1e73eb108ea6961bb8306aed3c0bbaf9127f3c69
2014-06-24 18:33:02 +02:00
8f953883bf anubis search continued
Former-commit-id: 95a08f242a03311d067303bfff07bf4890796da5
2014-06-24 18:23:46 +02:00
e8ea5881a5 lcp search
Former-commit-id: 925a5de8bc33256b594c369907f202e29f809f47
2014-05-15 22:20:31 +02:00
dd8b27cc23 anubis searcher stub
Former-commit-id: fe520e255c918c889bdd421a363668b2ed76c675
2014-05-14 16:29:44 +02:00
e99eb77b28 anonymizing sentences
Former-commit-id: 5d8bd7e16258fda7c02a7cc0e1da589d73418f0d
2014-04-29 14:46:04 +02:00
Rafał Jaworski
5eaf981bc0 working text utils
Former-commit-id: fa44e4578a007291948e4709a0cfd4278fd3af66
2014-04-24 14:26:35 +02:00
Rafał Jaworski
93c3f50b14 utf8case included
Former-commit-id: a330ce0a63a7f0b452eb95273321f165894849f4
2014-04-24 12:04:37 +02:00
9358863f8d text utils stub
Former-commit-id: d4459220f5696839d98848e9c30a61c084763a91
2014-04-24 08:36:48 +02:00
13c97f572d sentence anonymizer stub, regex replacement
Former-commit-id: edb1247f7b29fd62913114be84d3391507a0890e
2014-04-13 12:21:30 +02:00
8a38831306 logarithmic score
Former-commit-id: ec2704b3a206cc39ed42d19620bef6ce0fedbc7e
2014-03-14 12:05:06 +01:00
4b921decae limits control
Former-commit-id: 83d90cb63b3f1447938d16010e66f4345dfe0617
2014-03-14 11:30:17 +01:00