RandomSec/main/tests
Tom Morris 0562638ffa
Use standard text normalization - fixes #2898 (#2900)
* Use standard text normalization - fixes #2898

Fixes #2898. Fixes #409. Refs #650

Replaces homegrown ISO Latin-1 only character subsitition
with standard Java Normalize to NFD, followed by diacritic
removal and a few custom character expansions/replacements.

* Fix Mac build

* Improve compatibility with previous code

One intentional change is folding O with stroke to
oe instead of o.

- Use more powerful NFKD instead of NFD
- strip punctuation after decomposition since it can generate
  new punctuation
- Add compatibility test for old asciify() method
- Add some graphically similar characters to substitution table

* Add oe character/ligature & more long S forms

* More tests for ligatures and Latin Extended

* Add Latin-1 Supplement tests
2020-07-07 21:35:41 +02:00
..
client/src Correcting typo that was breaking one UI test 2013-12-17 01:40:45 -03:00
data Add Excel95 import test and improve other importer tests (#2844) 2020-06-30 08:20:56 +02:00
server Use standard text normalization - fixes #2898 (#2900) 2020-07-07 21:35:41 +02:00