RandomSec/main/tests/data/big5.txt

<EFBFBD><EFBFBD><EFBFBD>޵<EFBFBD><EFBFBD>L<EFBFBD><EFBFBD><EFBFBD>G<EFBFBD><EFBFBD><EFBFBD>y<EFBFBD>̤j<EFBFBD><EFBFBD><EFBFBD>R<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>A<EFBFBD>i<EFBFBD><EFBFBD><EFBFBD>γп<EFBFBD>
<EFBFBD><EFBFBD><EFBFBD>޵<EFBFBD><EFBFBD>L<EFBFBD><EFBFBD><EFBFBD>O<EFBFBD><EFBFBD><EFBFBD>y<EFBFBD>̤j<EFBFBD><EFBFBD><EFBFBD>R<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>A<EFBFBD>ѱi<EFBFBD><EFBFBD><EFBFBD>γп<EFBFBD><EFBFBD>A<EFBFBD><EFBFBD><EFBFBD>ѧK<EFBFBD>O<EFBFBD><EFBFBD><EFBFBD>R<EFBFBD>B<EFBFBD><EFBFBD><EFBFBD>L<EFBFBD>B<EFBFBD>P<EFBFBD>y<EFBFBD>B<EFBFBD>e<EFBFBD>R<EFBFBD>B<EFBFBD>m<EFBFBD>W<EFBFBD>B<EFBFBD>R<EFBFBD>W<EFBFBD>B<EFBFBD>e<EFBFBD>@<40>B<EFBFBD><42>ù<EFBFBD>B<EFBFBD>B<EFBFBD>աB<D5A1>I<EFBFBD>O<EFBFBD>B<EFBFBD>R<EFBFBD><52><EFBFBD>B<EFBFBD>ýt<C3BD>B<EFBFBD>R<EFBFBD>L<EFBFBD>B<EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD><EFBFBD>B<EFBFBD>߲z<DFB2><7A><EFBFBD><EFBFBD><EFBFBD>B<EFBFBD>ͨv<CDA8>ε<EFBFBD><CEB5>L<EFBFBD>ҵ{<7B>о<EFBFBD>
<EFBFBD><EFBFBD><EFBFBD>L,<2C><><EFBFBD>L<EFBFBD><4C><EFBFBD><EFBFBD>,<2C>K<EFBFBD>O<EFBFBD><4F><EFBFBD>L,<2C><><EFBFBD>L<EFBFBD>R<EFBFBD>L,<2C><><EFBFBD>L<EFBFBD>ҵ{,<2C>e<EFBFBD>@,<2C>e<EFBFBD>R
-												Restore character encoding guesser (#2755)

* Fixes #486. Builds on code from Steffen Stundzig

- Switch from ICU4J to juniversalchardet
  (Java port of Mozilla charset detector)
- Replace org.json code with Jackson
- Add tests
- Add TODO for multi-file character encoding mismatches

* Restore dependency lost in rebase

Co-authored-by: Steffen Stundzig <git@stundzig.de>
											
										
										
											2020-06-22 06:04:51 +02:00
+								<EFBFBD><EFBFBD><EFBFBD>޵<EFBFBD><EFBFBD>L<EFBFBD><EFBFBD><EFBFBD>G<EFBFBD><EFBFBD><EFBFBD>y<EFBFBD>̤j<EFBFBD><EFBFBD><EFBFBD>R<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>A<EFBFBD>i<EFBFBD><EFBFBD><EFBFBD>γп<EFBFBD>
 								<EFBFBD><EFBFBD><EFBFBD>޵<EFBFBD><EFBFBD>L<EFBFBD><EFBFBD><EFBFBD>O<EFBFBD><EFBFBD><EFBFBD>y<EFBFBD>̤j<EFBFBD><EFBFBD><EFBFBD>R<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>A<EFBFBD>ѱi<EFBFBD><EFBFBD><EFBFBD>γп<EFBFBD><EFBFBD>A<EFBFBD><EFBFBD><EFBFBD>ѧK<EFBFBD>O<EFBFBD><EFBFBD><EFBFBD>R<EFBFBD>B<EFBFBD><EFBFBD><EFBFBD>L<EFBFBD>B<EFBFBD>P<EFBFBD>y<EFBFBD>B<EFBFBD>e<EFBFBD>R<EFBFBD>B<EFBFBD>m<EFBFBD>W<EFBFBD>B<EFBFBD>R<EFBFBD>W<EFBFBD>B<EFBFBD>e<EFBFBD>@<40>B<EFBFBD><42>ù<EFBFBD>B<EFBFBD>B<EFBFBD>աB<D5A1>I<EFBFBD>O<EFBFBD>B<EFBFBD>R<EFBFBD><52><EFBFBD>B<EFBFBD>ýt<C3BD>B<EFBFBD>R<EFBFBD>L<EFBFBD>B<EFBFBD>A<EFBFBD><41><EFBFBD><EFBFBD><EFBFBD>B<EFBFBD>߲z<DFB2><7A><EFBFBD><EFBFBD><EFBFBD>B<EFBFBD>ͨv<CDA8>ε<EFBFBD><CEB5>L<EFBFBD>ҵ{<7B>о<EFBFBD>
 								<EFBFBD><EFBFBD><EFBFBD>L,<2C><><EFBFBD>L<EFBFBD><4C><EFBFBD><EFBFBD>,<2C>K<EFBFBD>O<EFBFBD><4F><EFBFBD>L,<2C><><EFBFBD>L<EFBFBD>R<EFBFBD>L,<2C><><EFBFBD>L<EFBFBD>ҵ{,<2C>e<EFBFBD>@,<2C>e<EFBFBD>R