Jehan 41d309e8a2 script, src: regenerate Russian models and add UTF-8/Russian support.
This fixes the broken Russian test in Windows-1251 which once again gets
a much better score with Russian. Also this adds UTF-8 support.

Same as Bulgarian, I wonder why I had not regenerated this earlier.

The new UTF-8 test comes from the 'Сурки' page of Wikipedia in Russian.

Note that now this broke the test zh:gb18030 (the score for KOI8-R / ru
(0.766388) beats GB18030 / zh (0.700000)). I think I'll have to look a
bit closer at our GB18030 dedicated prober.
2022-12-17 21:41:11 +01:00
..
ibm855.txt Add some Russian test files. 2015-11-27 18:17:20 +01:00
ibm866.txt Add some Russian test files. 2015-11-27 18:17:20 +01:00
iso-8859-5.txt Reorganize test files in language subdirectories. 2015-11-17 21:12:39 +01:00
koi8-r.txt Adding some more test files for Russian and Chinese. 2015-11-18 19:27:38 +01:00
mac-cyrillic.txt Add some Russian test files. 2015-11-27 18:17:20 +01:00
utf-8.txt script, src: regenerate Russian models and add UTF-8/Russian support. 2022-12-17 21:41:11 +01:00
windows-1251.txt Reorganize test files in language subdirectories. 2015-11-17 21:12:39 +01:00