mirror of
https://gitlab.freedesktop.org/uchardet/uchardet.git
synced 2025-12-14 23:50:05 +08:00
Previous technical text about charsets themselves were not relevant to identify a language. In particular the special characters different between ISO-8859-1 and ISO-8859-15 were used by themselves, out of a char sequence context. Therefore without language understanding, they could have as well been representing the ISO-8859-15 letters or the ISO-8859-1 symbols at the corresponding codepoints. Replacing with text from this Wikipedia page: https://fr.wikipedia.org/wiki/Œuf_(cuisine) This uses some of these same characters (in particular 'œ') but in contextual character sequences, making it relevant for our algorithm. |
||
|---|---|---|
| .. | ||
| iso-8859-1.txt | ||
| iso-8859-15.txt | ||
| utf-16be.txt | ||
| utf-16le.txt | ||
| utf-32be.txt | ||
| utf-32le.txt | ||