coffee/uchardet
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/uchardet/uchardet.git synced 2025-12-12 06:30:05 +08:00
Code Issues Packages Projects Releases Wiki Activity
uchardet/src
History
Jehan e5234d6b61 Stating endianness of UTF-16 and UTF-32 was an error when BOM present. 
According to RFC 2781, section 3.3: "Systems labelling UTF-16BE/LE text
MUST NOT prepend a BOM to the text."
Since uchardet cannot (and should not, obviously, it's not its role)
modify input text, when a BOM is present, we should always label the
encoding as "UTF-16" only.
Also it broke unit tests in using programs since a conversion from UTF-8
to UTF-16LE/BE would create a text without BOM, and a conversion from
UTF-16LE/BE to UTF-8 creates a UTF-8 text with a BOM, which changed
existing behaviours.
Same goes for UTF-32.
See also Unicode 5.0.0 standard, section 3.10 (tables 3.8 and 3.9 in
particular).
2015-12-04 19:19:39 +01:00
..
LangModels LangModels: rename Cyrillic models to Russian models. 2015-12-04 03:27:29 +01:00
tools Update uchardet URL in various places. 2015-12-03 19:48:29 +01:00
Big5Freq.tab Initial release. 2011-07-10 15:04:42 +08:00
CharDistribution.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
CharDistribution.h uchardet_get_charset() must return iconv-compatible names. 2015-11-17 16:15:21 +01:00
CMakeLists.txt LangModels: rename Cyrillic models to Russian models. 2015-12-04 03:27:29 +01:00
EUCKRFreq.tab Initial release. 2011-07-10 15:04:42 +08:00
EUCTWFreq.tab Initial release. 2011-07-10 15:04:42 +08:00
GB2312Freq.tab Initial release. 2011-07-10 15:04:42 +08:00
JISFreq.tab Initial release. 2011-07-10 15:04:42 +08:00
JpCntx.cpp Fixes boolean operation precedence warnings... 2015-11-18 19:38:12 +01:00
JpCntx.h Update code from upstream. 2011-07-11 14:42:50 +08:00
nsBig5Prober.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsBig5Prober.h uchardet_get_charset() must return iconv-compatible names. 2015-11-17 16:15:21 +01:00
nsCharSetProber.cpp Initial release. 2011-07-10 15:04:42 +08:00
nsCharSetProber.h Update code from upstream. 2011-07-11 14:42:50 +08:00
nsCodingStateMachine.h Update code from upstream. 2011-07-11 14:42:50 +08:00
nscore.h Update code from upstream. 2011-07-11 14:42:50 +08:00
nsEscCharsetProber.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsEscCharsetProber.h Update code from upstream. 2011-07-11 14:42:50 +08:00
nsEscSM.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsEUCJPProber.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsEUCJPProber.h Update code from upstream. 2011-07-11 14:42:50 +08:00
nsEUCKRProber.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsEUCKRProber.h Update code from upstream. 2011-07-11 14:42:50 +08:00
nsEUCTWProber.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsEUCTWProber.h uchardet_get_charset() must return iconv-compatible names. 2015-11-17 16:15:21 +01:00
nsGB2312Prober.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsGB2312Prober.h uchardet_get_charset() must return iconv-compatible names. 2015-11-17 16:15:21 +01:00
nsHebrewProber.cpp s/windows-1255/WINDOWS-1255/ to follow iconv uppercase naming. 2015-11-18 03:21:34 +01:00
nsHebrewProber.h Initial release. 2011-07-10 15:04:42 +08:00
nsLatin1Prober.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsLatin1Prober.h uchardet_get_charset() must return iconv-compatible names. 2015-11-17 16:15:21 +01:00
nsMBCSGroupProber.cpp uchardet_get_charset() must return iconv-compatible names. 2015-11-17 16:15:21 +01:00
nsMBCSGroupProber.h Update code from upstream. 2011-07-11 14:42:50 +08:00
nsMBCSSM.cpp uchardet_get_charset() must return iconv-compatible names. 2015-11-17 16:15:21 +01:00
nsPkgInt.h Update code from upstream. 2011-07-11 14:42:50 +08:00
nsSBCharSetProber.cpp Single Byte charsets: high ctrl character ratio lowers confidence. 2015-12-04 00:04:43 +01:00
nsSBCharSetProber.h LangModels: rename Cyrillic models to Russian models. 2015-12-04 03:27:29 +01:00
nsSBCSGroupProber.cpp LangModels: rename Cyrillic models to Russian models. 2015-12-04 03:27:29 +01:00
nsSBCSGroupProber.h LangModels: add ISO-8859-11 and regenerate TIS-620 Thai models. 2015-12-04 03:14:52 +01:00
nsSJISProber.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsSJISProber.h uchardet_get_charset() must return iconv-compatible names. 2015-11-17 16:15:21 +01:00
nsUniversalDetector.cpp Stating endianness of UTF-16 and UTF-32 was an error when BOM present. 2015-12-04 19:19:39 +01:00
nsUniversalDetector.h Fix minor space issues. 2015-11-24 00:15:44 +01:00
nsUTF8Prober.cpp Update code from upstream. 2011-07-11 14:42:50 +08:00
nsUTF8Prober.h Update code from upstream. 2011-07-11 14:42:50 +08:00
prmem.h Initial release. 2011-07-10 15:04:42 +08:00
symbols.cmake fix: export symbols were not passed to the linker as intended 2015-06-20 12:28:01 +09:00
uchardet.cpp uchardet_get_charset() should never return NULL... 2015-11-16 17:33:16 +01:00
uchardet.h Update doc comment. 2015-12-03 20:36:09 +01:00