coffee/uchardet
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/uchardet/uchardet.git synced 2026-10-28 18:07:22 +08:00
Code Issues Packages Projects Releases Wiki Activity

src: the EUC-KR prober now returns "UHC" as encoding name.

Browse Source 
"UHC" is the "Unified Hangul Code" (aka Windows-949 or CP949). It is
apparently "mostly" upward compatible with EUC-KR so returning UHC for
a strict EUC-KR document is usually not to be considered wrong.
Yet I can read that EUC-KR has its own way of representing hangul
syllables not available in precomposed form, and this is not supported
in UHC (since this latter has all possible precomposed syllables), hence
the "mostly" upward-compatibility.
My personal daily experience with Korean documents though is that I
encounter a lot of UHC-encoded files, probably because of predominance
of Microsoft operating systems, which spread this encoding.
So until we get 2 separate detection machines, let's just return EUC-KR
files as being "UHC".
This commit is contained in:
Jehan 2016-09-19 01:22:45 +02:00
parent f14519a0fe
commit 157de1dc65
1 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
src/nsEUCKRProber.h
View File

@ -51,7 +51,12 @@ public:
} }
virtual ~nsEUCKRProber(void){delete mCodingSM;} virtual ~nsEUCKRProber(void){delete mCodingSM;}
nsProbingState HandleData(const char* aBuf, PRUint32 aLen); nsProbingState HandleData(const char* aBuf, PRUint32 aLen);
const char* GetCharSetName() {return "EUC-KR";} /* "Unified Hangul Code", also called "CP949" or "Windows-949" is a
* superset of EUC-KR. Though not fully ok to return UHC here (a
* separate prober would be better), it is acceptable, since many
* Korean documents are actually created with this character set.
*/
const char* GetCharSetName() {return "UHC";}
nsProbingState GetState(void) {return mState;} nsProbingState GetState(void) {return mState;}
void Reset(void); void Reset(void);
float GetConfidence(void); float GetConfidence(void);