coffee/uchardet
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/uchardet/uchardet.git synced 2025-12-06 08:46:40 +08:00
Code Issues Packages Projects Releases Wiki Activity
102 Commits 5 Branches 7 Tags 1.7 MiB
C++ 71.8%
Python 24.6%
C 2.4%
CMake 1%
Berry 0.2%
Go to file
Jehan c4fa728e7a Merge branch 'master' of https://github.com/lovasoa/uchardet into lovasoa-master 
Let's shortcut Single Byte charset detection on invalid codepoints.
Merging and fixing the contributor's commit conflicts after code
redesign: in particular we added an illegal character concept (they were
mixed with control characters in current charmaps. Yet ctrl characters
are NOT to be considered invalid) and constants instead of hardcoded
numbers ('ILL' rather than 255).
2015-12-03 19:26:19 +01:00
build-mac Added shared scheme 2014-10-26 00:24:03 -07:00
doc Update README and manual... 2015-11-27 18:27:11 +01:00
script LangModels: update the French language models. 2015-11-30 19:20:55 +01:00
src Merge branch 'master' of https://github.com/lovasoa/uchardet into lovasoa-master 2015-12-03 19:26:19 +01:00
test Re-enable Hungarian language models. 2015-12-02 22:24:36 +01:00
.gitignore Add a .gitignore. 2015-11-29 02:27:42 +01:00
AUTHORS Update AUTHORS. 2015-11-17 21:51:59 +01:00
CMakeLists.txt Release: version 0.0.3. 2015-11-19 15:18:11 +01:00
COPYING Add authors. 2011-07-13 20:16:23 +08:00
INSTALL Add description of installation. 2011-07-11 23:14:57 +08:00
README.md Re-enable Hungarian language models. 2015-12-02 22:24:36 +01:00
uchardet.pc.in Refine Description in pkgconfig file 2015-09-21 09:37:36 +08:00

README.md

uchardet

uchardet is a C language binding of the original C++ implementation of the universal charset detection library by Mozilla.

uchardet is an encoding detector library, which takes a sequence of bytes in an unknown character encoding without any additional information, and attempts to determine the encoding of the text. Returned encoding names are iconv-compatible.

The original code of universalchardet is available at http://lxr.mozilla.org/seamonkey/source/extensions/universalchardet/

Techniques used by universalchardet are described at http://www.mozilla.org/projects/intl/UniversalCharsetDetection.html

Supported Encodings

  • International (Unicode)
    • UTF-8
    • UTF-16BE / UTF-16LE
    • UTF-32BE / UTF-32LE / X-ISO-10646-UCS-4-34121 / X-ISO-10646-UCS-4-21431
  • Chinese
    • ISO-2022-CN
    • BIG5
    • EUC-TW
    • GB18030
    • HZ-GB-2312
  • Japanese
    • ISO-2022-JP
    • SHIFT_JIS
    • EUC-JP
  • Korean
    • ISO-2022-KR
    • EUC-KR
  • Cyrillic
    • ISO-8859-5
    • KOI8-R
    • WINDOWS-1251
    • MAC-CYRILLIC
    • IBM866
    • IBM855
  • Greek
    • ISO-8859-7
    • WINDOWS-1253
  • Hebrew
    • ISO-8859-8
    • WINDOWS-1255
  • Thai
    • TIS-620
  • French
    • ISO-8859-1
    • ISO-8859-15
  • English
    • ASCII
  • Hungarian:
    • ISO-8859-2
    • WINDOWS-1250
  • Others
    • WINDOWS-1252

Installation

Debian/Ubuntu/Mint

apt-get install uchardet libuchardet-dev

Mageia

urpmi libuchardet libuchardet-devel

Mac

brew install uchardet

Build from source

cmake .
make
make install

Usage

Command Line

uchardet Command Line Tool
Version 0.0.3

Author: BYVoid
Bug Report: http://code.google.com/p/uchardet/issues/entry

Usage:
 uchardet [Options] [File]...

Options:
 -v, --version         Print version and build information.
 -h, --help            Print this help.

Library

See uchardet.h

Related Projects

License

Mozilla Public License Version 1.1