94b10b9b29
... en:ascii and ja:iso-2022-jp tests. I don't know much about this part of the code at this point. Yet I can clearly deduct that the length of the charLenTable is supposed to be the classFactor of the SMModel. Therefore 2 classes were missing in ISO2022JPCharLenTable, hence a buffer overflow happens when trying to reach these. I am not sure of the values I should add there. For now, let's set 0 to both, but adding also a comment so that I can review this code later on, when I will get to read and understand this piece of code in more depth. |
||
|---|---|---|
| build-mac | ||
| doc | ||
| script | ||
| src | ||
| test | ||
| .gitignore | ||
| AUTHORS | ||
| CMakeLists.txt | ||
| COPYING | ||
| INSTALL | ||
| README.md | ||
| uchardet.doap | ||
| uchardet.pc.in | ||
uchardet
uchardet is an encoding detector library, which takes a sequence of bytes in an unknown character encoding without any additional information, and attempts to determine the encoding of the text. Returned encoding names are iconv-compatible.
uchardet started as a C language binding of the original C++ implementation of the universal charset detection library by Mozilla. It can now detect more charsets, and more reliably than the original implementation.
The original code of universalchardet is available at http://lxr.mozilla.org/seamonkey/source/extensions/universalchardet/
Techniques used by universalchardet are described at http://www.mozilla.org/projects/intl/UniversalCharsetDetection.html
Supported Languages/Encodings
- International (Unicode)
- UTF-8
- UTF-16BE / UTF-16LE
- UTF-32BE / UTF-32LE / X-ISO-10646-UCS-4-34121 / X-ISO-10646-UCS-4-21431
- Arabic
- ISO-8859-6
- WINDOWS-1256
- Bulgarian
- ISO-8859-5
- WINDOWS-1251
- Chinese
- ISO-2022-CN
- BIG5
- EUC-TW
- GB18030
- HZ-GB-2312
- Croatian:
- ISO-8859-2
- ISO-8859-13
- ISO-8859-16
- Windows-1250
- IBM852
- MAC-CENTRALEUROPE
- Czech
- Windows-1250
- ISO-8859-2
- IBM852
- MAC-CENTRALEUROPE
- Danish
- ISO-8859-1
- ISO-8859-15
- WINDOWS-1252
- English
- ASCII
- Esperanto
- ISO-8859-3
- Estonian
- ISO-8859-4
- ISO-8859-13
- ISO-8859-13
- Windows-1252
- Windows-1257
- Finnish
- ISO-8859-1
- ISO-8859-4
- ISO-8859-9
- ISO-8859-13
- ISO-8859-15
- WINDOWS-1252
- French
- ISO-8859-1
- ISO-8859-15
- WINDOWS-1252
- German
- ISO-8859-1
- WINDOWS-1252
- Greek
- ISO-8859-7
- WINDOWS-1253
- Hebrew
- ISO-8859-8
- WINDOWS-1255
- Hungarian:
- ISO-8859-2
- WINDOWS-1250
- Irish Gaelic
- ISO-8859-1
- ISO-8859-9
- ISO-8859-15
- WINDOWS-1252
- Italian
- ISO-8859-1
- ISO-8859-3
- ISO-8859-9
- ISO-8859-15
- WINDOWS-1252
- Japanese
- ISO-2022-JP
- SHIFT_JIS
- EUC-JP
- Korean
- ISO-2022-KR
- EUC-KR / UHC
- Lithuanian
- ISO-8859-4
- ISO-8859-10
- ISO-8859-13
- Latvian
- ISO-8859-4
- ISO-8859-10
- ISO-8859-13
- Maltese
- ISO-8859-3
- Polish:
- ISO-8859-2
- ISO-8859-13
- ISO-8859-16
- Windows-1250
- IBM852
- MAC-CENTRALEUROPE
- Portuguese
- ISO-8859-1
- ISO-8859-9
- ISO-8859-15
- WINDOWS-1252
- Romanian:
- ISO-8859-2
- ISO-8859-16
- Windows-1250
- IBM852
- Russian
- ISO-8859-5
- KOI8-R
- WINDOWS-1251
- MAC-CYRILLIC
- IBM866
- IBM855
- Slovak
- Windows-1250
- ISO-8859-2
- IBM852
- MAC-CENTRALEUROPE
- Slovene
- ISO-8859-2
- ISO-8859-16
- Windows-1250
- IBM852
- MAC-CENTRALEUROPE
- Spanish
- ISO-8859-1
- ISO-8859-15
- WINDOWS-1252
- Swedish
- ISO-8859-1
- ISO-8859-4
- ISO-8859-9
- ISO-8859-15
- WINDOWS-1252
- Thai
- TIS-620
- ISO-8859-11
- Turkish:
- ISO-8859-3
- ISO-8859-9
- Vietnamese:
- VISCII
- Windows-1258
- Others
- WINDOWS-1252
Installation
Debian/Ubuntu/Mint
apt-get install uchardet libuchardet-dev
Mageia
urpmi libuchardet libuchardet-devel
Fedora
dnf install uchardet uchardet-devel
Mac
brew install uchardet
Build from source
Releases are available from: https://www.freedesktop.org/software/uchardet/releases/
If you prefer a development version, clone the git repository:
git clone git://anongit.freedesktop.org/uchardet/uchardet
The source can be browsed at: https://cgit.freedesktop.org/uchardet/uchardet/
cmake .
make
make install
Build with flatpak-builder
Here is a working "module" section to include in your Flatpak's json manifest:
"modules": [
{
"name": "uchardet",
"buildsystem": "cmake",
"builddir": true,
"config-opts": [ "-DCMAKE_INSTALL_LIBDIR=lib" ],
"sources": [
{
...
}
]
}
]
Usage
Command Line
uchardet Command Line Tool
Version 0.0.6
Authors: BYVoid, Jehan
Bug Report: https://bugs.freedesktop.org/enter_bug.cgi?product=uchardet
Usage:
uchardet [Options] [File]...
Options:
-v, --version Print version and build information.
-h, --help Print this help.
Library
See uchardet.h
Related Projects
- python-chardet Python port
- ruby-rchardet Ruby port
- juniversalchardet Java port of universalchardet
- jchardet Java port of chardet
- nuniversalchardet C# port of universalchardet
- nchardet C# port of chardet
- uchardet-enhanced A fork of mozilla universalchardet
- rust-uchardet Rust language binding of uchardet
- libchardet Another C/C++ API wrapping Mozilla code.
Licenses
- Mozilla Public License Version 1.1
- GNU General Public License, version 2.0 or later.
- GNU Lesser General Public License, version 2.1 or later.
See the file COPYING for the complete text of these 3 licenses.