coffee/uchardet
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/uchardet/uchardet.git synced 2025-12-06 08:46:40 +08:00
Code Issues Packages Projects Releases Wiki Activity
188 Commits 5 Branches 7 Tags 1.7 MiB
Commit Graph

188 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jehan
771d78b7df Update the URL links: uchardet is now a freedesktop project. 2016-07-20 01:47:50 +02:00
Jehan
20eb319359 README: make the licenses as a list. 
This was breaking as markdown by not creating linefeeds.
 2016-07-20 00:21:07 +02:00
Jehan
602c1ab0fc README, COPYING: adding links and text of licenses GPL 2.0 and LGPL 2.1. 
Thanks to Ilya Tumaykin for reporting the missing info.
 2016-06-04 14:21:38 +02:00
Jehan
210e52d99a LangModels: update the Greek language models. 
I did this to improve the model after a user reported a Greek sutitle
badly detected (see commit e0eec3b).
It didn't help, but well... since I updated it with much more data from
Wikipedia. Let's just commit it!
 2016-05-25 17:39:10 +02:00
Jehan
e0eec3bae8 src: give a little weight to "probable sequences". 
Up to now, we were only considering positive sequences, which are
sequences of 2 characters which happen the most. Yet our data gather
4 categories of sequences (the last one being called "negative", since
they never happened in our data).
I will call the category below positive: probable sequences. They may
happen, yet not often. The last category could be called "neutral".
This seems to fix the detection of a user's subtitle example without
breaking any of our current unit tests.
Probably I should still review this whole logics more in details later.
 2016-05-25 17:38:20 +02:00
Jehan
4287d3accc src: trailing whitespace removed. 2016-05-25 16:07:17 +02:00
Jehan
6cd8c322ad script: stupid bug on BuildLangModel.py. 2016-05-25 15:23:36 +02:00
Jehan
fb1d544007 pkg-config: use GNUInstallDirs CMAKE_ variables in pc.in template. 2016-03-27 20:31:58 +02:00
Jehan
74b4f6a62b Merge pull request #30 from Coacher/use-gnuinstalldirs-cmake-module 
Use GNUInstallDirs cmake module, fix library filename bug, minor cleanups.
 2016-03-27 20:31:17 +02:00
Ilya Tumaykin
2a3e41a6c3
cmake: drop useless PACKAGE_NAME redefinition 2016-03-22 01:23:06 +03:00
Ilya Tumaykin
6db8b6f8fe
cmake: minor comment cleanups 2016-03-22 01:23:06 +03:00
Ilya Tumaykin
d0e7ddd8ab
cmake: fix library filename and SONAME 
Make library filename respect the current uchardet version and
make library SONAME respect the current major version.
 2016-03-22 01:23:05 +03:00
Ilya Tumaykin
dbeee08335
cmake: use lowercase suffix for debug build 2016-03-22 01:23:05 +03:00
Ilya Tumaykin
ad647d2e0a
cmake: keep compiler definitions in one place 2016-03-22 01:23:05 +03:00
Ilya Tumaykin
29f18210b1
cmake: hardcode less 2016-03-22 01:23:04 +03:00
Ilya Tumaykin
7201835c98
cmake: export UCHARDET_LIBRARY to the topmost scope 2016-03-22 01:23:04 +03:00
Ilya Tumaykin
e7feb35627
cmake: rename UCHARDET_STATIC_{TARGET -> LIBRARY} for clarity 2016-03-22 01:23:04 +03:00
Ilya Tumaykin
1a1f4bfbd8
cmake: rename UCHARDET_{TARGET -> LIBRARY} for clarity 2016-03-22 01:23:03 +03:00
Ilya Tumaykin
31a53570d6
cmake: use GNUInstallDirs cmake module 
Available in cmake >= 2.8.5.
 2016-03-22 01:23:03 +03:00
Ilya Tumaykin
d0e29dc934
cmake: bump the minimum version to 2.8.5 
Required for the GNUInstallDirs cmake module. See the next commit.
 2016-03-22 01:21:58 +03:00
Jehan
ad7db2769e Merge pull request #26 from Coacher/uniform-indent 
cmake: uniform indent everywhere.
 2016-03-21 00:22:19 +01:00
Ilya Tumaykin
b44be77be6
cmake: uniform indent everywhere 
Indent with tabs, remove leading/trailing blank lines and spaces.
 2016-03-21 01:07:41 +03:00
Jehan
b88a66f3f1 Merge pull request #28 from Coacher/cmake-updates 
cmake: use PACKAGE_NAME variable instead of hardcoding it.
 2016-03-19 14:24:52 +01:00
Carbo Kuo
e28dfe3776 Merge pull request #29 from wiiaboo/ab-suite 
CMake: Fix regression in f53cb8c building in paths with spaces
 2016-03-18 16:31:31 +01:00
Ricardo Constantino (:RiCON)
78b55ec9fe CMake: Fix regression in f53cb8c building in paths with spaces 
Tested with Ninja and Make in Windows and Archlinux with paths
with and without spaces.
 2016-03-18 03:37:12 +00:00
Ilya Tumaykin
6c1e310f9b
cmake: hardcode less 2016-03-18 02:56:21 +03:00
Jehan
fcc525a64f Merge pull request #25 from Coacher/master 
cmake: purge remnants of opencc after b6d872bb
 2016-03-17 19:10:39 +01:00
Jehan
d255184609 Merge pull request #24 from wiiaboo/ab-suite 
Improving build with more options.

Building only static possible, uchardet command line tool build can be disabled, bindir can be customized…
 2016-03-17 19:09:30 +01:00
Ricardo Constantino (:RiCON)
86755b1f57 CMake: Don't build static more than once 2016-03-16 19:31:00 +00:00
Ricardo Constantino (:RiCON)
b908b689a0 CMake: Add static lib destination to UCHARDET_TARGET 2016-03-16 19:30:54 +00:00
Ricardo Constantino (:RiCON)
81ed86a26b CMake: Use only CMAKE_INSTALL_BINDIR instead of DIR_BIN 
This way it always shows up in ccmake, even if not defined.

A string is used instead of path because I personally think it makes more
sense in the following use-cases:

STRING:
-DCMAKE_INSTALL_PREFIX=/home/user -DCMAKE_INSTALL_BINDIR=bins
installs everything to /home/user/{lib,etc,share,(...)} and executables to
${CMAKE_INSTALL_PREFIX}/bins

-DCMAKE_INSTALL_PREFIX=/home/user -DCMAKE_INSTALL_BINDIR=/opt/bin
everything to /home/user/{lib,etc,share,(...)} and executables to
/opt/bin

PATH:
-DCMAKE_INSTALL_PREFIX=/home/user -DCMAKE_INSTALL_BINDIR=bins
everything to /home/user/{lib,etc,share,(...)} and executables to
$(pwd)/bins (!)
-DCMAKE_INSTALL_PREFIX=/home/user -DCMAKE_INSTALL_BINDIR=/opt/bin
same as STRING
 2016-03-16 19:11:33 +00:00
Ilya Tumaykin
aa4c2aeada
cmake: purge remnants of opencc after b6d872bb 2016-03-16 19:43:58 +03:00
Ricardo Constantino (:RiCON)
50b2e0802f CMake: Allow not building executable 2016-03-16 14:34:03 +00:00
Ricardo Constantino (:RiCON)
6500f09931 CMake: Allow building static-only builds 
Add stdc++ to static libs in pkg-config
 2016-03-16 14:30:15 +00:00
Ricardo Constantino (:RiCON)
f53cb8cddd CMake: fix linking with Ninja 2016-03-16 14:17:47 +00:00
Ricardo Constantino (:RiCON)
36665da832 CMake: allow installing binary to non-default dir 2016-03-16 14:17:25 +00:00
Jehan
198190461e script: move the Wikipedia title syntax cleaning to BuildLangModel.py. 2016-02-21 16:20:22 +01:00
Jehan
d24bd7d578 script: Wikipedia API's python wrapper does not return garbage text anymore. 
I can't see new commits since 2014. So I am assuming the issue was on
Wikipedia side and that it has been fixed.
 2016-02-21 16:07:10 +01:00
Jehan
37024460fe script: add a README file dedicated to adding new support. 2016-02-21 16:06:11 +01:00
Jehan
42c6b42f65 Add a DOAP file. 
All URLs are still referring to the github project, because we have
no other homepage or bug tracker yet.
 2016-02-21 15:19:50 +01:00
Jehan
d5dba26e04 README: add Danish support for 3 charsets. 2016-02-19 19:11:56 +01:00
Jehan
923d264470 LangModels: add Danish support (Windows-1252, ISO-8859-1 and ISO-8859-15). 
Test for ISO-8859-1 is disabled for now since the difference is not big
enough, as for characters used in Danish, between ISO-8859-1 and
ISO-8859-15. Therefore the first to be declared "wins".
Let's see to improve this later.
Test contents from:
https://da.wikipedia.org/wiki/Eurosymbol
https://da.wikipedia.org/wiki/Dansk_%28sprog%29
 2016-02-19 19:10:41 +01:00
Jehan
1694999bce README: update with VISCII support. 2016-02-13 03:52:06 +01:00
Jehan
98b5e52252 LangModels: add VISCII encoding support and retrain Vietnamese model. 2016-02-13 03:51:18 +01:00
Jehan
600cf76a76 BuildLangModel: try using iconv for conversion when support missing... 
... in python. For instance I had the case where the VISCII encoding is
supported by iconv but not by encode/decode() function in core python.
 2016-02-13 03:47:41 +01:00
Jehan
178c6119b8 LangModels: add Windows-1258 support for Vietnamese. 
I was planning on adding VISCII support as well, but Python encode()
method does not have any support for it apparently, so I cannot generate
the proper statistics data with the current version of the string.
 2016-02-13 02:32:57 +01:00
Jehan
27135a8880 BuildLangModel: printing a message when discarding a page. 2016-02-13 02:27:15 +01:00
Jehan
0446e24c8d README: uchardet now available on Fedora. 
Already in Fedora devel and soon to be added as update on Fedora 23,
if I get it correctly. See:
https://bugzilla.redhat.com/show_bug.cgi?id=1264713
https://admin.fedoraproject.org/pkgdb/package/rpms/uchardet/
 2016-02-12 17:53:22 +01:00
Jehan
248d6dbd35 tools: exit with non-zero value on uchardet error. 2016-01-21 18:16:42 +01:00
Jehan
b6d872bbec app: package name wrong in CMakeLists.txt. 
Probably coming from a copy-paste error when the build system was
originally created.
 2015-12-15 21:40:16 +01:00