7 Commits

Author	SHA1	Message	Date
Jehan	0270b1e856	Adding French Windows-1252 support.	2015-12-03 21:22:30 +01:00
Jehan	192f8de165	BuildLangModel: build models with computed frequent characters count.	2015-11-30 00:04:44 +01:00
Jehan	429448199f	French language model: fix a start page. ... Because of a bug in the Wikipedia querying Python library.	2015-11-29 23:55:03 +01:00
Jehan	b64831ff89	BuildLangModel: allow a list of start pages... ... ... and add a page with a word with œ in French to make sure we have such words in our stats.	2015-11-29 15:51:23 +01:00
Jehan	7f290975ba	BuildLangModel: map different cases of the same character together. ... With the new case_mapping lang property, we can consider upper and lower case versions of the same character as one character. This makes sense in some language, and would allow to enter some rarer characters (but still in the main alphabet) inside the frequent character list. For instance 'œ' and 'Œ' in French.	2015-11-29 02:14:48 +01:00
Jehan	00a78faa1d	BuildLangModel: the max_depth should be a script option... ... ... rather than a language property.	2015-11-29 01:59:28 +01:00
Jehan	0314f98ece	BuildLangModel.py: some in-progress script to build language models.	2015-11-29 01:30:04 +01:00