AuroraMiddleware/qt5base-lts
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9551d62d97
qt5base-lts/util/local_database/cldr2qlocalexml.py

604 lines
25 KiB
Python
Raw Normal View History

Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
#!/usr/bin/env python2
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
#############################################################################
##
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
## Copyright (C) 2017 The Qt Company Ltd.
Updated license headers From Qt 5.7 -> tools & applications are lisenced under GPL v3 with some exceptions, see http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/ Updated license headers to use new GPL-EXCEPT header instead of LGPL21 one (in those files which will be under GPL 3 with exceptions) Change-Id: I42a473ddc97101492a60b9287d90979d9eb35ae1 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
2016-01-15 12:36:27 +00:00
## Contact: https://www.qt.io/licensing/
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
##
## This file is part of the test suite of the Qt Toolkit.
##
Updated license headers From Qt 5.7 -> tools & applications are lisenced under GPL v3 with some exceptions, see http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/ Updated license headers to use new GPL-EXCEPT header instead of LGPL21 one (in those files which will be under GPL 3 with exceptions) Change-Id: I42a473ddc97101492a60b9287d90979d9eb35ae1 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
2016-01-15 12:36:27 +00:00
## $QT_BEGIN_LICENSE:GPL-EXCEPT$
Change copyrights from Nokia to Digia Change copyrights and license headers from Nokia to Digia Change-Id: If1cc974286d29fd01ec6c19dd4719a67f4c3f00e Reviewed-by: Lars Knoll <lars.knoll@digia.com> Reviewed-by: Sergio Ahumada <sergio.ahumada@digia.com>
2012-09-19 12:28:29 +00:00
## Commercial License Usage
## Licensees holding valid commercial Qt licenses may use this file in
## accordance with the commercial license agreement provided with the
## Software or, alternatively, in accordance with the terms contained in
Update copyright headers Qt copyrights are now in The Qt Company, so we could update the source code headers accordingly. In the same go we should also fix the links to point to qt.io. Outdated header.LGPL removed (use header.LGPL21 instead) Old header.LGPL3 renamed to header.LGPL3-COMM to match actual licensing combination. New header.LGPL-COMM taken in the use file which were using old header.LGPL3 (src/plugins/platforms/android/extract.cpp) Added new header.LGPL3 containing Commercial + LGPLv3 + GPLv2 license combination Change-Id: I6f49b819a8a20cc4f88b794a8f6726d975e8ffbe Reviewed-by: Matti Paaso <matti.paaso@theqtcompany.com>
2015-01-28 08:44:43 +00:00
## a written agreement between you and The Qt Company. For licensing terms
Updated license headers From Qt 5.7 -> tools & applications are lisenced under GPL v3 with some exceptions, see http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/ Updated license headers to use new GPL-EXCEPT header instead of LGPL21 one (in those files which will be under GPL 3 with exceptions) Change-Id: I42a473ddc97101492a60b9287d90979d9eb35ae1 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
2016-01-15 12:36:27 +00:00
## and conditions see https://www.qt.io/terms-conditions. For further
## information use the contact form at https://www.qt.io/contact-us.
Change copyrights from Nokia to Digia Change copyrights and license headers from Nokia to Digia Change-Id: If1cc974286d29fd01ec6c19dd4719a67f4c3f00e Reviewed-by: Lars Knoll <lars.knoll@digia.com> Reviewed-by: Sergio Ahumada <sergio.ahumada@digia.com>
2012-09-19 12:28:29 +00:00
##
Updated license headers From Qt 5.7 -> tools & applications are lisenced under GPL v3 with some exceptions, see http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/ Updated license headers to use new GPL-EXCEPT header instead of LGPL21 one (in those files which will be under GPL 3 with exceptions) Change-Id: I42a473ddc97101492a60b9287d90979d9eb35ae1 Reviewed-by: Thiago Macieira <thiago.macieira@intel.com> Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
2016-01-15 12:36:27 +00:00
## GNU General Public License Usage
## Alternatively, this file may be used under the terms of the GNU
## General Public License version 3 as published by the Free Software
## Foundation with exceptions as appearing in the file LICENSE.GPL3-EXCEPT
## included in the packaging of this file. Please review the following
## information to ensure the GNU General Public License requirements will
## be met: https://www.gnu.org/licenses/gpl-3.0.html.
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
##
## $QT_END_LICENSE$
##
#############################################################################
Document how to use the CLDR-related scripts Say where to find the data, how to unpack it and how to run the scripts to update our source tree from it. Change-Id: I5b680090bb5850f29c1e7d61efd1d450856b9949 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-23 13:24:35 +00:00
"""Convert CLDR data to qLocaleXML
The CLDR data can be downloaded from CLDR_, which has a sub-directory
for each version; you need the ``core.zip`` file for your version of
choice (typically the latest). This script has had updates to cope up
to v29; for later versions, we may need adaptations. Unpack the
downloaded ``core.zip`` and check it has a common/main/ sub-directory:
pass the path of that sub-directory to this script as its single
command-line argument. Save its standard output (but not error) to a
file for later processing by ``./qlocalexml2cpp.py``
.. _CLDR: ftp://unicode.org/Public/cldr/
"""
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
import os
import sys
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
import re
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
import enumdata
import xpathlite
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
from xpathlite import DraftResolution, findAlias, findEntry, findTagsInFile
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
from dateconverter import convert_date
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
from localexml import Locale
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
findEntryInFile = xpathlite._findEntryInFile
def parse_number_format(patterns, data):
# this is a very limited parsing of the number format for currency only.
def skip_repeating_pattern(x):
p = x.replace('0', '#').replace(',', '').replace('.', '')
seen = False
result = ''
for c in p:
if c == '#':
if seen:
continue
seen = True
else:
seen = False
result = result + c
return result
patterns = patterns.split(';')
result = []
for pattern in patterns:
pattern = skip_repeating_pattern(pattern)
pattern = pattern.replace('#', "%1")
# according to http://www.unicode.org/reports/tr35/#Number_Format_Patterns
# there can be doubled or trippled currency sign, however none of the
# locales use that.
pattern = pattern.replace(u'\xa4', "%2")
pattern = pattern.replace("''", "###").replace("'", '').replace("###", "'")
pattern = pattern.replace('-', data['minus'])
pattern = pattern.replace('+', data['plus'])
result.append(pattern)
return result
def parse_list_pattern_part_format(pattern):
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
# This is a very limited parsing of the format for list pattern part only.
return pattern.replace("{0}", "%1").replace("{1}", "%2").replace("{2}", "%3")
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
Add byte-based units to CLDR data Scan CLDR for {,kilo,mega,giga,tera,peta,exa}byte forms and their IEC equivalents, providing SI and IEC defaults when missing (which all of IEC are) in addition to the usual numeric data. Extrapolate from any present data (e.g. French's ko, Mo, Go, To imply Po, Eo and, for IEC, Kio, Mio, etc.), since CLDR only goes up to tera. Propagate this data to QLocale's database ready for use by QLocale::formattedDataSize(). Change-Id: Ie6ee978948c68be9f71ab784a128cbfae3d80ee1 Reviewed-by: Shawn Rutledge <shawn.rutledge@qt.io>
2017-05-30 12:55:33 +00:00
def unit_quantifiers(find, path, stem, suffix, known,
# Stop at exa/exbi: 16 exbi = 2^{64} < zetta =
# 1000^7 < zebi = 2^{70}, the next quantifiers up:
si_quantifiers = ('kilo', 'mega', 'giga', 'tera', 'peta', 'exa')):
"""Work out the unit quantifiers.
Unfortunately, the CLDR data only go up to terabytes and we want
all the way to exabytes; but we can recognize the SI quantifiers
as prefixes, strip and identify the tail as the localized
translation for 'B' (e.g. French has 'octet' for 'byte' and uses
ko, Mo, Go, To from which we can extrapolate Po, Eo).
Should be called first for the SI quantifiers, with suffix = 'B',
then for the IEC ones, with suffix = 'iB'; the list known
(initially empty before first call) is used to let the second call
know what the first learned about the localized unit.
"""
if suffix == 'B': # first call, known = []
tail = suffix
for q in si_quantifiers:
it = find(path, stem % q)
# kB for kilobyte, in contrast with KiB for IEC:
q = q[0] if q == 'kilo' else q[0].upper()
if not it:
it = q + tail
elif it.startswith(q):
rest = it[1:]
tail = rest if all(rest == k for k in known) else suffix
known.append(rest)
yield it
else: # second call, re-using first's known
assert suffix == 'iB'
if known:
byte = known.pop()
if all(byte == k for k in known):
suffix = 'i' + byte
for q in si_quantifiers:
yield find(path, stem % q[:2],
# Those don't (yet, v31) exist in CLDR, so we always fall back to:
q[0].upper() + suffix)
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
def generateLocaleInfo(path):
if not path.endswith(".xml"):
return {}
Use likelySubtags to instantiate a locale id from it's short form ...just like described in http://www.unicode.org/reports/tr35/#Likely_Subtags. This is much more effective than current "guessing" algorithm + makes it possible to instantiate a locale by the script or territory code only. Change-Id: I674f8476e65b01c56960b6e83a1a346df0715274 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-21 04:08:24 +00:00
# skip legacy/compatibility ones
alias = findAlias(path)
if alias:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
raise xpathlite.Error('alias to "%s"' % alias)
Use likelySubtags to instantiate a locale id from it's short form ...just like described in http://www.unicode.org/reports/tr35/#Likely_Subtags. This is much more effective than current "guessing" algorithm + makes it possible to instantiate a locale by the script or territory code only. Change-Id: I674f8476e65b01c56960b6e83a1a346df0715274 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-21 04:08:24 +00:00
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
def code(tag):
return findEntryInFile(path, 'identity/' + tag, attribute="type")[0]
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
return _generateLocaleInfo(path, code('language'), code('script'),
code('territory'), code('variant'))
[QLocaleData] Extract defaultContent locales This adds some locales missing in the common/main/ directory, namely: bss_CM, cch_NG, dv_MV, gaa_GH, gez_ET, ha_Arab_NG, iu_Cans_CA, kaj_NG, kcg_NG, kpe_LR, ku_Latn_TR, mi_NZ, ms_Arab_MY, mn_Mong_CN, nds_DE, ny_MW, oc_FR, sa_IN, sid_ET, tk_Latn_TM, trv_TW, tt_RU, ug_Arab_CN, wa_BE, wo_Latn_SN See http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content for more info. Change-Id: I6b3082d370a21da64fbd5e72ab6344e1d7a6a3c9 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2015-03-20 21:12:30 +00:00
def _generateLocaleInfo(path, language_code, script_code, country_code, variant_code=""):
if not path.endswith(".xml"):
return {}
if language_code == 'root':
# just skip it
return {}
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
# we do not support variants
# ### actually there is only one locale with variant: en_US_POSIX
# does anybody care about it at all?
if variant_code:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
raise xpathlite.Error('we do not support variants ("%s")' % variant_code)
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
language_id = enumdata.languageCodeToId(language_code)
Assign correct codes to QLocale::AnyCountry and AnyScript as their meaning, in fact, is unknown (or default) country/script. Change-Id: Id75a70d4b33c2092de414f3ac357f6bcb627ba47 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-14 16:09:02 +00:00
if language_id <= 0:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
raise xpathlite.Error('unknown language code "%s"' % language_code)
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
script_id = enumdata.scriptCodeToId(script_code)
Assign correct codes to QLocale::AnyCountry and AnyScript as their meaning, in fact, is unknown (or default) country/script. Change-Id: Id75a70d4b33c2092de414f3ac357f6bcb627ba47 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-14 16:09:02 +00:00
if script_id == -1:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
raise xpathlite.Error('unknown script code "%s"' % script_code)
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
[SIC/BIC] QLocale: update enums up to CLRD 1.9.1 This patch adds some missing codes (two-letter ones only), removes an outdated ones, and updates some names. The legacy language codes are handled in QLocalePrivate::codeToLanguage() (fortunately, there are only 4 of such codes). Change-Id: Iff50aecd1c762b6399cd151aebb955f341d366c6 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-21 13:45:18 +00:00
# we should handle fully qualified names with the territory
if not country_code:
return {}
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
country_id = enumdata.countryCodeToId(country_code)
Assign correct codes to QLocale::AnyCountry and AnyScript as their meaning, in fact, is unknown (or default) country/script. Change-Id: Id75a70d4b33c2092de414f3ac357f6bcb627ba47 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-14 16:09:02 +00:00
if country_id <= 0:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
raise xpathlite.Error('unknown country code "%s"' % country_code)
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
# So we say we accept only those values that have "contributed" or
# "approved" resolution. see http://www.unicode.org/cldr/process.html
# But we only respect the resolution for new datas for backward
# compatibility.
draft = DraftResolution.contributed
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
result = dict(
language=enumdata.language_list[language_id][0],
language_code=language_code, language_id=language_id,
script=enumdata.script_list[script_id][0],
script_code=script_code, script_id=script_id,
country=enumdata.country_list[country_id][0],
country_code=country_code, country_id=country_id,
variant_code=variant_code)
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
[QLocaleData] Extract defaultContent locales This adds some locales missing in the common/main/ directory, namely: bss_CM, cch_NG, dv_MV, gaa_GH, gez_ET, ha_Arab_NG, iu_Cans_CA, kaj_NG, kcg_NG, kpe_LR, ku_Latn_TR, mi_NZ, ms_Arab_MY, mn_Mong_CN, nds_DE, ny_MW, oc_FR, sa_IN, sid_ET, tk_Latn_TM, trv_TW, tt_RU, ug_Arab_CN, wa_BE, wo_Latn_SN See http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content for more info. Change-Id: I6b3082d370a21da64fbd5e72ab6344e1d7a6a3c9 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2015-03-20 21:12:30 +00:00
(dir_name, file_name) = os.path.split(path)
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
def from_supplement(tag,
path=os.path.join(dir_name, '..', 'supplemental',
'supplementalData.xml')):
return findTagsInFile(path, tag)
currencies = from_supplement('currencyData/region[iso3166=%s]' % country_code)
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
result['currencyIsoCode'] = ''
result['currencyDigits'] = 2
result['currencyRounding'] = 1
if currencies:
for e in currencies:
if e[0] == 'currency':
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
t = [x[1] == 'false' for x in e[1] if x[0] == 'tender']
if t and t[0]:
pass
elif not any(x[0] == 'to' for x in e[1]):
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
result['currencyIsoCode'] = (x[1] for x in e[1] if x[0] == 'iso4217').next()
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
break
if result['currencyIsoCode']:
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
t = from_supplement("currencyData/fractions/info[iso4217=%s]"
% result['currencyIsoCode'])
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
if t and t[0][0] == 'info':
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
result['currencyDigits'] = (int(x[1]) for x in t[0][1] if x[0] == 'digits').next()
result['currencyRounding'] = (int(x[1]) for x in t[0][1] if x[0] == 'rounding').next()
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
numbering_system = None
try:
numbering_system = findEntry(path, "numbers/defaultNumberingSystem")
except:
pass
def findEntryDef(path, xpath, value=''):
try:
return findEntry(path, xpath)
except xpathlite.Error:
return value
def get_number_in_system(path, xpath, numbering_system):
if numbering_system:
try:
return findEntry(path, xpath + "[numberSystem=" + numbering_system + "]")
except xpathlite.Error:
Update QLocale data from CLDR v1.8.1 to CLDR v1.9.1 Change-Id: Ic84bbc82b364b92605c1bba64b6ec815bff970cb Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
2012-03-22 12:07:07 +00:00
# in CLDR 1.9 number system was refactored for numbers (but not for currency)
# so if previous findEntry doesn't work we should try this:
try:
return findEntry(path, xpath.replace("/symbols/", "/symbols[numberSystem=" + numbering_system + "]/"))
except xpathlite.Error:
# fallback to default
pass
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
return findEntry(path, xpath)
Update QLocale data from CLDR v1.8.1 to CLDR v1.9.1 Change-Id: Ic84bbc82b364b92605c1bba64b6ec815bff970cb Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
2012-03-22 12:07:07 +00:00
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
result['decimal'] = get_number_in_system(path, "numbers/symbols/decimal", numbering_system)
result['group'] = get_number_in_system(path, "numbers/symbols/group", numbering_system)
result['list'] = get_number_in_system(path, "numbers/symbols/list", numbering_system)
result['percent'] = get_number_in_system(path, "numbers/symbols/percentSign", numbering_system)
Update QLocale data from CLDR v1.8.1 to CLDR v1.9.1 Change-Id: Ic84bbc82b364b92605c1bba64b6ec815bff970cb Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
2012-03-22 12:07:07 +00:00
try:
numbering_systems = {}
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
for ns in findTagsInFile(os.path.join(cldr_dir, '..', 'supplemental',
'numberingSystems.xml'),
'numberingSystems'):
Update QLocale data from CLDR v1.8.1 to CLDR v1.9.1 Change-Id: Ic84bbc82b364b92605c1bba64b6ec815bff970cb Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
2012-03-22 12:07:07 +00:00
tmp = {}
id = ""
for data in ns[1:][0]: # ns looks like this: [u'numberingSystem', [(u'digits', u'0123456789'), (u'type', u'numeric'), (u'id', u'latn')]]
tmp[data[0]] = data[1]
if data[0] == u"id":
id = data[1]
numbering_systems[id] = tmp
result['zero'] = numbering_systems[numbering_system][u"digits"][0]
except e:
sys.stderr.write("Native zero detection problem:\n" + str(e) + "\n")
result['zero'] = get_number_in_system(path, "numbers/symbols/nativeZeroDigit", numbering_system)
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
result['minus'] = get_number_in_system(path, "numbers/symbols/minusSign", numbering_system)
result['plus'] = get_number_in_system(path, "numbers/symbols/plusSign", numbering_system)
result['exp'] = get_number_in_system(path, "numbers/symbols/exponential", numbering_system).lower()
result['quotationStart'] = findEntry(path, "delimiters/quotationStart")
result['quotationEnd'] = findEntry(path, "delimiters/quotationEnd")
result['alternateQuotationStart'] = findEntry(path, "delimiters/alternateQuotationStart")
result['alternateQuotationEnd'] = findEntry(path, "delimiters/alternateQuotationEnd")
result['listPatternPartStart'] = parse_list_pattern_part_format(findEntry(path, "listPatterns/listPattern/listPatternPart[start]"))
result['listPatternPartMiddle'] = parse_list_pattern_part_format(findEntry(path, "listPatterns/listPattern/listPatternPart[middle]"))
result['listPatternPartEnd'] = parse_list_pattern_part_format(findEntry(path, "listPatterns/listPattern/listPatternPart[end]"))
result['listPatternPartTwo'] = parse_list_pattern_part_format(findEntry(path, "listPatterns/listPattern/listPatternPart[2]"))
result['am'] = findEntry(path, "dates/calendars/calendar[gregorian]/dayPeriods/dayPeriodContext[format]/dayPeriodWidth[wide]/dayPeriod[am]", draft)
result['pm'] = findEntry(path, "dates/calendars/calendar[gregorian]/dayPeriods/dayPeriodContext[format]/dayPeriodWidth[wide]/dayPeriod[pm]", draft)
result['longDateFormat'] = convert_date(findEntry(path, "dates/calendars/calendar[gregorian]/dateFormats/dateFormatLength[full]/dateFormat/pattern"))
result['shortDateFormat'] = convert_date(findEntry(path, "dates/calendars/calendar[gregorian]/dateFormats/dateFormatLength[short]/dateFormat/pattern"))
result['longTimeFormat'] = convert_date(findEntry(path, "dates/calendars/calendar[gregorian]/timeFormats/timeFormatLength[full]/timeFormat/pattern"))
result['shortTimeFormat'] = convert_date(findEntry(path, "dates/calendars/calendar[gregorian]/timeFormats/timeFormatLength[short]/timeFormat/pattern"))
endonym = None
if country_code and script_code:
endonym = findEntryDef(path, "localeDisplayNames/languages/language[type=%s_%s_%s]" % (language_code, script_code, country_code))
if not endonym and script_code:
endonym = findEntryDef(path, "localeDisplayNames/languages/language[type=%s_%s]" % (language_code, script_code))
if not endonym and country_code:
endonym = findEntryDef(path, "localeDisplayNames/languages/language[type=%s_%s]" % (language_code, country_code))
if not endonym:
endonym = findEntryDef(path, "localeDisplayNames/languages/language[type=%s]" % (language_code))
result['language_endonym'] = endonym
result['country_endonym'] = findEntryDef(path, "localeDisplayNames/territories/territory[type=%s]" % (country_code))
currency_format = get_number_in_system(path, "numbers/currencyFormats/currencyFormatLength/currencyFormat/pattern", numbering_system)
currency_format = parse_number_format(currency_format, result)
result['currencyFormat'] = currency_format[0]
result['currencyNegativeFormat'] = ''
if len(currency_format) > 1:
result['currencyNegativeFormat'] = currency_format[1]
result['currencySymbol'] = ''
result['currencyDisplayName'] = ''
if result['currencyIsoCode']:
result['currencySymbol'] = findEntryDef(path, "numbers/currencies/currency[%s]/symbol" % result['currencyIsoCode'])
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
result['currencyDisplayName'] = ';'.join(
findEntryDef(path, 'numbers/currencies/currency[' + result['currencyIsoCode']
+ ']/displayName' + tail)
for tail in ['',] + [
'[count=%s]' % x for x in ('zero', 'one', 'two', 'few', 'many', 'other')
]) + ';'
Add byte-based units to CLDR data Scan CLDR for {,kilo,mega,giga,tera,peta,exa}byte forms and their IEC equivalents, providing SI and IEC defaults when missing (which all of IEC are) in addition to the usual numeric data. Extrapolate from any present data (e.g. French's ko, Mo, Go, To imply Po, Eo and, for IEC, Kio, Mio, etc.), since CLDR only goes up to tera. Propagate this data to QLocale's database ready for use by QLocale::formattedDataSize(). Change-Id: Ie6ee978948c68be9f71ab784a128cbfae3d80ee1 Reviewed-by: Shawn Rutledge <shawn.rutledge@qt.io>
2017-05-30 12:55:33 +00:00
def findUnitDef(path, stem, fallback=''):
# The displayName for a quantified unit in en.xml is kByte
# instead of kB (etc.), so prefer any unitPattern provided:
for count in ('many', 'few', 'two', 'other', 'zero', 'one'):
try:
ans = findEntry(path, stem + 'unitPattern[count=%s]' % count)
except xpathlite.Error:
continue
# TODO: epxloit count-handling, instead of discarding placeholders
if ans.startswith('{0}'):
ans = ans[3:].lstrip()
if ans:
return ans
return findEntryDef(path, stem + 'displayName', fallback)
# First without quantifier, then quantified each way:
result['byte_unit'] = findEntryDef(
path, 'units/unitLength[type=long]/unit[type=digital-byte]/displayName',
'bytes')
stem = 'units/unitLength[type=short]/unit[type=digital-%sbyte]/'
known = [] # cases where we *do* have a given version:
result['byte_si_quantified'] = ';'.join(unit_quantifiers(findUnitDef, path, stem, 'B', known))
# IEC 60027-2
# http://physics.nist.gov/cuu/Units/binary.html
result['byte_iec_quantified'] = ';'.join(unit_quantifiers(findUnitDef, path, stem % '%sbi', 'iB', known))
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
# Used for month and day data:
namings = (
('standaloneLong', 'stand-alone', 'wide'),
('standaloneShort', 'stand-alone', 'abbreviated'),
('standaloneNarrow', 'stand-alone', 'narrow'),
('long', 'format', 'wide'),
('short', 'format', 'abbreviated'),
('narrow', 'format', 'narrow'),
)
# Month data:
for cal in ('gregorian',): # We shall want to add to this
stem = 'dates/calendars/calendar[' + cal + ']/months/'
for (key, mode, size) in namings:
prop = 'monthContext[' + mode + ']/monthWidth[' + size + ']/'
result[key + 'Months'] = ';'.join(
findEntry(path, stem + prop + "month[%d]" % i)
for i in range(1, 13)) + ';'
# Day data (for Gregorian, at least):
stem = 'dates/calendars/calendar[gregorian]/days/'
days = ('sun', 'mon', 'tue', 'wed', 'thu', 'fri', 'sat')
for (key, mode, size) in namings:
prop = 'dayContext[' + mode + ']/dayWidth[' + size + ']/day'
result[key + 'Days'] = ';'.join(
findEntry(path, stem + prop + '[' + day + ']')
for day in days) + ';'
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
return Locale(result)
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
def addEscapes(s):
result = ''
for c in s:
n = ord(c)
if n < 128:
result += c
else:
result += "\\x"
result += "%02x" % (n)
return result
def unicodeStr(s):
utf8 = s.encode('utf-8')
return "<size>" + str(len(utf8)) + "</size><data>" + addEscapes(utf8) + "</data>"
def usage():
print "Usage: cldr2qlocalexml.py <path-to-cldr-main>"
sys.exit()
def integrateWeekData(filePath):
if not filePath.endswith(".xml"):
return {}
Simplify cldr2qlocalexml.integrateWeekData() It had a separate variable for each of three lists for each day of the week; and used each list only once. Iterate the days of the week for each lookup, discarding it once used. Change-Id: I32c8bd5bfcbb99f0a8697d374e63112761f18dbb Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 17:34:09 +00:00
def lookup(key):
return findEntryInFile(filePath, key, attribute='territories')[0].split()
days = ('mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun')
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
firstDayByCountryCode = {}
Simplify cldr2qlocalexml.integrateWeekData() It had a separate variable for each of three lists for each day of the week; and used each list only once. Iterate the days of the week for each lookup, discarding it once used. Change-Id: I32c8bd5bfcbb99f0a8697d374e63112761f18dbb Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 17:34:09 +00:00
for day in days:
for countryCode in lookup('weekData/firstDay[day=%s]' % day):
firstDayByCountryCode[countryCode] = day
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
weekendStartByCountryCode = {}
Simplify cldr2qlocalexml.integrateWeekData() It had a separate variable for each of three lists for each day of the week; and used each list only once. Iterate the days of the week for each lookup, discarding it once used. Change-Id: I32c8bd5bfcbb99f0a8697d374e63112761f18dbb Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 17:34:09 +00:00
for day in days:
for countryCode in lookup('weekData/weekendStart[day=%s]' % day):
weekendStartByCountryCode[countryCode] = day
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
weekendEndByCountryCode = {}
Simplify cldr2qlocalexml.integrateWeekData() It had a separate variable for each of three lists for each day of the week; and used each list only once. Iterate the days of the week for each lookup, discarding it once used. Change-Id: I32c8bd5bfcbb99f0a8697d374e63112761f18dbb Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 17:34:09 +00:00
for day in days:
for countryCode in lookup('weekData/weekendEnd[day=%s]' % day):
weekendEndByCountryCode[countryCode] = day
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
for (key, locale) in locale_database.iteritems():
countryCode = locale.country_code
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
if countryCode in firstDayByCountryCode:
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
locale.firstDayOfWeek = firstDayByCountryCode[countryCode]
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
else:
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
locale.firstDayOfWeek = firstDayByCountryCode["001"]
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
if countryCode in weekendStartByCountryCode:
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
locale.weekendStart = weekendStartByCountryCode[countryCode]
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
else:
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
locale.weekendStart = weekendStartByCountryCode["001"]
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
if countryCode in weekendEndByCountryCode:
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
locale.weekendEnd = weekendEndByCountryCode[countryCode]
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
else:
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
locale.weekendEnd = weekendEndByCountryCode["001"]
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
if len(sys.argv) != 2:
usage()
cldr_dir = sys.argv[1]
if not os.path.isdir(cldr_dir):
usage()
cldr_files = os.listdir(cldr_dir)
locale_database = {}
[QLocaleData] Extract defaultContent locales This adds some locales missing in the common/main/ directory, namely: bss_CM, cch_NG, dv_MV, gaa_GH, gez_ET, ha_Arab_NG, iu_Cans_CA, kaj_NG, kcg_NG, kpe_LR, ku_Latn_TR, mi_NZ, ms_Arab_MY, mn_Mong_CN, nds_DE, ny_MW, oc_FR, sa_IN, sid_ET, tk_Latn_TM, trv_TW, tt_RU, ug_Arab_CN, wa_BE, wo_Latn_SN See http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content for more info. Change-Id: I6b3082d370a21da64fbd5e72ab6344e1d7a6a3c9 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2015-03-20 21:12:30 +00:00
# see http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content
defaultContent_locales = {}
Clean up in CLDR scripts for QLocale Use python more fluently; DRY - use functions to avoid repetition; use dict-comprehensions; use os.path.join() in preference to arithmetic with path strings; use elsif to avoid the need for a local variable; set() can take a generator directly, no need to go via a list; don't end lines in semicolon (this is python). Test isdir() once instead of exists() twice on the same name. Just, generally, use python's feature-set. Change-Id: Ib114aa016f70b3be09e968d9cfc069b057f49d41 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-31 19:42:11 +00:00
for ns in findTagsInFile(os.path.join(cldr_dir, '..', 'supplemental',
'supplementalMetadata.xml'),
'metadata/defaultContent'):
[QLocaleData] Extract defaultContent locales This adds some locales missing in the common/main/ directory, namely: bss_CM, cch_NG, dv_MV, gaa_GH, gez_ET, ha_Arab_NG, iu_Cans_CA, kaj_NG, kcg_NG, kpe_LR, ku_Latn_TR, mi_NZ, ms_Arab_MY, mn_Mong_CN, nds_DE, ny_MW, oc_FR, sa_IN, sid_ET, tk_Latn_TM, trv_TW, tt_RU, ug_Arab_CN, wa_BE, wo_Latn_SN See http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content for more info. Change-Id: I6b3082d370a21da64fbd5e72ab6344e1d7a6a3c9 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2015-03-20 21:12:30 +00:00
for data in ns[1:][0]:
if data[0] == u"locales":
defaultContent_locales = data[1].split()
for file in defaultContent_locales:
items = file.split("_")
if len(items) == 3:
language_code = items[0]
script_code = items[1]
country_code = items[2]
else:
if len(items) != 2:
Give reasons when skipping files Change-Id: I397d17ad252363dd0d7d9b4f53b7d105075da306 Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com> Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-23 13:57:40 +00:00
sys.stderr.write('skipping defaultContent locale "' + file + '" [neither lang_script_country nor lang_country]\n')
[QLocaleData] Extract defaultContent locales This adds some locales missing in the common/main/ directory, namely: bss_CM, cch_NG, dv_MV, gaa_GH, gez_ET, ha_Arab_NG, iu_Cans_CA, kaj_NG, kcg_NG, kpe_LR, ku_Latn_TR, mi_NZ, ms_Arab_MY, mn_Mong_CN, nds_DE, ny_MW, oc_FR, sa_IN, sid_ET, tk_Latn_TM, trv_TW, tt_RU, ug_Arab_CN, wa_BE, wo_Latn_SN See http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content for more info. Change-Id: I6b3082d370a21da64fbd5e72ab6344e1d7a6a3c9 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2015-03-20 21:12:30 +00:00
continue
language_code = items[0]
script_code = ""
country_code = items[1]
if len(country_code) == 4:
Give reasons when skipping files Change-Id: I397d17ad252363dd0d7d9b4f53b7d105075da306 Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com> Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-23 13:57:40 +00:00
sys.stderr.write('skipping defaultContent locale "' + file + '" [long country code]\n')
[QLocaleData] Extract defaultContent locales This adds some locales missing in the common/main/ directory, namely: bss_CM, cch_NG, dv_MV, gaa_GH, gez_ET, ha_Arab_NG, iu_Cans_CA, kaj_NG, kcg_NG, kpe_LR, ku_Latn_TR, mi_NZ, ms_Arab_MY, mn_Mong_CN, nds_DE, ny_MW, oc_FR, sa_IN, sid_ET, tk_Latn_TM, trv_TW, tt_RU, ug_Arab_CN, wa_BE, wo_Latn_SN See http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content for more info. Change-Id: I6b3082d370a21da64fbd5e72ab6344e1d7a6a3c9 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2015-03-20 21:12:30 +00:00
continue
try:
l = _generateLocaleInfo(cldr_dir + "/" + file + ".xml", language_code, script_code, country_code)
if not l:
Give reasons when skipping files Change-Id: I397d17ad252363dd0d7d9b4f53b7d105075da306 Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com> Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-23 13:57:40 +00:00
sys.stderr.write('skipping defaultContent locale "' + file + '" [no locale info generated]\n')
[QLocaleData] Extract defaultContent locales This adds some locales missing in the common/main/ directory, namely: bss_CM, cch_NG, dv_MV, gaa_GH, gez_ET, ha_Arab_NG, iu_Cans_CA, kaj_NG, kcg_NG, kpe_LR, ku_Latn_TR, mi_NZ, ms_Arab_MY, mn_Mong_CN, nds_DE, ny_MW, oc_FR, sa_IN, sid_ET, tk_Latn_TM, trv_TW, tt_RU, ug_Arab_CN, wa_BE, wo_Latn_SN See http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content for more info. Change-Id: I6b3082d370a21da64fbd5e72ab6344e1d7a6a3c9 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2015-03-20 21:12:30 +00:00
continue
except xpathlite.Error as e:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
sys.stderr.write('skipping defaultContent locale "%s" (%s)\n' % (file, str(e)))
[QLocaleData] Extract defaultContent locales This adds some locales missing in the common/main/ directory, namely: bss_CM, cch_NG, dv_MV, gaa_GH, gez_ET, ha_Arab_NG, iu_Cans_CA, kaj_NG, kcg_NG, kpe_LR, ku_Latn_TR, mi_NZ, ms_Arab_MY, mn_Mong_CN, nds_DE, ny_MW, oc_FR, sa_IN, sid_ET, tk_Latn_TM, trv_TW, tt_RU, ug_Arab_CN, wa_BE, wo_Latn_SN See http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content for more info. Change-Id: I6b3082d370a21da64fbd5e72ab6344e1d7a6a3c9 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2015-03-20 21:12:30 +00:00
continue
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
locale_database[(l.language_id, l.script_id, l.country_id, l.variant_code)] = l
[QLocaleData] Extract defaultContent locales This adds some locales missing in the common/main/ directory, namely: bss_CM, cch_NG, dv_MV, gaa_GH, gez_ET, ha_Arab_NG, iu_Cans_CA, kaj_NG, kcg_NG, kpe_LR, ku_Latn_TR, mi_NZ, ms_Arab_MY, mn_Mong_CN, nds_DE, ny_MW, oc_FR, sa_IN, sid_ET, tk_Latn_TM, trv_TW, tt_RU, ug_Arab_CN, wa_BE, wo_Latn_SN See http://www.unicode.org/reports/tr35/tr35-info.html#Default_Content for more info. Change-Id: I6b3082d370a21da64fbd5e72ab6344e1d7a6a3c9 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2015-03-20 21:12:30 +00:00
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
for file in cldr_files:
Use likelySubtags to instantiate a locale id from it's short form ...just like described in http://www.unicode.org/reports/tr35/#Likely_Subtags. This is much more effective than current "guessing" algorithm + makes it possible to instantiate a locale by the script or territory code only. Change-Id: I674f8476e65b01c56960b6e83a1a346df0715274 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-21 04:08:24 +00:00
try:
l = generateLocaleInfo(cldr_dir + "/" + file)
if not l:
Give reasons when skipping files Change-Id: I397d17ad252363dd0d7d9b4f53b7d105075da306 Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com> Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-23 13:57:40 +00:00
sys.stderr.write('skipping file "' + file + '" [no locale info generated]\n')
Use likelySubtags to instantiate a locale id from it's short form ...just like described in http://www.unicode.org/reports/tr35/#Likely_Subtags. This is much more effective than current "guessing" algorithm + makes it possible to instantiate a locale by the script or territory code only. Change-Id: I674f8476e65b01c56960b6e83a1a346df0715274 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-21 04:08:24 +00:00
continue
except xpathlite.Error as e:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
sys.stderr.write('skipping file "%s" (%s)\n' % (file, str(e)))
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
continue
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
locale_database[(l.language_id, l.script_id, l.country_id, l.variant_code)] = l
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
integrateWeekData(cldr_dir+"/../supplemental/supplementalData.xml")
locale_keys = locale_database.keys()
locale_keys.sort()
cldr_version = 'unknown'
ldml = open(cldr_dir+"/../dtd/ldml.dtd", "r")
for line in ldml:
if 'version cldrVersion CDATA #FIXED' in line:
cldr_version = line.split('"')[1]
print "<localeDatabase>"
print " <version>" + cldr_version + "</version>"
print " <languageList>"
for id in enumdata.language_list:
l = enumdata.language_list[id]
print " <language>"
print " <name>" + l[0] + "</name>"
print " <id>" + str(id) + "</id>"
print " <code>" + l[1] + "</code>"
print " </language>"
print " </languageList>"
print " <scriptList>"
for id in enumdata.script_list:
l = enumdata.script_list[id]
print " <script>"
print " <name>" + l[0] + "</name>"
print " <id>" + str(id) + "</id>"
print " <code>" + l[1] + "</code>"
print " </script>"
print " </scriptList>"
print " <countryList>"
for id in enumdata.country_list:
l = enumdata.country_list[id]
print " <country>"
print " <name>" + l[0] + "</name>"
print " <id>" + str(id) + "</id>"
print " <code>" + l[1] + "</code>"
print " </country>"
print " </countryList>"
QLocale: replace hard-coded default country-for-language map ...with a generated one in a way similar to what http://www.unicode.org/reports/tr35/#Likely_Subtags suggests. The supplemental/likelySubtags.xml contains all the required data. This changes some default countries to a most-expected ones. Change-Id: I920a5623601d8661a943e78197d3bcc838191483 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-19 17:12:58 +00:00
def _parseLocale(l):
language = "AnyLanguage"
script = "AnyScript"
country = "AnyCountry"
Use likelySubtags to instantiate a locale id from it's short form ...just like described in http://www.unicode.org/reports/tr35/#Likely_Subtags. This is much more effective than current "guessing" algorithm + makes it possible to instantiate a locale by the script or territory code only. Change-Id: I674f8476e65b01c56960b6e83a1a346df0715274 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-21 04:08:24 +00:00
if l == "und":
raise xpathlite.Error("we are treating unknown locale like C")
QLocale: replace hard-coded default country-for-language map ...with a generated one in a way similar to what http://www.unicode.org/reports/tr35/#Likely_Subtags suggests. The supplemental/likelySubtags.xml contains all the required data. This changes some default countries to a most-expected ones. Change-Id: I920a5623601d8661a943e78197d3bcc838191483 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-19 17:12:58 +00:00
items = l.split("_")
language_code = items[0]
if language_code != "und":
language_id = enumdata.languageCodeToId(language_code)
if language_id == -1:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
raise xpathlite.Error('unknown language code "%s"' % language_code)
QLocale: replace hard-coded default country-for-language map ...with a generated one in a way similar to what http://www.unicode.org/reports/tr35/#Likely_Subtags suggests. The supplemental/likelySubtags.xml contains all the required data. This changes some default countries to a most-expected ones. Change-Id: I920a5623601d8661a943e78197d3bcc838191483 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-19 17:12:58 +00:00
language = enumdata.language_list[language_id][0]
if len(items) > 1:
script_code = items[1]
country_code = ""
if len(items) > 2:
country_code = items[2]
if len(script_code) == 4:
script_id = enumdata.scriptCodeToId(script_code)
if script_id == -1:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
raise xpathlite.Error('unknown script code "%s"' % script_code)
QLocale: replace hard-coded default country-for-language map ...with a generated one in a way similar to what http://www.unicode.org/reports/tr35/#Likely_Subtags suggests. The supplemental/likelySubtags.xml contains all the required data. This changes some default countries to a most-expected ones. Change-Id: I920a5623601d8661a943e78197d3bcc838191483 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-19 17:12:58 +00:00
script = enumdata.script_list[script_id][0]
else:
country_code = script_code
if country_code:
country_id = enumdata.countryCodeToId(country_code)
if country_id == -1:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
raise xpathlite.Error('unknown country code "%s"' % country_code)
QLocale: replace hard-coded default country-for-language map ...with a generated one in a way similar to what http://www.unicode.org/reports/tr35/#Likely_Subtags suggests. The supplemental/likelySubtags.xml contains all the required data. This changes some default countries to a most-expected ones. Change-Id: I920a5623601d8661a943e78197d3bcc838191483 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-19 17:12:58 +00:00
country = enumdata.country_list[country_id][0]
return (language, script, country)
print " <likelySubtags>"
for ns in findTagsInFile(cldr_dir + "/../supplemental/likelySubtags.xml", "likelySubtags"):
tmp = {}
for data in ns[1:][0]: # ns looks like this: [u'likelySubtag', [(u'from', u'aa'), (u'to', u'aa_Latn_ET')]]
tmp[data[0]] = data[1]
Use likelySubtags to instantiate a locale id from it's short form ...just like described in http://www.unicode.org/reports/tr35/#Likely_Subtags. This is much more effective than current "guessing" algorithm + makes it possible to instantiate a locale by the script or territory code only. Change-Id: I674f8476e65b01c56960b6e83a1a346df0715274 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-21 04:08:24 +00:00
try:
(from_language, from_script, from_country) = _parseLocale(tmp[u"from"])
except xpathlite.Error as e:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
sys.stderr.write('skipping likelySubtag "%s" -> "%s" (%s)\n' % (tmp[u"from"], tmp[u"to"], str(e)))
QLocale: replace hard-coded default country-for-language map ...with a generated one in a way similar to what http://www.unicode.org/reports/tr35/#Likely_Subtags suggests. The supplemental/likelySubtags.xml contains all the required data. This changes some default countries to a most-expected ones. Change-Id: I920a5623601d8661a943e78197d3bcc838191483 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-19 17:12:58 +00:00
continue
Use likelySubtags to instantiate a locale id from it's short form ...just like described in http://www.unicode.org/reports/tr35/#Likely_Subtags. This is much more effective than current "guessing" algorithm + makes it possible to instantiate a locale by the script or territory code only. Change-Id: I674f8476e65b01c56960b6e83a1a346df0715274 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-21 04:08:24 +00:00
try:
(to_language, to_script, to_country) = _parseLocale(tmp[u"to"])
except xpathlite.Error as e:
Use python more competently in CLDR/qLocaleXML scripts Make the python2 dependency explicit (sooner or later, python3 shall be the default /bin/env python), make time-zone script executable (it had a shebang). Use triple-quoted strings, or single quotes, to avoid extra backslashes, remove some simply spurious backslashes. Use generators rather than map or filter with lambdas and iterate rather than duplicating code. Clarify some comments. Regenerated headers: this upates the date of generation, cuts back a double-blank-line to single and skips a spurious trailing comma-newline on an array's data. Change-Id: I54439f0dec132865991fe5147d509cea0f9419a0 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-12 09:59:18 +00:00
sys.stderr.write('skipping likelySubtag "%s" -> "%s" (%s)\n' % (tmp[u"from"], tmp[u"to"], str(e)))
QLocale: replace hard-coded default country-for-language map ...with a generated one in a way similar to what http://www.unicode.org/reports/tr35/#Likely_Subtags suggests. The supplemental/likelySubtags.xml contains all the required data. This changes some default countries to a most-expected ones. Change-Id: I920a5623601d8661a943e78197d3bcc838191483 Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2012-11-19 17:12:58 +00:00
continue
# substitute according to http://www.unicode.org/reports/tr35/#Likely_Subtags
if to_country == "AnyCountry" and from_country != to_country:
to_country = from_country
if to_script == "AnyScript" and from_script != to_script:
to_script = from_script
print " <likelySubtag>"
print " <from>"
print " <language>" + from_language + "</language>"
print " <script>" + from_script + "</script>"
print " <country>" + from_country + "</country>"
print " </from>"
print " <to>"
print " <language>" + to_language + "</language>"
print " <script>" + to_script + "</script>"
print " <country>" + to_country + "</country>"
print " </to>"
print " </likelySubtag>"
print " </likelySubtags>"
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
print " <localeList>"
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
Locale.C().toXml()
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
for key in locale_keys:
Rework locale serialization and parsing with less repetition ... because copy-and-paste is worth discouraging. Moved code that writes and digests our Q Local XML form of the data into a common class, localexml.Locale, for use by the scripts that write and read it. Hopefully, it'll be easier to keep what's written and read in sync hereafter. Inlined some trivial functions in the process; and only create a day-number mapping dictionary once, instead of once per use. Also made it easier to see which attributes get which special handling (and documented this); and revised an assertion to be more helpful. Change-Id: I711b6a193a4ad94b5ff714c025f2732cd1a965a7 Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2017-05-30 13:50:47 +00:00
locale_database[key].toXml()
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
print " </localeList>"
print "</localeDatabase>"
Reference in New Issue Copy Permalink