Commit Graph

28 Commits

Author SHA1 Message Date
Edward Welbourne
a0c5a7c483 Wrap CLDR-derived time-zone ID tables to within a 100-column margin
Ugly display in code-review bugged me ...

Change-Id: If9243907973cad49f1a5c8d78ff23c14fab2d4b4
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
2023-08-07 19:51:09 +02:00
Edward Welbourne
0de7e7a066 LocaleDB: Make passing qtbase root dir on command-lines optional
We can easily enough obtain the root of the present source tree using
the value of __file__, so might as well do so.

Change-Id: If14773ac1127278b6018a090c0b376437b9c6eec
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
2022-11-01 19:14:14 +02:00
Yuhang Zhao
42fe677584 Core: make CLDR data constexpr
Task-number: QTBUG-100485
Pick-to: 6.3 6.2
Change-Id: Ib8c5160ca0994662a5fdc2293dc734c1bdcac4f2
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-05-27 09:33:28 +08:00
Lucie Gérard
05fc3aef53 Use SPDX license identifiers
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.

Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-16 16:37:38 +02:00
Ievgenii Meshcheriakov
2c505eea01 locale_database: Add entry for 'South Sudan Standard Time'
This timezone is new in CLDR 40, Olson database calls it Africa/Juba.
The offset is UTC+2.

Reference: https://techcommunity.microsoft.com/t5/daylight-saving-time-time-zone/2021-time-zone-updates-for-republic-of-south-sudan-now-available/ba-p/2234981

Task-number: QTBUG-94358
Pick-to: 6.2
Change-Id: Ib70dbd9b472eb9cf8cb62a0eb5e241199148c077
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-11-10 00:36:11 +01:00
Edward Welbourne
ce868ffb7c Remove trailing zero rows from qtimezoneprivate_data_p.h's tables
They are not needed. Iterations over the table track their sizes.
The size-of-table constants just needed their -1s removed.
Incidentally use std::size() rather than sizeof(array)/sizeof(element).

Change-Id: Ie20eef9f6f5786d93c10b830a87e006d3c5bcc1a
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2021-09-10 19:33:06 +02:00
Ievgenii Meshcheriakov
e9519d207e locale_database: Use context manager interface to update source files
Use context manager interface (with statement) to atomically update source
files. This ensures that all files are properly closed and the temporary
file is removed even in case of errors.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I18cd96f1d03e467dea6212f6576a41e65f414ce1
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2021-07-20 16:51:51 +02:00
Ievgenii Meshcheriakov
2d0c2c9f1c locale_database: Use pathlib to manipulate paths in Python code
pathlib's API is more modern and easier to use than os.path. It
also allows to distinguish between paths and other strings in type
annotations.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ie6d9b4e35596f7f6befa4c9635f4a65ea3b20025
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-19 22:05:54 +02:00
Ievgenii Meshcheriakov
5ef5dce53b locale_database: Use argparse module to parse command line arguments
arparse is the standard way to parse command line arguments in Python.
It provides help and usage information for free and is easier to extend
than a custom argument parser.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I1e4c9cd914449e083d01932bc871ef10d26f0bc2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-16 19:04:20 +02:00
Ievgenii Meshcheriakov
41458fafa0 locale_database: Use f-strings in Python code
Replace most uses of str.format() and string arithmetic by f-strings.
This results in more compact code and the code is easier to read
when using an appropriate editor.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I3409f745b5d0324985cbd5690f5eda8d09b869ca
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-16 19:04:20 +02:00
Ievgenii Meshcheriakov
53382b7b07 locale_database: Don't use u prefix for strings in python files
This prefix is useless with Python 3.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ic008d53fe506865759e9a5003f439f7ac107b9e6
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-15 17:06:53 +02:00
Ievgenii Meshcheriakov
b02d17c5c0 Convert CLDR scripts to Python 3
The convertion is moslty done using 2to3 script with manual cleanup
afterwards.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I4d33b04e7269c55a83ff2deb876a23a78a89f39d
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-15 17:06:53 +02:00
Ievgenii Meshcheriakov
2300146085 locale_database: Don't attempt to access property 'message' of IOError
IOError does not have property 'message' in Python 3. Instead of
attempting to access it, just use the string representation of
the exception object. This produces the error message possibly combined
with additional arguments in both Python 2 and Python 3.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Icb198a409e7f80b832e474d8390b770fdeacc6c2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-07 19:32:52 +02:00
Edward Welbourne
e51831260a Nomenclature change: s/countr/territor/g in locale scripts
Change the nomenclature used in the scripts and the QLocaleXML data
format to use "territory" and "territories" in place of "country" and
"countries". Does not change the generated source files.

Change-Id: I4b208d8d01ad2bfc70d289fa6551f7e0355df5ef
Reviewed-by: JiDe Zhang <zhangjide@uniontech.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-05-26 18:00:01 +02:00
Edward Welbourne
05e67fbcab Update to CLDR v38.1, adding Yukon Standard Time
No change to QLocale's data, one addition to the Windows time-zone
data. What was formerly "Us Mountain Standard time / Canada" is now
Yukon Standard Time.

Fixes: QTBUG-89784
Pick-to: 6.0 5.15
Change-Id: I4c9a23620e74ea379be8a4c5ba0896d35fe9b594
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2021-01-27 15:00:57 +01:00
Edward Welbourne
4ab6358039 Reorder locale enums alphabetically
Binary-incompatible change: change the numeric values of QLocale's
Language, Script and Country enums, as encouraged by a comment in the
generator script enumdata.py and clarify documentation around that.

In the process (since I was changing almost every line anyway),
convert the dictionary values from (mutable) lists of length two to
tuples, since they are (and should be) immutable data.

Change-Id: I26222bce45b9f5074b1d81ed70015a75ac34adcd
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2020-11-08 13:01:18 +01:00
Edward Welbourne
48ab30e02a Update util/locale_database/'s README and timezone script instructions
The script told me the wrong path to pass as first argument, so
correct that; and the README didn't mention the need to run it.
CLDR v37 makes no change to the actual generated data, though.
Tweaked wording of a comment in the script.

Task-number: QTBUG-84669
Change-Id: I56b510c666f414d9719cef650aeec6192c4fde6e
Reviewed-by: Andrei Golubev <andrei.golubev@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2020-10-09 14:39:55 +02:00
Dimitrios Apostolou
1e546595e9 Remove unused imports
As found by LGTM.com.

Change-Id: I1704f10f9bab1b11ab22824aca0cfcdcb47fef2f
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2020-07-10 02:36:54 +02:00
Edward Welbourne
c834dbc6fb Move cldr2qtimezone.py's CLDR-reading to a CldrAccess class
This begins the process of replacing xpathlite.py, adding low-level
DOM-access classes to ldml.py and the CldrAccess class to cldr.py

Moved a format comment from cldr2qtimezone.py's doc-string to the
method of CldrAccess that does the actual reading.

Task-number: QTBUG-81344
Change-Id: I46ae3f402f8207ced6d30a1de5cedaeef47b2bcf
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:43:13 +01:00
Edward Welbourne
5b1c33cc78 Rework cldr2qtimezone.py into more maintainable form
Broke out the updating of a source file to a ZoneIdWriter helper
class, which enables tidying away the temporary file if we fail.
Collected up the rest of the script into a main() that's now
called from a __name__ == '__main__' block.
Rationalized the imports.

Eliminated an inefficient lookup function by constructing a suitable
dict() before entering the loop that needed it.

Separated the "data you might need to update" tables from the code
that does the work, to make it easier for those adding support for new
zones to see what they're doing.

Removed the spurious $Revision$ from the output and reworded the
premable of the generated file. (It would seem CLDR no longer uses an
RCS-based version-control system.) Generated output is otherwise
unchanged.

Task-number: QTBUG-81344
Change-Id: I7d9de8357ebcb599d154de9f862e25f7ade00390
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:42:50 +01:00
Edward Welbourne
c3dea1ffca Move some shared code to a localetools module
The time-zone script was importing two functions from the locale data
generation script. Move them to a separate module, to which I'll
shortly add some more shared utilities. Cleaned up some imports in the
process.

Combined qlocalexml2cpp's and xpathlit's error classes into a new
Error class in the new module and made it a bit more like a proper
python error class.

Task-number: QTBUG-81344
Change-Id: Idbe0139ba9aaa2f823b8f7216dee1d2539c18b75
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 19:42:40 +01:00
Edward Welbourne
43f64b4dc8 Update CLDR to v36
Released on October 4th.
Adds Windows names for two time zones, Qyzylorda and Volgograd.
Added languages Chickasaw (cic), Muscogee (mus) and Silesian (szl).

Norwegian number formatting has flipped back to using colon rather
than dot as time separator; it's flipped back and forth over the last
several CLDR releases.  The dot form is present as a variant, the
colon form was long given as the normal pattern, then went away; but
now it's back as a contributed draft and that's what we pick up.

The MS-Win time-zone ID script was iterating a dict, causing random
reshuffling when new entries are added. Fixed that by doing the
critical iteration in sorted order.

Omitted locales ccp_BD and ccp_IN due to QTBUG-69324.

Task-number: QTBUG-79418
Change-Id: I43869ee1810ecc1fe876523947ddcbcddf4e550a
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2019-10-25 11:44:48 +02:00
Edward Welbourne
8bfae093ed Add data for Windows Time-Zone IDs added in the last two years
We've not run util/locale_database/cldr2qtimezone.py for a while, so
CLDR has had time to add several more zones.  Catch up, inserting the
new entries in order.

Change-Id: I8625548b0f7775958230eccbd89b897d7afed9e9
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2019-07-01 17:48:53 +02:00
Edward Welbourne
13242673cf Tidy up in cldr2qtimezone.py and document the need to run it
It wasn't mentioned in cldr2qlocalexml.py's instructions, so I didn't
know to run it.  The data it used in an illustration was out of date.
Two tests could be combined with no loss.

Change-Id: I26e619e6210ea5b1258326fc4bc2b6aee9d6a999
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2019-07-01 17:48:32 +02:00
Edward Welbourne
bbd64f64b2 cldr2qtimezone.py: report all missing zones, rather than just the first
When scanning the CLDR data, the script raised an exception if it
didn't recognize a zone ID.  Instead, collect up such unrecognized IDs
in a list and report them all at the end, so that whoever runs this
can do them all in one go, rather than doing one, running the script,
doing the next, running the script, ad nauseam.

Change-Id: Ia659f1d1c7e1c1b4ccb87cc23828a0588a5bf958
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2019-07-01 17:48:16 +02:00
Edward Welbourne
2f19a3053e Use simpler data structures in cldr2qtimezone.py
Use tuples for the fixed data.  The numbering of rows in the data
tables isn't part of any public API, so we can change it freely; it is
thus unnecessary, as we can just enumerate a tuple of the data values
to generate sequential indices on the fly.  (Updates to the data shall
no longer need to renumber in order to insert entries.)

Restore ordering of the data tables, and remove wanton spacing from
inside parens, in the process.

Change-Id: I59956cfb6191fe729300b57070671b7e66bd0379
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2019-07-01 17:47:51 +02:00
Edward Welbourne
548513a4bd Separate out the time, zone, date code from corelib/tools/
We'll be adding calendar code here as well, and tools/ was getting
rather crowded, so it looks like time to move out a reasonably
coherent sub-bundle of it all.

Change-Id: I7e8030f38c31aa307f519dd918a43fc44baa6aa1
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2019-06-06 15:54:32 +02:00
Edward Welbourne
248b6756da Rename util/locale_database/ to include the e that was missing
It was misnamed local_database, quite missing the point of its name.

Change-Id: I73a4fdf24f53daac12304de1f443636d89afacb2
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
2019-05-20 20:42:10 +02:00