The table was almost sorted by offset - its UTC entry, with offset 0,
was at the front rather than first among the offset 0 entries. The
lookups in it were being done as if the IDs were in space-joined lists
(as for the IANA IDs in the Windows table), splitting on space,
despite the fact that it had separate entries for different IDs at the
same offset (this only arose for offset 0). So actually massage the
input table in python to combine IDs with the same offset using space,
placing UTC first among the offset 0 entries, and ensure the C++ table
is sorted. Regenerated the CLDR data tables using the updated script.
In the process, fix an off-by-one error in the iteration over
space-joined IDs, where the search only advanced to the space, rather
than to just after it. That wasn't a problem before, but now would be.
Change-Id: Ib49c27bac269b557166fa10738c3e396d58456c0
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The former is overtly forbidden by ISO 8601 but should be recognized
due to RFC 3339's use of it. The "Serialising Extended Data About
Times and Events Working Group" (sedate WG) has established how to
resolve this, so document that conclusion and note the consequent
inadvisability of using UTC-00:00 as a zone ID (although it should
still be accepted).
Change-Id: Ib9fbbe6765117bfa9a84e726d0e75f7397a4c813
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QItemDelegate was superseded since Qt4 by QStyledItemDelegate but it
took until Qt6.7 to remove the last occurrences in qtbase.
- remove unused includes / replace with qabstractitemdelegate.h
- replace references in the documentation with QStyledItemDelegate
- adjust the examples and tests to use QStyledItemDelegate
Pick-to: 6.5 6.6
Change-Id: I246755004ce2d01192a726ca0972106c237df0cc
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Now that the "and" is only seen in enumdata.py and comments, we can
s/And/and/ in all the various territory names that used it.
Change-Id: Ic376d5904b6f5ab54931f96230c1dd5b7f357b8d
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Various comments need to continue using the enumdata.py names, as they
associate data with particular enum members, but we can now correctly
use the en.xml versions of their names when we report them, rather
than the enum-friendly names we use in the code. Since this now means
the data may stray outside plain ASCII - it'll be UTF-8-encoded - this
implies replacing the QLatin1StringView()s of the code that formerly
read this data with QString::fromUtf8().
Fixes: QTBUG-94460
Change-Id: Id3b08875a46af58c0555c3e303b0e15a19441509
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
We could already use dashes in some, rather than spaces, and now no
longer need to capitalize each word. This changes the *_name_list[]
entries for affected languages to more closely match what CLDR gives
as their names. It also amends various comments. Added tests for the
QLocale::*ToString() functions to cover the entries changed.
Task-number: QTBUG-94460
Change-Id: I0163795cb282881f15a97be00a5311c1936c3a09
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The former needed the latter's .dupes to do the job, so can now just
take a method as a tool to do the job instead, letting .dupes become
private. In the process refine the munging to free enumdata.py from
having to capitalize each word in its names. This will, in due course,
let us use more natural forms in various comments. This causes no
change to generted data.
Update enumdata.py's introduction doc, mainly to reflect this but also
fixing the out-of-date names (old *_list have long been *_map) and
adding some details to other paragraphs.
Task-number: QTBUG-94460
Change-Id: If195b2e94a53a495fc4f1f216bed07a910439fa7
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Moving it makes it easier to document what it's up to and why, while
leaving __checkEnum() easier to read; and I'm going to need it
elsewhere anyway. This makes no difference to generated data.
Task-number: QTBUG-94460
Change-Id: I684375bc926d5d54928fbf5b5e08978528aef487
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Prefer stand-alone versions of the names when available. This saves
the need for a Han-specific kludge in the check for discrepancies
between our enum names and the en.xml names. Causes no change to
generated locale data.
Pick-to: 6.6 6.5
Change-Id: I162f3107d6ffc1f8b893b206e0b78b61cf7254f6
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
The string data tables have a mix of digit-length tokens, up to
four-digit hex; we can fit 12 of those per line within our margins.
Leave the one-row-per-locale tables as they are, though, despite long
lines.
Change-Id: I655fddecf24133c26d16187b7a5a8fbc25553e07
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
In the Windows zone-ID code, we tokenize() a text extracted from CLDR
data. However, a leading or trailing space (or a repeated internal
space) would then give an empty "IANA ID" for us to match, causing the
empty ID to be mapped to the Windows ID for the entry with the
superfluous space. This was uncovered by an entry with a trailing
space in CLDR v43's data.
Canonicalize spacing in the IANA ID lists extracted from CLDR so as to
ensure this doesn't happen. (We could pass Qt::SkipEmptyParts to the
tokenize() call, but fixing the issue when generating the data is
cheaper and more robust than fixing it at run-time every time it's
consulted.)
Task-number: QTBUG-111550
Change-Id: Ib3883419558d6574141e9ab0bc929ade2d73e020
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Also add a comment to check the locales new additions enable do have
substantial data. Some of those added in the past are more or less
stubs, for all that they're officially present.
Task-number: QTBUG-111550
Change-Id: I04d46ee96303ecec56c056a0deff6a9457b863e9
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Mate Barany <mate.barany@qt.io>
The digit-grouping and fractional-part separators need to be distinct
for parsing to be able to distinguish between two thousand and one vs
two and one thousandth. Thakfully ldml.py asserted this, so caught the
glitch in CLDR v43's data where mn_Mong_MN over-rode mn's decimal, but
not group, and thereby clashed with group. Fortunately the over-ride
is marked as draft="contributed" so we can back out of the collision
and limit the selection to draft="approved" values (but only when
there *is* such a conflict, as plenty of locales have (compatible)
draft data), thereby ignoring the conflicting contribution.
Brought to the attention of cldr-users at:
https://groups.google.com/a/unicode.org/g/cldr-users/c/6kW9kC6fz3g
hopefully that'll lead to a saner resolution at v44.
Task-number: QTBUG-111550
Change-Id: I1332486e60481cb4494446c0c87d89d74bd317d4
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
From CLDR v43, "The parentLocale elements now have an optional
component attribute, with a value of segmentations or
collations. These should be used for inheritance for those respective
elements." Since we aren't extracting collation or segmentation data
for the present, omit these elements from the scan for parentLocale
information.
Task-number: QTBUG-111550
Change-Id: I42871929f539c1852471812801953f2fc8be0e8a
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
The script and territory to exclude from reports about unused ones
were swapped, so we excluded a territory from the script list (which
didn't contain it anyway) and vice versa.
TheTest for whether to report used the non-existend .territories
attribute by mistake for .__territories
Change-Id: I29e9d9f8f34883d7c3a5ac15470d9e7a0366e3db
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Amends commit 9a8b9473d5 - apparently
the enumdata.py entries were tidied up after the data had been
generated, leading to them being inconsistent (and I missed that in
review). That, in turn, meant the next update would have changed the
public API enum members, backwards-incompatibly; so make enumdata.py
consistent with the released public API. We'll be tidying the order up
at Qt 7 in any case.
Task-number: QTBUG-110333
Change-Id: I3eed2924ce8b69deb552e923d9b0dc142c5f3a65
Reviewed-by: Konrad Kujawa <konrad.kujawa@qt.io>
Reviewed-by: Mate Barany <mate.barany@qt.io>
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Pack some of the arrays that contain locale data more tightly. The
AlphaCode struct is a char[4] but always holds only [a-z]{,3} which
could be fit into 16 bits, halving the size of an AlphaCode struct.
With the new constructor the initialization of the AlphaCode struct
also changes - modify qlocalexml2cpp.py to reflect this change and
regenerate the languageCodeList.
Fixes: QTBUG-105050
Change-Id: I2b1e93ab7cc3f2d667bf67b45769b74a15211931
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Revise a comment in ldml.py about Suzhou "digits", since it turns out
they aren't the same as hanidec, which is far from contiguous.
Change-Id: Ia3947dbc5a927772026e55fe197c8ebce2540da2
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
New languages (and one local for each) added with v42
- Haryanvi
- Moksha
- Northern Frisian
- Obolo
- Pijin
- Rajasthani
- Toki Pona
It also appears that Canada has changed its date format. Modify the
relevant test case to reflect this change.
Task-number: QTBUG-110333
Pick-to: 6.5
Change-Id: Ia8975c2866cd54c9e565543d05bacd52f4987909
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
We can easily enough obtain the root of the present source tree using
the value of __file__, so might as well do so.
Change-Id: If14773ac1127278b6018a090c0b376437b9c6eec
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
This commit extends functionality for QLocale::codeToLanguage()
and QLocale::languageToCode() by adding an additional argument
that allows selection of the ISO 639 code-set to consider for
those operations.
The following ISO 639 codes are supported:
* Part 1
* Part 2 bibliographic
* Part 2 terminological
* Part 3
As a result of this change the codeToLanguage() overload without
the additional argument now returns a Language value if it matches
any know code. Previously a valid language was returned only if
the function argument matched the first code defined for that
language from the above list.
[ChangeLog][QtCore][QLocale] Added overloads for codeToLanguage()
and languageToCode() that support specifying which ISO 639 codes
to consider.
Fixes: QTBUG-98129
Change-Id: I4da8a89e2e68a673cf63a621359cded609873fa2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Update the locale generation script to support Kaingang and
Nheengatu languages. These are new in CLDR v40. Regenerate
the locale data.
Task-number: QTBUG-94358
Change-Id: I5195d5161d8c4d9f17129bbcfde39dfd3fcf1cd5
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
FTP is insecure and is not supported by modern browsers anymore.
See also: https://mywiki.wooledge.org/FtpMustDie
Change-Id: Iad65d29912e79a4f3fadb9317bb5d9c5fe9b68d7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
They are not needed. Iterations over the table track their sizes.
The size-of-table constants just needed their -1s removed.
Incidentally use std::size() rather than sizeof(array)/sizeof(element).
Change-Id: Ie20eef9f6f5786d93c10b830a87e006d3c5bcc1a
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Use context manager interface (with statement) to atomically update source
files. This ensures that all files are properly closed and the temporary
file is removed even in case of errors.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I18cd96f1d03e467dea6212f6576a41e65f414ce1
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Using NamedTemporaryFile instead mkstemp + fdopen simplifies the code.
It also makes it easier to switch to using context managers for handling
source file modification.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ibeae840ac6dde3d0b49cd7f985cfa6cd775b7f47
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
pathlib's API is more modern and easier to use than os.path. It
also allows to distinguish between paths and other strings in type
annotations.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ie6d9b4e35596f7f6befa4c9635f4a65ea3b20025
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
arparse is the standard way to parse command line arguments in Python.
It provides help and usage information for free and is easier to extend
than a custom argument parser.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I1e4c9cd914449e083d01932bc871ef10d26f0bc2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Replace most uses of str.format() and string arithmetic by f-strings.
This results in more compact code and the code is easier to read
when using an appropriate editor.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I3409f745b5d0324985cbd5690f5eda8d09b869ca
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The schema is in RelaxNG Compact syntax. It can be used to validate
files produced by the cldr2qlocalexml.py script and also gives an
overview of the file format.
Change-Id: I344978f2201c5e67e236ab580a12ad33262f33cb
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
This is the standard way to call base class methods in Python 3 and
it is shorter than the custom one used now.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ifaff591a46e92148fbf514856109ff794a50c9f7
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Instead of implementing all the intricacies of a cmp for the python
sort-function, support for which is due to be dropped at Python 3 in
any case, implement a much simpler key function that achieves the same
result.
In the process, eliminate the ugly kludge of setting an attribute on a
function to, in effect, communicate with it via a global. Instead,
instantiate a class, that wraps the value previously given to the
attribute and whose instance provides the key-function.
Thanks to Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io> for
pointing out that a key function is the way of the future - and
sorted() is a nicer way to sort.
Pick-to: 6.2
Change-Id: Icf1ed5597fedf420d054fbc860e3e7fc6615875c
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
The ordering function used to sort the locale data generated for
QLocale attempted to sort the default territory for a given language
and script before other territories, but was too tangled for it to be
obvious this is what it was doing. The result turned out to be
non-transitive. Replace with code that implements the same preference
but only applies it where the result is compatible with transitivity.
This leads to a shuffling of the order of the Serbian-language
locales, which sorts the Cyrillic ones before the Latin ones. This is
consistent with my reading of the CLDR data, which fills in Cyrillic
and Serbia for Serbian; Serbian/Cyrillic/Serbia did previously sort
before all other Serbian variants.
Thanks to Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io> for
discovering the non-transitivity.
Pick-to: 6.2
Change-Id: I0ce9f78e620e714f980f32b85b7100ed0f92ad74
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
The behavior of StopIteration in generators was changed in Python 3
(see https://www.python.org/dev/peps/pep-0479/). Not raising that
exception makes it easier to port the code to Python 3.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Iac6e3f6f1e1e8ef3a1a0d89b19d2ac2d186434f5
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
IOError does not have property 'message' in Python 3. Instead of
attempting to access it, just use the string representation of
the exception object. This produces the error message possibly combined
with additional arguments in both Python 2 and Python 3.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Icb198a409e7f80b832e474d8390b770fdeacc6c2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Name 'stem' is undefined inside CalendarDataWriter.write(). The error
was repoted by flake8.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ib816b40d0bde2afd3112da76deee0ce39985693a
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
This way the output is easier to compare between versions.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: If4053c574c4ad200a179b06276bd889f2cb9e1c6
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Output of cldr2qlocalexml.py looks weird without the final new line.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I5d675e475c57cdc8101887c39052007ba0a19857
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
This is not a script that can be run independently.
Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I82a93b9ab37ae759b789058d48e94298ecd29b6f
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
We should at least know when members of QLocale's enums aren't adding
any value, and it may make sense to deprecate the unused ones.
Change-Id: Icf202f81d2a35904c13ccdc202d41985bcb3f2e6
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Change the nomenclature used in the scripts and the QLocaleXML data
format to use "territory" and "territories" in place of "country" and
"countries". Does not change the generated source files.
Change-Id: I4b208d8d01ad2bfc70d289fa6551f7e0355df5ef
Reviewed-by: JiDe Zhang <zhangjide@uniontech.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
These variables provide mappings, not lists, so name them non-deceptively.
Change-Id: Idf15e78ad73790bc86dd8b9d4f248d1c4f73993c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
The only reason cldr.py imported enumdata was so as to pass what it
imported to writer.enumData(); that method might as well do the import
itself.
Change-Id: Ie77dcd29058f926b8cca4deef35837f30505859f
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
