Commit Graph

31 Commits

Author SHA1 Message Date
Edward Welbourne
d853816307 Fix handling of Suzhou numbering system
This only arises when the system locale tells us to use its zero as
our zero digit, since no CLDR locale uses it by default. Adapt an
MS-specific QLocale::system() test to use Suzhou numbering, so as to
test this.

While updating the locale-restoration code to also restore the digits
being set in that test, add restore code for the long time format,
where previously only the short time format was restored. Add a
comment to make it less likely one of those shall be missed in future.

Fixes: QTBUG-85409
Change-Id: I343324bb563ee0e455dfe77d4825bf8c3082ca30
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-07-17 12:19:01 +02:00
Edward Welbourne
bb6a73260e Support digit-grouping correctly
Read three more values from CLDR and add a byte to the bit-fields at
the end of QLocaleData, indicating the three group sizes. This adds
three new parameters to various low-level formatting functions. At the
same time, rename ThousandsGroup to GroupDigits, more faithfully
expressing what this (internal) option means.

This replaces commit 27d1391280 with a
fuller implementation that handles digit-grouping in any of the ways
that CLDR supports. The formerly "Indian" formatting now also applies
to at least some locales for Bangladesh, Bhutan and Sri Lanka.

Fixed Costa Rica currency formatting test that wrongly put a separator
after the first digit; the locale (in common with several Spanish
locales) requires at least two digits before the first separator.

[ChangeLog][QtCore][Important Behavior Changes] Some locales require
more than one digit before the first grouping separator; others use
group sizes other than three. The latter was partially supported (only
for India) at 5.15 but is now systematically supported; the former is
now also supported.

Task-number: QTBUG-24301
Fixes: QTBUG-81050
Change-Id: I4ea4e331f3254d1f34801cddf51f3c65d3815573
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-07-14 14:52:08 +02:00
Edward Welbourne
3f8eae848e Fix floating-point 'g'-format's choice between 'e' and 'f' forms
During review of a refactor (coming shortly), Thiago wondered what the
magic numbers were. On closer examination, I concluded that they were
wrong and wrote some tests to prove it. This commit adds those tests;
replaces the misguided old code with something that passes them; and
documents the reasons for the various parts of its decisions.

In the process, tidy up QLocaleData::doubleToString() somewhat and
rename some of its variables to conform to Qt coding style.

Change-Id: Ibee43659b1bdb0707639cdb444cfe941c31d409f
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-07-14 14:52:08 +02:00
Lars Knoll
ecacb3c62f Port QLocale::quoteString from QStringRef to QStringView
Task-number: QTBUG-84319
Change-Id: I29726ebfda7a5f51a0a6ee29e905b0b904256c8d
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-06-11 02:50:16 +02:00
Oliver Wolff
45b0f1be68 Remove winrt
Macros and the await helper function from qfunctions_winrt(_p).h are
needed in other Qt modules which use UWP APIs on desktop windows.

Task-number: QTBUG-84434
Change-Id: Ice09c11436ad151c17bdccd2c7defadd08c13925
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2020-06-06 20:25:49 +02:00
Edward Welbourne
449b347644 Rework QLocale and QDateTime tests that expect CET abbreviation
The name CET is locale-dependent; but QLocale doesn't know about
localization of time zone names. Such abbreviated zone names are, in
any case, potentially ambiguous - various zones around the world have
collisions - so they can't be relied on.

QTimeZone's various backends have differing handlings of how to
abbreviate zone names (MS's provides no abbreviated names at all); and
it appears macOS actually follows the relevant localizations.

So it is hopeless to hard-code the expected zone abbreviations.
Changed the tests to consult QTimeZone for the abbreviation and
compare what it gets with the results of checks which should match
this. This is less stringent, but it is at least robustly correct,
thereby getting rid of assorted kludges and #if-ery.

Pick-to: 5.15
Task-number: QTBUG-70149
Change-Id: I0c565de3fd8b5987c8f5a3f785ebd8f6e941e055
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2020-05-28 15:15:17 +02:00
Lars Knoll
2b8863a98c Get rid of some QTextCodec leftovers
There's no real dependency to QTextCodec in those files anymore.

Change-Id: Ifaf19ab554fd108fa26095db4e2bd4a3e9ea427f
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:51:20 +02:00
Andrei Golubev
b428e98052 Make qtbase compile without QT_CONFIG(timezone)
Fixes: QTBUG-83795
Pick-to: 5.15
Change-Id: I05eaaf57d87a9111d3609ebab81bc707f8af98f0
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2020-05-11 14:42:00 +03:00
Edward Welbourne
48e5a37536 Move a test of QLocale's methods out of tst_QDateTime
It clearly belonged in tst_QDate::toDateTime(), for which it adds a
few more test-case and in which it inspires some further testing.
The new testing of case-insensitivity doesn't work if the format
contains stray non-format characters, so added a new data column to
take care of that.

Pick-to: 5.15
Change-Id: I73619be02091c97024a84cb963c7029e9fd0569a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-08 09:50:24 +02:00
Sona Kurazyan
7e1dacc27a Port qtbase/tests/auto/corelib/text tests to CMake
Task-number: QTBUG-78220
Change-Id: I497da6ed489854bdee5a1ead9a3f34118c78d001
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-27 14:34:51 +02:00
Edward Welbourne
300aaec2f9 Fix digit grouping when digits are surrogat pairs
This is a follow-up to commit ed2b110b6a
to fix indexing errors. Added the test that should have accompanied
that commit, which found some bugs, and refined the Indian number
formatting test (on which it's based).

Make variable i in the loops that insert grouping characters in a
number be consistently a *character* offset - which, when each digit
is a surrogate pair, isn't the same as an index into the
QString. Apply the needed scaling when indexing with it, not when
setting it or decrementing it. Don't assume the separator has the same
width as a digit.

Differences in index no longer give the number of digits between two
points in a string, so actively track how many digits we've seen in a
group when converting a numeric string to the C locale. Partially
cleaned up the code for that in the process (more shall follow when I
sort out digit grouping properly, without special-casing India).

Change-Id: I13d0f24efa26e599dfefb5733e062088fa56d375
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-04-17 13:43:50 +02:00
Qt Forward Merge Bot
8823bb8d30 Merge remote-tracking branch 'origin/5.15' into dev
Conflicts:
	examples/opengl/doc/src/cube.qdoc
	src/corelib/global/qlibraryinfo.cpp
	src/corelib/text/qbytearray_p.h
	src/corelib/text/qlocale_data_p.h
	src/corelib/time/qhijricalendar_data_p.h
	src/corelib/time/qjalalicalendar_data_p.h
	src/corelib/time/qromancalendar_data_p.h
	src/network/ssl/qsslcertificate.h
	src/widgets/doc/src/graphicsview.qdoc
	src/widgets/widgets/qcombobox.cpp
	src/widgets/widgets/qcombobox.h
	tests/auto/corelib/tools/qscopeguard/tst_qscopeguard.cpp
	tests/auto/widgets/widgets/qcombobox/tst_qcombobox.cpp
	tests/benchmarks/corelib/io/qdiriterator/qdiriterator.pro
	tests/manual/diaglib/debugproxystyle.cpp
	tests/manual/diaglib/qwidgetdump.cpp
	tests/manual/diaglib/qwindowdump.cpp
	tests/manual/diaglib/textdump.cpp
	util/locale_database/cldr2qlocalexml.py
	util/locale_database/qlocalexml.py
	util/locale_database/qlocalexml2cpp.py

Resolution of util/locale_database/ are based on:
https://codereview.qt-project.org/c/qt/qtbase/+/294250
and src/corelib/{text,time}/*_data_p.h were then regenerated by
running those scripts.

Updated CMakeLists.txt in each of
	tests/auto/corelib/serialization/qcborstreamreader/
	tests/auto/corelib/serialization/qcborvalue/
	tests/auto/gui/kernel/
and generated new ones in each of
	tests/auto/gui/kernel/qaddpostroutine/
	tests/auto/gui/kernel/qhighdpiscaling/
	tests/libfuzzer/corelib/text/qregularexpression/optimize/
	tests/libfuzzer/gui/painting/qcolorspace/fromiccprofile/
	tests/libfuzzer/gui/text/qtextdocument/sethtml/
	tests/libfuzzer/gui/text/qtextdocument/setmarkdown/
	tests/libfuzzer/gui/text/qtextlayout/beginlayout/
by running util/cmake/pro2cmake.py on their changed .pro files.

Changed target name in
	tests/auto/gui/kernel/qaction/qaction.pro
	tests/auto/gui/kernel/qaction/qactiongroup.pro
	tests/auto/gui/kernel/qshortcut/qshortcut.pro
to ensure unique target names for CMake

Changed tst_QComboBox::currentIndex to not test the
currentIndexChanged(QString), as that one does not exist in Qt 6
anymore.

Change-Id: I9a85705484855ae1dc874a81f49d27a50b0dcff7
2020-04-08 20:11:39 +02:00
Edward Welbourne
0c2d6c163f Fix deprecation warning in tst_QLocale()'s use of QProcess::start()
Change-Id: I6f5dfa2d40984f86670288bdee4d1b7b060850ac
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-04-03 15:13:23 +02:00
Edward Welbourne
89bd12b9ad Change QLocale to use CLDR's accounting formats for currencies
In particular, this changed the US currency formats for negative
amounts to be parenthesised versions of the positive amount forms,
rather than having a minus sign after the $ sign. Test updated.

[ChangeLog][QtCore][QLocale] Currency formats are now based on CLDR's
accounting formats, where they were previously mostly based (more or
less by accident) on standard formats. In particular, this now means
negative currency formats are specified, where available, where they
(mostly) were not previously.

Task-number: QTBUG-79902
Change-Id: Ie0c07515ece8bd518a74a6956bf97ca85e9894eb
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2020-04-02 20:43:34 +02:00
Lars Knoll
66f06a930d Make QLocale(QString) explicit
We should not implicitly convert a QString to a QLocale object. It can
easily create unwanted side effects.

Change-Id: I7bd9b4a4e4512c0e60176ee4d241d172f00fdc32
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2020-04-01 09:29:26 +01:00
Qt Forward Merge Bot
18b69ae8a4 Merge remote-tracking branch 'origin/5.15' into dev
Change-Id: Ia79c2457f20f3428ef1b4358c1094e8dc1bbc33e
2020-03-18 11:45:08 +01:00
Cristian Adam
7150b07041 QLocale: Fix test on MinGW 8.1.0 32 bit
The call of _control87 would crash because of the previous test.

Change-Id: I254efe9c2e9892a473a02663e5ff7016791d5d6d
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@qt.io>
2020-03-17 15:35:34 +01:00
Edward Welbourne
ed2b110b6a Allow surrogate pairs for various "single character" locale data
Extract the character in its proper unicode form and encode it in a
new single_character_data table of locale data. Record each entry as
the range within that table that encodes it. Also added an assertion
in the generator script to check that the digits CLDR gives us are a
contiguous sequence in increasing order, as has been assumed by the
C++ code for some time. Lots of number-formatting code now has to take
account of how wide the digits are.

This leaves nowhere for updateSystemPrivate() to record values read
from sys_locale->query(), so we must always consult that function when
accessing these members of the systemData() object. Various internal
users of these single-character fields need the system-or-CLDR value
rather than the raw CLDR value, so move QLocalePrivate's methods to
supply them down to QLocaleData and ensure they check for system
values, where appropriate first.

This allows us to finally support the Chakma language and script, for
whose number system UTF-16 needs surrogate pairs.

Costs 10.8 kB in added data, much of it due to adding two new locales
that need surrogates to represent digits.

[ChangeLog][QtCore][QLocale] Various QLocale methods that returned
single QChar values now return QString values to accommodate those
locales which need a surrogate pair to represent the (single
character) return value.

Fixes: QTBUG-69324
Fixes: QTBUG-81053
Change-Id: I481722d6f5ee266164f09031679a851dfa6e7839
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-02-17 14:55:24 +01:00
Liang Qi
6b2535ea15 Merge remote-tracking branch 'origin/5.15' into dev
Conflicts:
	examples/widgets/graphicsview/boxes/scene.h
	src/corelib/Qt5CoreMacros.cmake
	src/corelib/Qt6CoreMacros.cmake
	src/network/ssl/qsslsocket.cpp
	src/network/ssl/qsslsocket.h
	src/platformsupport/fontdatabases/windows/qwindowsfontenginedirectwrite.cpp
	src/testlib/CMakeLists.txt
	src/testlib/.prev_CMakeLists.txt
	tests/auto/corelib/tools/qscopeguard/tst_qscopeguard.cpp

Disabled building manual tests with CMake for now, because qmake
doesn't do it, and it confuses people.

Done-With: Alexandru Croitor <alexandru.croitor@qt.io>
Done-With: Volker Hilsheimer <volker.hilsheimer@qt.io>
Change-Id: I865ae347bd01f4e59f16d007b66d175a52f1f152
2020-02-13 18:31:40 +01:00
Edward Welbourne
26f6aa3e50 Configure the MS-Win long time format rather than assuming we know it
In tst_QLocale::windowsDefaultLocale(), we configure the long and
short date formats and the short time format eccentrically, then
verify that QLocale::system() does actually get these eccentric
formats. However, we did not configure the long time format (whose
MS-API name doesn't match that of the other formats), so had to rely
on a guess at the system locale's format. That, however, is not
robust; so now configure the long time format, too.

Removed a duplicated test, at the same time.

Fixes: QTBUG-36306
Change-Id: I04dc22c7eb1b58af55412b598873868f79e9c74f
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-02-06 12:56:58 +01:00
Qt Forward Merge Bot
97417e8f28 Merge remote-tracking branch 'origin/5.14' into 5.15
Conflicts:
	.qmake.conf
	examples/widgets/widgets/imageviewer/imageviewer.cpp
	src/corelib/text/qchar.cpp
	src/corelib/time/qdatetime.cpp

Change-Id: I9762f5c4ff650799219729d6aee79ac07ce9024a
2020-02-04 10:44:00 +01:00
Edward Welbourne
71fa90a37c Enable system locale to skip digit-grouping if configured to do so
On macOS it's possible to configure the system locale to not do digit
grouping (separating "thousands", in most western locales); it then
returns an empty string when asked for the grouping character, which
QLocale's system-configuration then ignored, falling back on using the
base UI locale's grouping separator. This could lead to the same
separator being used for decimal and grouping, which should never
happen, least of all when configured to not group at all.

In order to notice when this happens, query() must take care to return
an empty QString (as a QVariant, which is then non-null) when it *has*
a value for the locale property, and that value is empty, as opposed
to a null QVariant when it doesn't find a configured value. The caller
can then distinguish the two cases.

Furthermore, the group and decimal separators need to be distinct, so
we need to take care to avoid cases where the system overrides one
with what the CLDR has given for the other and doesn't over-ride that
other.

Only presently implemented for macOS and MS-Win, since the (other)
Unix implementation of the system locale returns single QChar values
for the numeric tokens - see QTBUG-69324, QTBUG-81053.

Fixes: QTBUG-80459
Change-Id: Ic3fbb0fb86e974604a60781378b09abc13bab15d
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
2020-02-03 15:34:02 +01:00
Edward Welbourne
264ed73052 Move old tst_QLocale::negativeZero() to tst_QString::number()
It was more complex than it needed to be and was a test of QString,
not of QLocale. This leaves tst_QLocale::negativeZero() available to
now test how QLocale handles negative zero.

Change-Id: Ic9aae250c29f579e6d60fba8404b38673a3b489f
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-01-30 17:57:40 +01:00
Edward Welbourne
d5bb8d5150 Preserve the case of the exponent separator CLDR supplies
We have long (since 4.5.1) coerced it to lower-case, for no readily
apparent, much less documented, reason. CLDR says most locales use an
upper-case E for this - let's actually use what CLDR says we should
use.

The code that matches the exponent separator was doing so
case-insensitively in any case; that needed adaptation now that the
separator's case isn't pre-determined; and, in any case, should have
been done using case-folding rather than upper-casing. In the process,
removed some spurious checks for "'e' or 'E'" in the result, since the
exponent separator is always represented by 'e' (and an 'e' might also
be present for the separate reason of its use as a beyond-decimal
digit representing fourteen).

[ChangeLog][QtCore][QLocale] QLocale::exponential() now preserves the
case of the CLDR source, where previously it was lower-cased.

Change-Id: Ic9ac02136cff79cb9f136d72141b5dbf54d9e0a6
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-01-30 17:56:14 +01:00
Qt Forward Merge Bot
51df9bd251 Merge remote-tracking branch 'origin/5.14' into 5.15
Change-Id: I7b6e6c687d8d60b4a54e6b9dada025ef66c53d96
2020-01-02 01:00:07 +01:00
Tuomas Heimonen
27d1391280 QLocale: Support Indian number formatting
When QLocale::Country is set to QLocale::India numbers are written so that
after first three from the right and then after every second will be comma.
E.g. 10000000 is written as 1,00,00,000

Task-number: QTBUG-24301
Change-Id: Ic06241c127b0af1824104f94f7e2ce6e2058a070
Reviewed-by: Venugopal Shivashankar <Venugopal.Shivashankar@qt.io>
Reviewed-by: Teemu Holappa <teemu.holappa@qt.io>
2019-12-30 13:04:23 +02:00
Allan Sandfeld Jensen
af2daafde7 Deprecate constructing QFlags from a pointer
This was used to support QFlags f = 0 initialization, but with 0 used
as a pointer literal now considered bad form, it had been changed many
places to QFlags f = nullptr, which is meaningless and confusing.

Change-Id: I4bc592151c255dc5cab1a232615caecc520f02e8
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2019-11-20 19:43:38 +01:00
Edward Welbourne
4218c3044d Revert "[macOS] Skip test that triggers a buffer overflow in CoreFoundation"
Allegedly Apple has fixed the bug that made this necessary, so we
should be able to include these two test-cases once more.

This reverts commit ba9585bd02.

Fixes: QTBUG-69875
Change-Id: I5ac6019c0d647691eda6cdbb2a53e7471859d4a3
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2019-11-18 10:15:39 +01:00
Edward Welbourne
43f64b4dc8 Update CLDR to v36
Released on October 4th.
Adds Windows names for two time zones, Qyzylorda and Volgograd.
Added languages Chickasaw (cic), Muscogee (mus) and Silesian (szl).

Norwegian number formatting has flipped back to using colon rather
than dot as time separator; it's flipped back and forth over the last
several CLDR releases.  The dot form is present as a variant, the
colon form was long given as the normal pattern, then went away; but
now it's back as a contributed draft and that's what we pick up.

The MS-Win time-zone ID script was iterating a dict, causing random
reshuffling when new entries are added. Fixed that by doing the
critical iteration in sorted order.

Omitted locales ccp_BD and ccp_IN due to QTBUG-69324.

Task-number: QTBUG-79418
Change-Id: I43869ee1810ecc1fe876523947ddcbcddf4e550a
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2019-10-25 11:44:48 +02:00
Marc Mutz
adab531771 Port from QStringViewLiteral to u""
Now that all our supported compilers know char16_t, we no longer need
QStringViewLiteral, whose only purpose in life was to turn u"" into
L"" for MSVC < 2015.

Change-Id: I25a094fe7992d9d5dbeb4a524d9e99e043dcb8ce
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2019-07-29 11:17:28 +03:00
Edward Welbourne
a9aa206b7b Move text-related code out of corelib/tools/ to corelib/text/
This includes byte array, string, char, unicode, locale, collation and
regular expressions.

Change-Id: I8b125fa52c8c513eb57a0f1298b91910e5a0d786
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2019-07-10 17:05:30 +02:00