Commit Graph

14 Commits

Author SHA1 Message Date
Edward Welbourne
1c0b69eac5 Only read the first BOM as a BOM; the rest are ZWNBS !
QUtf32::convertToUnicode() was forgetting to set headerdone when it
dealt with the header (for contrast, Utf16::convertToUnicode() does).

Fixes: QTBUG-62011
Change-Id: Ia254782ce0967a6cf9ce0e81eb06d41521150eed
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-02-14 19:13:37 +01:00
Marc Mutz
6b9a1824a4 Extend tst_qstringapisymmetry for member compare()
There were a few surprises:

- QByteArray::compare() are missing noexcept (will add)
- ibid., called with non-ascii content and CaseInsensitive fails
  (this was discussed on the ML, with tentative agreement that
  it's a feature, not a bug; waiting for QUtf8String(View) for a
  fix, then).
- As was the case when we did this exercise with the relational
  operators, QString(Ref)/QChar is not noexcept (will fix)

These have been QEXPECT_FAIL'ed.

Not much of the cartesian product is implemented at all, yet.  These
have been #ifdef'ed with NOT_YET_IMPLEMENTED to see what's still
missing.

Change-Id: I7d9b21e292b98f980aacdc6248e88188f7472ba2
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-02-09 17:43:20 +00:00
Edward Welbourne
41b2c477b7 Take care of NULL data from QStringView in QCollator
Back-ends need to catch NULL data so as not to call system APIs with
invalid pointers.

[ChangeLog][QtCore][QCollator] Fixed a regression introduced in 5.14.0
that caused QCollator not to operate with default-constructed QStrings
and print a warning on Windows.

Fixes: QTBUG-81673
Change-Id: I2eafe1e188b436afcca3cf2ecdf98bba707c44c9
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-02-05 10:23:43 +01:00
Edward Welbourne
71fa90a37c Enable system locale to skip digit-grouping if configured to do so
On macOS it's possible to configure the system locale to not do digit
grouping (separating "thousands", in most western locales); it then
returns an empty string when asked for the grouping character, which
QLocale's system-configuration then ignored, falling back on using the
base UI locale's grouping separator. This could lead to the same
separator being used for decimal and grouping, which should never
happen, least of all when configured to not group at all.

In order to notice when this happens, query() must take care to return
an empty QString (as a QVariant, which is then non-null) when it *has*
a value for the locale property, and that value is empty, as opposed
to a null QVariant when it doesn't find a configured value. The caller
can then distinguish the two cases.

Furthermore, the group and decimal separators need to be distinct, so
we need to take care to avoid cases where the system overrides one
with what the CLDR has given for the other and doesn't over-ride that
other.

Only presently implemented for macOS and MS-Win, since the (other)
Unix implementation of the system locale returns single QChar values
for the numeric tokens - see QTBUG-69324, QTBUG-81053.

Fixes: QTBUG-80459
Change-Id: Ic3fbb0fb86e974604a60781378b09abc13bab15d
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
2020-02-03 15:34:02 +01:00
Giuseppe D'Angelo
577d698b8e QString::isLower/isUpper: redo the implementation
Use QStringIterator rather than indexed loops. This fixes handling of
non-BMP code points (which may be lower or uppercase, see the test).

Change also the semantics of the functions, adopting Unicode §3.13
definitions: a string is lowercase/uppercase if it's equal to its
own toLower/toUpper folding.

As a side effect, empty strings are now correctly reported to be
lowercase AND uppercase.

[ChangeLog][Important Behavior Changes] The semantics of
QString::isLower() and QString::isUpper() have been changed to match the
Unicode specification. Now lowercase (resp. uppercase) strings are
allowed to contain any character; a string is considered lowercase
(resp. uppercase) if it's equal to its own toLower() (resp. toUpper())
folding.  Previously, a non-letter character would make the string not
lowercase nor uppercase, and the mere presence of an uppercase (resp.
lowercase) letter would make isLower() (resp. isUpper()) return false,
even if the letter wouldn't change under case folding.  As a
consequence, now empty strings are lowercase and uppercase.

[ChangeLog][QtCore][QString] Fixed a number of bugs of
QString::isLower() and QString::isUpper(). Empty strings are now
correctly reported to be lowercase (resp. uppercase), and strings
containing code points outside the BMP are now correctly handled.
Note that the behavior of these functions has also been changed.

Change-Id: Iba1398279a072399a9f21295fe75f6e414f3f813
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-01-11 19:28:50 +01:00
Tuomas Heimonen
27d1391280 QLocale: Support Indian number formatting
When QLocale::Country is set to QLocale::India numbers are written so that
after first three from the right and then after every second will be comma.
E.g. 10000000 is written as 1,00,00,000

Task-number: QTBUG-24301
Change-Id: Ic06241c127b0af1824104f94f7e2ce6e2058a070
Reviewed-by: Venugopal Shivashankar <Venugopal.Shivashankar@qt.io>
Reviewed-by: Teemu Holappa <teemu.holappa@qt.io>
2019-12-30 13:04:23 +02:00
Edward Welbourne
4218c3044d Revert "[macOS] Skip test that triggers a buffer overflow in CoreFoundation"
Allegedly Apple has fixed the bug that made this necessary, so we
should be able to include these two test-cases once more.

This reverts commit ba9585bd02.

Fixes: QTBUG-69875
Change-Id: I5ac6019c0d647691eda6cdbb2a53e7471859d4a3
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2019-11-18 10:15:39 +01:00
Edward Welbourne
43f64b4dc8 Update CLDR to v36
Released on October 4th.
Adds Windows names for two time zones, Qyzylorda and Volgograd.
Added languages Chickasaw (cic), Muscogee (mus) and Silesian (szl).

Norwegian number formatting has flipped back to using colon rather
than dot as time separator; it's flipped back and forth over the last
several CLDR releases.  The dot form is present as a variant, the
colon form was long given as the normal pattern, then went away; but
now it's back as a contributed draft and that's what we pick up.

The MS-Win time-zone ID script was iterating a dict, causing random
reshuffling when new entries are added. Fixed that by doing the
critical iteration in sorted order.

Omitted locales ccp_BD and ccp_IN due to QTBUG-69324.

Task-number: QTBUG-79418
Change-Id: I43869ee1810ecc1fe876523947ddcbcddf4e550a
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2019-10-25 11:44:48 +02:00
Edward Welbourne
fd3db1dc3a Don't try to define QT_NO_CAST_TO_ASCII when the test undefines it
The qmake config for tst_QString tried to impose QT_NO_CAST_TO_ASCII
on it, but the source file explicitly #undef-s this symbol and its
friends. Leave the define commented out in the .pro so that a comment
can explain why it's no good.

Change-Id: I7620f4e104f0cdab05fdc246b903c40026e63d76
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2019-10-25 11:41:44 +02:00
Edward Welbourne
946c701883 Move a test for feature ICU from .pro to .cpp
Test QT_CONFIG(icu) in the code instead of testing qtConfig(icu) in
the profile and setting an extra define just to shadow what's already
defined. Also remove the matching define from qcollator.pro, whose
test code didn't use it.

Noticed while reviewing the conversions to CMake.

Change-Id: I19d3b1026b2a8f50ec424c450614e721500fd38a
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2019-10-25 11:41:30 +02:00
Edward Welbourne
61ec1abc08 Suppress deprecation warnings on a test of a deprecated QString method
QString::fromAscii() is deprecated since 5.0 but still tested.
So suppress deprecations for its code.

Change-Id: Ic048a843c43551021da39a16d94c3222201573dc
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
2019-10-09 11:18:22 +02:00
Marc Mutz
c58ca4256d Deprecate QStringViewLiteral
As a macro, we can't directly deprecate it, but need to make it call
something deprecated. That is a new ctor with a new enum type
added. The type might be useful for other such ventures, so put it
into qglobal.h

Remove the QT_NO_UNICODE_LITERAL protection, as it's always false
these days, and QT_UNICODE_LITERAL is unconditionally #defined a 20
lines above.

[ChangeLog][QtCore][QStringView] Deprecated the (undocumented)
QStringViewLiteral macro. Just use u"" or QStringView(u"") instead.

Change-Id: I9141320225037e1bc6b7f920bf01a9d0144fdac2
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2019-08-10 22:13:49 +02:00
Marc Mutz
adab531771 Port from QStringViewLiteral to u""
Now that all our supported compilers know char16_t, we no longer need
QStringViewLiteral, whose only purpose in life was to turn u"" into
L"" for MSVC < 2015.

Change-Id: I25a094fe7992d9d5dbeb4a524d9e99e043dcb8ce
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2019-07-29 11:17:28 +03:00
Edward Welbourne
a9aa206b7b Move text-related code out of corelib/tools/ to corelib/text/
This includes byte array, string, char, unicode, locale, collation and
regular expressions.

Change-Id: I8b125fa52c8c513eb57a0f1298b91910e5a0d786
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2019-07-10 17:05:30 +02:00