Commit Graph

269 Commits

Author SHA1 Message Date
Edward Welbourne
37b4c4d82f Improve and simplify tst_QByteArray's tests
Don't go via QString when we don't need to.
Put expected after actual, not the other way round.
Give tests and test-cases sensible names.
Prefer function-style cast over C-style.

Change-Id: I0b79534a9cc95f2e312a85394693ac674ff3d1d6
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-19 15:06:49 +02:00
Giuseppe D'Angelo
11d1dcc6e2 QString: use the QRegularExpression operations on QStringView
There's no need of duplicating code all over the place; QString can
reuse the implementation of the indexOf/contains/count/lastIndexOf
family of functions already existing for QStringView.

For simplicity, the warning messages (that our autotests actually check)
have been made more generic, rather than introducing some other
parameter (as in, "which class is using this functionality so to emit
a more precise warning"), which would have just complicated things as
the implementation of these functions is exported and used by inline
QStringView member functions.

Change-Id: I85cd94a31c82b00d61341b3058b954749a2d6c6b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-19 08:49:07 +02:00
Giuseppe D'Angelo
f0d1f50e02 QRegularExpression: fix matching over null/empty QString(View)
An empty QString(View) is allowed to have nullptr as its data pointer
(of course, only if its size is 0). This wasn't properly
checked in QRegularExpression, which passed such nullptr to
PCRE, and that resulted in PCRE raising an error (PCRE_ERROR_NULL).
Detect this case and pass a dummy pointer to keep PCRE happy.

Fixing and testing this in turn exposed a problem with QStringView
support in QRegularExpression when used over a null QString: the
code is supposed to use the QStringView(QString) constructor and NOT
qToStringViewIgnoringNull. That's because QRE distinguishes null
and empty subjects; when using qToStringViewIgnoringNull over
a null QString, one gets a non-null QStringView (!). Again, this in
turn exposed a problem with a QRegularExpression autotest that assumed
that a null match could only mean "no match" (instead, it can happen at
position 0 of a null QString(View)).

Change-Id: Ifb3cf14dec42ce76fcdbcb07ea1d80784d52ef65
Pick-to: 6.1 6.2
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-08-19 08:48:52 +02:00
Giuseppe D'Angelo
6cee204d56 QS(V)/QBA(V)/QL1S::lastIndexOf: fix the offset calculations
When trying to fix 0-length matches at the end of a QString,
be83ff65c4 actually introduced a
regression due to how lastIndexOf interprets its `from` parameter.

The "established" (=legacy) interpretation of a negative `from` is that
it is supposed to indicate that we want the last match at offset `from +
size()`. With the default from of -1, that means we want a match
starting at most at position `size() - 1` inclusive, i.e. *at* the last
position in the string. The aforementioned commit changed that, by
allowing a match at position `size()` instead, and this behavioral
change broke code.

The problem the commit tried to fix was that empty matches *are* allowed
to happen at position size(): the last match of regexp // inside the
string "test" is indeed at position 4 (the regexp matches 5 times).

Changing the meaning of negative from to include that last position (in
general: to include position `from+size()+1` as the last valid matching
position, in case of a negative `from`) has unfortunately broken client
code. Therefore, we need to revert it. This patch does that, adapting
the tests as necessary (drive-by: a broken #undef is removed).

Reverting the patch however is not sufficient. What we are facing here
is an historical API mistake that forces the default `from` (-1) to
*skip* the truly last possible match; the mistake is that thre is simply
no way to pass a negative `from` and obtain that match. This means that
the revert will now cause code like this:

  str.lastIndexOf(QRE("")); // `from` defaulted to -1

NOT to return str.size(), which is counter-intuitive and wrong. Other
APIs expose this inconsistency: for instance, using
QRegularExpressionIterator would actually yield a last match at position
str.size(). Similarly, using QString::count would return `str.size()+1`.
Note that, in general, it's still possible for clients to call

  str.lastIndexOf(~~~, str.size())

to get the "truly last" match.

This patch also tries to fix this case ("have our cake and eat it").

First and foremost, a couple of bugs in QByteArray and QString code are
fixed (when dealing with 0-length needles).

Second, a lastIndexOf overload is added. One overload is the "legacy"
one, that will honor the pre-existing semantics of negative `from`. The
new overload does NOT take a `from` parameter at all, and will actually
match from the truly end (by simply calling `lastIndexOf(~~~, size())`
internally).

These overloads are offered for all the existing lastIndexOf()
overloads, not only the ones taking QRE.

This means that code simply using `lastIndexOf` without any `from`
parameter get the "correct" behavior for 0-length matches, and code that
specifies one gets the legacy behavior. Matches of length > 0 are not
affected anyways, as they can't match at position size().

[ChangeLog][Important Behavior Changes] A regression in the behavior of
the lastIndexOf() function on text-related containers and views
(QString, QStringView, QByteArray, QByteArrayView, QLatin1String) has
been fixed, and the behavior made consistent and more in line with
user expectations. When lastIndexOf() is invoked with a negative `from`
position, the last match has now to start at the last character in the
container/view (before, it was at the position *past* the last
character). This makes a difference when using lastIndexOf() with a
needle that has 0 length (for instance an empty string, a regular
expression that can match 0 characters, and so on); any other case is
unaffected. To retrieve the "truly last" match, one can pass a
positive `from` offset to lastIndexOf() (basically, pass `size()` as the
`from` parameter). To make calls such as `text.lastIndexOf(~~~);`, that
do not pass any `from` parameter, behave properly, a new lastIndexOf()
overload has been added to all the text containers/views. This overload
does not take a `from` parameter at all, and will search starting from
one character past the end of the text, therefore returning a correct
result when used with needles that may yield 0-length matches. Client
code may need to be recompiled in order to use this new overload.
Conversely, client code that needs to skip the "truly last" match now
needs to pass -1 as the `from` parameter instead of relying on the
default.

Change-Id: I5e92bdcf1a57c2c3cca97b6adccf0883d00a92e5
Fixes: QTBUG-94215
Pick-to: 6.2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-08-19 01:55:01 +02:00
Edward Welbourne
7b74a65857 tst_QLocale::formatTime(): add some actual locale-dependence
All the tests were using the C locale, so were equivalent to tests of
QTime::toString(). Add a locale column and some test-cases in
preparation for a change to am/pm indicators.

Task-number: QTBUG-95790
Change-Id: I3ad917b7a6f3d3bfe31d6a5a5da596025f173e81
Reviewed-by: Andrei Golubev <andrei.golubev@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-18 21:31:30 +02:00
Mårten Nordheim
4603d7e231 QString/QByteArray::number(double): Test big numbers
Exhausts the entire buffer which double-conversion is left to work with.
Also has a large amount of precision, which apparently we need to store
temporarily.

Task-number: QTBUG-88484
Change-Id: I87e8c323676465f1b8695e086020df1240d0d0d7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-17 03:53:45 +02:00
Edward Welbourne
6f833eff92 Add QByteArrayView::trimmed()
Unlike simplified(), it just moves the end-points, without needing to
modify contents, so it makes sense (as for QStringView and
QLatin1String) to provide it. Moved QByteArray's trimmed() tests to
tst_QByteArrayApiSymmetry so that QBAV can now share them.

[ChangeLog][QtCore][QByteArrayView] Added trimmed().

Change-Id: Ifd7a752adb5f3d3e2ad0aa8220efa7e7d2d39baa
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
2021-08-11 22:32:42 +02:00
Edward Welbourne
d055abd195 Assert that special handling of '0' padding does what it should
When formatting numbers, if the fill character used to left-pad to
field widths is '0', the code delegates that padding to the
QLocaleData's ZeroPadded formatting option. Since we want the zeros
before any minus sign, and don't want to subsequently add more zeros
before it, check that this has worked as expected when calling
replaceArgEscapes(), to confirm that it doesn't need to worry about
that.

Add some tests that verify the expected behavior.

In the process, tidy up the code doing this. Rename a local variable
to match our coding style, split a long line.

Pick-to: 6.2
Change-Id: I7cc430c5bceb006cf4e226bca33da16bd2bb1937
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2021-08-10 19:43:03 +02:00
Mårten Nordheim
9b860ece42 QString/QByteArray: Fix setNum docs and add tests
Amends 260168d9d7

Task-number: QTBUG-53706
Pick-to: 6.2
Change-Id: I01c8cdc6a3cb46ec8e49e15ad71b6d707c0d272f
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-08-06 16:49:08 +02:00
Edward Welbourne
2f7d4f478e tst_QByteArray::qUncompressCorruptedData(): MS-Win no longer hangs
Oliver Wolff reports that this test no longer hangs on Windows; and
the other plafroms for which it was skipped are no longer supported,
so remove the #if-ery that skips this test for platfroms on which
uncompressing corrupt data used to hang.

Change-Id: I94a3fd4b83338fe6e3a97ab055fe05e2f15b6b45
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
2021-08-05 01:07:41 +02:00
Mårten Nordheim
e00928e48c QByteArray::number(double): Extend the test
Move out and share the test data from the QString::number_double() test
and re-use it for this one.

Task-number: QTBUG-88484
Change-Id: I6502d1d360657f6077e5c46636f537ddfdde3a83
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-04 23:45:47 +02:00
Edward Welbourne
826e1963e3 Rework tst_QString::remove_regexp() and its data table
The test was producing a warning about the invalid test, for which
replace_regexp() had anticipated that warning; do the same in
remove_regexp(). The two tests shared a date() method, but the remove
test was a no-op on the tests with non-empty replacement text; move
the column set-up and data rows with empty replacement to remove's
data() function, from replace's, and reverse the direction of calling
each other between data() functions, so each test gets the cases that
are relevant to it and no spurious PASSes happen for no-op tests. In
the process, give moved test-cases informative names; relocate the
(entirely re-written) remove data function to beside its test; and
eliminate a pointless local variable from both tests (it used to be
needed when testing both QRegExp and QRegularExpression).

Pick-to: 6.2
Change-Id: I93dcfc444f984edf5c029f99306aff6bc95d554a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-04 22:22:20 +02:00
Edward Welbourne
a13942791d Refine localeAwareCompare() test-case filtering on ICU
The code to work around setlocale() mis-describing en_US as C ensured
that we didn't accept the C test-cases when the locale was really
en_US; but neglected to accept the en_US test-cases when the locale
really was en_US but was misdescribed as C. This lead to no tests
being run when the locale was en_US.

Tweak the logic of the test filtering to compare the wanted locale
against the system locale both when C is wanted and when it isn't.
Make the skip-messages a little more informative.

Pick-to: 6.2 6.1
Change-Id: I4e072e12819144b2941b87a5f486534047d9a579
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-04 22:22:20 +02:00
Tor Arne Vestbø
bef57b317f testlib: Deprecate QWARN() in favor of qWarning()
The QtTest best practices documentations recommends using output
mechanisms such as qDebug() and qWarning() for diagnostic messages,
and this is also what most of our own tests do.

The QWARN() macro and corresponding internal QTest::qWarn() function
was added when QtTest was first implemented, but was likely meant as
an internal implementation detail, like its cousin QTestLog::info(),
which does not have any corresponding macro.

This theory is backed by our own QtTest self-test (tst_silent)
describing the output from QWARN() as "an internal testlib warning".

The only difference between QWARN() and qWarning(), besides the much
richer feature set of the latter, is that qWarning() will not pass
on file and line number information in release mode, but QWARN() will.
This is an acceptable loss of functionality, considering that the user
can override this behavior by defining QT_MESSAGELOGCONTEXT.

[ChangeLog][QtTest] QWARN() has been deprecated in favor of qWarning()

Pick-to: 6.2
Change-Id: I5a2431ce48c47392244560dd520953b9fc735c85
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-04 19:31:51 +02:00
Edward Welbourne
260168d9d7 QByteArray: don't coerce negative to unsigned for any base
This follows up on commit 98666c8afc,
which did the same for QString. If someone wants to get formatting
suitable to an unsigned value, they can cast the value to that
unsigned type and the correct overload shall pick it up.

[ChangeLog][Important Behavior Changes] QByteArray's formatting of
negative whole numbers to bases other than ten now, like QString's
(since Qt 6.0), formats the absolute value and prepends a minus sign.

Task-number: QTBUG-53706
Pick-to: 6.2
Change-Id: I91fee23d25ac0d5d5bcfcbeccbac1386627c004a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-02 23:40:56 +02:00
Mårten Nordheim
64368bb527 Add a few more cases to the QString::number(double) test
Formatting using scientific notation with extra precision wasn't tested

Change-Id: I7a89a0f3d6468515604e43e52fc366dedf3c39ea
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-07-30 12:00:37 +02:00
Ivan Solovev
08a1bcfa9b QByteArray: extend unit tests
This patch introduces some test improvements to check the calls of
different methods on an empty default-constructed string.
Apart from that, many other tests are added to extend code coverage.

Task-number: QTBUG-91736
Pick-to: 6.2 6.1
Change-Id: If86ef3d8611a678798b1bcc60a1a4f5598fd2179
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-29 15:00:35 +02:00
Ivan Solovev
e150bcfe4d QByteArray: fix indexOf/lastIndexOf
This patch fixed two bugs in indexOf/lastIndexOf:
1. The lastIndexOf(char, qsizetype) overload was crashing with an empty
   QByteArray. It was unconditionally calling lastIndexOfCharHelper()
   which assumes that this QBA is not empty. An explicit check for
   the empty case is added.
2. The indexOf(QByteArray, qsizetype) overload was behaving incorrectly
   while searching for an empty QByteArray. In this case it
   unconditionally returned its second parameter (from). However, from
   can be negative, or even exceed the size of this QByteArray. This
   patch handles this cases properly.

As a drive-by: this patch adjusts the QByteArray::indexOf(char, qsizetype)
and QByteArray::lastIndexOf(char, qsizetype) overloads to match with the
QByteArrayView implementation. This is done to have similar code paths
in both cases and avoid tricky bugs in future.
Ideally we had to adjust the QByteArrayView implementation, but it's
fully inline, so can't be changed without breaking BC.

Task-number: QTBUG-91736
Pick-to: 6.2 6.1
Change-Id: Iaef2fdc5b99cce6aa342cca2d17544a1ad7ca677
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2021-07-29 15:00:35 +02:00
Ivan Solovev
59df5dacd5 Fix QString test compilation without QRegularExpression
The QString itself can be compiled without QRegularExpression, but
the tests do not check if they are supported or not.
This patch fixes the issue by introducing the proper #ifdef guards.

Task-number: QTBUG-91736
Pick-to: 6.2 6.1
Change-Id: I797691f78a34d4f78a86af99c78bf06e26e846d1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-07-29 14:36:49 +02:00
Mårten Nordheim
a52e5689aa QString: extend number(double) formatting test
The number(double) testing done in tst_qstring was a bit lacking,
so other tests (like tst_uic) had to be run to properly test changes.

Task-number: QTBUG-88484
Change-Id: I2fc6cba27788ab4fab6d625257f35868e2b684e3
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-28 00:29:32 +02:00
Ivan Solovev
8ac686e9f4 QString: extend unit tests
This patch introduces some test improvements to check the calls of
different methods on an empty default-constructed string.
Apart from that, some other tests are added to extend code coverage.

As a drive-by:
* fix int -> qsizetype in the test data
* fix int -> enum in the test data

Task-number: QTBUG-91736
Pick-to: 6.2 6.1
Change-Id: I159473b7f5dcbea1bdaf2966979e066296351208
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Andrei Golubev <andrei.golubev@qt.io>
2021-07-22 16:08:33 +02:00
Ievgenii Meshcheriakov
a11d7663a8 tst_qlocale: Add tests for parsing Chakma date and time strings
Add tests to verify parser behavior with locales that use code points
outside of Unicode BMP to represent dates. ccp is used for this because
Chakma language uses code points outside of BMP for both letters and
digits. QDateTimeParser currently is not able to handle this locale
correctly, so the tests are marked as expected failures.

Task-number: QTBUG-87111
Change-Id: I1cc6fe7304b47f19950ae0ad3179c4ffa946adb3
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-19 22:05:54 +02:00
Marc Mutz
af5f91fdf9 tst_qstringapisymmetry: add checks for common overload sets
- QString/QStringView overloads were designed to be compatible for all
  possible argument types, so check that it stays that way

- QString/QAnyStringView overloads have several known ambiguities that
  we cannot and don't want to fix, because it would make
  QAnyStringView less versatile, but at one should at least be able to
  overload QString and weak-QAnyStringView.

Change-Id: I5e5ae3c96060c93bfe070f6c19213328dae9c5f9
Reviewed-by: Giuseppe D'Angelo <giuseppe.dangelo@kdab.com>
2021-06-25 16:20:41 +02:00
Ulf Hermann
e99e07cb5c QString: Respect precision when reading data for %.*s format string
If we disregard the precision we may read a very large string that we
subsequently discard. Furthermore, people use this to read
non-null-terminated strings, which randomly crashes.

Pick-to: 5.15 6.1 6.2
Change-Id: Ifa255dbe71c82d3d4fb46adfef7a9dc74bd40cee
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@gmx.de>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-06-22 22:08:16 +02:00
Edward Welbourne
cc64959602 Tidy up tst_QString::localeAwareCompare()
It was suppressed unconditionally (albeit in different places and with
different levels of obviousness) almost everywhere, due to inability
to set the system locale in use by the implementation. Several
test-cases used ISO-8859-1 encoding on Q_OS_MAC, where the tests were
all suppressed anyway. A block of no_NO tests was #if 0'd out; and
should, in any case, have been nb_NO. Tests of any locale but en_US
were skipped on Q_OS_WIN because we can't set the locale (but
including the tests of en_US tacitly assumed that's the system
locale). If setlocale() failed, for ICU or DARWIN, the test was
skipped; but we might as well check for this in the _data() to save
repetition.

The test was laboriously going through the sign cases; relocate the
QString::compare() test's sign() function so that we can use it here,
too, and simply QCOMPARE() signs. Introduce a TransientLocale class,
copied from tst_QLocale, to take care of setting and restoring the
locale using setlocale(). Change the locale name to a QByteArray so
that we save having to convert it to one in order to pass it to
setlocale(). Since that changed every _data() row, reformat those rows
in the process - most of them were long lines.

On the systems where we can't set the locale used by the function
being tested, condition each block of tests in the _data() on whether
LC_COLLATE looks like the locale to be tested, and report how a
determined developer at least can (by repeatedly running the test with
different locales set) test all the cases; and we'll attempt the ones
that we can, when one of the relevant locales is in use. If that
leaves us with no tests we can do, QSKIP() in the _data() to avoid an
assert failure for "Test data requested, but no testdata available."

Change-Id: I75709fda8827dcbe74f80c4136042054da6fcb13
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2021-06-16 17:42:37 +02:00
Edward Welbourne
cea662550e Turn tst_QString::setRawData()'s QSKIP into QEXPECT_FAIL()s
In commit 287ace562e, part of this test
was suppressed without filing a Jira ticket (or, at least, without
recording it in the QSKIP message). Since it's a known failure, it
should at least be a QEXPECT_FAIL, not a QSKIP. Since only some of the
subsequent parts of the test fail, I used QEXPECT_FAIL(,,Continue) on
each of the failing tests.

Task-number: QTBUG-94450
Change-Id: Iebc6801210c289b4502e59116e71d5901b71aa46
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2021-06-15 16:58:46 +02:00
Edward Welbourne
a57f3b44bf Don't use APPLESS testing for QString
Its collation (and everything locale-related) is only supported during
the lifetime of QApplication.

Change-Id: Ide97795664d45768e66248d47c3b1da83935b0d6
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-06-15 16:58:46 +02:00
Mårten Nordheim
e0ae1af278 QLatin1String: Add a constructor taking QByteArrayView
Change-Id: Ie90645486431d7af3fe8128417b0fb6bd02a88b5
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-06-12 03:08:59 +02:00
Mårten Nordheim
f9b867216b String API Symmetry: test for indexOf with large negative offset
Small change needed to make QString_char16 and QString_QChar return -1
in this case, but other combinations already returns -1.

[ChangeLog][QtCore][Behavior Change] QString::indexOf(QChar) and
QString::indexOf(char16_t) now treat a negative start-position, from,
bigger than the string's size as invalid. It previously
clipped such start-positions to the start of the string, inconsistently
with other QString indexOf overloads.

Change-Id: Ic56c8a558bf40a94845c649647db569892d4df02
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-06-11 22:04:04 +00:00
Edward Welbourne
e6457d9b60 Ensure test using setlocale() doesn't leave it permanently changed
tst_QStringList::sort() wants to use the C locale; but, if it failed,
it left that in force, since it only restored the prior locale on
success. It should also use C.UTF-8, since Qt now wants UTF-8.

Change-Id: If62e3d8da682081bf969075a719d03caebf09233
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2021-06-12 00:03:53 +02:00
Edward Welbourne
421c69fde5 Add .UTF-8 suffixes to locale-names passed to setlocale()
We use UTF-8 locales by default since Qt 6; and relatively few systems
have the encoding-unspecified locales we were trying to use, with the
result that the setlocale() calls all failed.

Task-number: COIN-689
Change-Id: Id791ba269bf4abac29da3daa4fd01684ca9caa7a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2021-06-11 23:30:09 +02:00
Edward Welbourne
81fbcc4dc6 Remove two QEXPECT_FAIL()s that no longer appear to be valid
Because they were followed by a QVERIFY(false), no-one noticed until
now. Dates from 2011, when ICU support was first added. I guess
someone fixed the problem in the intervening decade.

Change-Id: I847816c297156e65397c652767f286bc4de193a2
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-06-11 23:30:08 +02:00
Mårten Nordheim
1c164ec7f2 QLatin1String: Add overloads taking QLatin1Char
Without the overloads using QLatin1Char in QL1S member functions results
in the QL1Char being converted to QChar and QL1String being converted to
QString.

Change-Id: Ic19545539a207f025a6293f0b2d929de475dc166
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-06-11 20:25:05 +02:00
Edward Welbourne
182afbe335 Hangul composition: use < base + count checks, not <= checks
Before Unicode 4.1.0 there was an error in the example code for Hangul
normalization that used <= on the ends of some ranges of values, where
they should have used < tests. This was faithfully copied but the need
for correction has only lately come to light.

Thanks to Ma Lin for pointing this out and providing the fix and
test-cases.

Fixes: QTBUG-71894
Pick-to:  6.2 6.1 5.15
Change-Id: I5c7fec1f9fac1f7a25b2d5e9c3109a90a7ff49e1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-06-11 13:57:19 +02:00
Edward Welbourne
cce3445e70 Always include standard name in QLocale::uiLanguages()
Previously, for locales other than the system locale, no entry was
added to the list for the actual locale whose variants - with and
without likely sub-tags - were being appended. In most cases the
standard name will in fact coincide with the variant without likely
sub-tags, so this was unlikely to cause a problem, but it should be
present regardless.

At the same time, turn tst_QLocale::uiLanguages() into a data-driven
test and add another row to its table.

Change-Id: I5cb2d805d78fc3415d82b169caa6154b0f284708
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2021-06-07 17:14:14 +02:00
Edward Welbourne
b2871765ce Fix augmentation of UI language list
Previously, the augmentation wasn't being applied to the system
locale, due to a mistaken test claiming the locale's name didn't
resemble the string from which it was constructed. The test dates from
before various fixes to likely sub-tag processing that should make it
redundant now. This makes QLocalePrivate::rawName() also redundant
(and its conversion of QLatin1String to QByteArray relied on '\0'
termination which wasn't actually present in the various codes).

Expanded the test of systemLocale() to also test uiLanguages() turns a
single entry into the list we expect; and add two new test-cases.
(The test uses a mock system locale class, making this independent of
the platform backend.)

Fixes: QTBUG-92234
Pick-to: 6.1 5.15
Change-Id: I0cdf6eae152a42dc377f4ea3e62c282ff4be1764
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2021-06-07 17:14:14 +02:00
Mårten Nordheim
a5dc381b4c QByteArrayView: add compare
There was previously no way to compare QByteArrayView to with another
QByteArrayView case-insensitively without allocating memory.

[ChangeLog][QtCore][QByteArrayView] Added compare(), enabling case
sensitive and insensitive comparison with other QByteArrayViews.

Change-Id: I7582cc414563ddbde26da35a568421edcc649f93
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-05-20 17:02:38 +02:00
Edward Welbourne
358df462a0 Fix assertion on matchingLocales(Abhkazian, Any, Any)
CLDR v39 has no locales for Abkhazian, so the locale_index[] entry for
it actually indexes the last entry before the next language up the
enum. This has m_language_id less than Abkhazian.

Change-Id: If8b88f30476a981b3ee00ff8760a46ede0b7aab7
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2021-05-05 16:28:53 +02:00
Andrei Golubev
e6f7202e34 Add more tests for QList/QString/QBA
The major part is stability tests for QList operations,
Also added std::shared_ptr to the Custom type. shared_ptr
accesses the memory which does not directly belong to
QList, so using it inside a passed-to-qlist type is
beneficial (e.g. ASan could catch extra issues)

Basic prepend-aware cases added to QString/QBA tests

Task-number: QTBUG-93019
Change-Id: I50e742bdf10ea9de2de66539a7dbb9abc4352f82
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
(cherry picked from commit adb41bbe00b2b853d4dd26cd9ee77ae5ed541576)
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2021-04-27 14:12:34 +02:00
Edward Welbourne
6235893d54 Update CLDR-derived data to newly-released v39
Routine update with minor changes to locale data, no new languages,
territories or scripts. Various Spanish locales change m_grouping_top
from 1 to 2, reversing a change to a test of Costa Rica's currency
formatting made in commit bb6a73260e.
Includes updates to time-zone IDs.

Fixes: QTBUG-91478
Pick-to: 6.1 6.0 5.15
Change-Id: I78ee161275b3c456c5800a7317a96947c932cf8e
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-04-26 10:59:32 +02:00
Giuseppe D'Angelo
a794c5e287 Unicode: fix the extended grapheme cluster algorithm
UAX #29 in Unicode 11 changed the EGC algorithm to its current form.
Although Qt has upgraded the Unicode tables all the way up to
Unicode 13, the algorithm has never been adapted; in other words,
it has been working by chance for years. Luckily, MOST
of the cases were dealt with correctly, but emoji handling
actually manages to break it.

This commit:

* Adds parsing of emoji-data.txt into the unicode table generator.
  That is necessary to extract the Extended_Pictographic property,
  which is used by the EGC algorithm.

* Regenerates the tables.

* Removes some obsoleted grapheme cluster break properties, and
  adds the ones added in the meanwhile.

* Rewrites the EGC algorithm according to Unicode 13. This is
  done by simplifying a lot the lookup table. Some rules (GB11,
  GB12, GB13) can't be done by the table alone so some hand-rolled
  code is necessary in that case.

* Thanks to these fixes, the complete upstream GraphemeBreakTest
  now passes. Remove the "edited" version that ignored some rows
  (because they were failing).

Change-Id: Iaa07cb2e6d0ab9deac28397f46d9af189d2edf8b
Pick-to: 6.1 6.0 5.15
Fixes: QTBUG-92822
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
2021-04-16 20:31:39 +02:00
JiDe Zhang
50a7eb8cf7 Add the "Territory" enumerated type for QLocale
The use of "Country" is misleading as some entries in the enumeration
are not countries (eg, HongKong), for all that most are. The Unicode
Consortium's Common Locale Data Repository (CLDR, from which QLocale's
data is taken) calls these territories, so introduce territory-based
names and prepare to deprecate the country-based ones in due course.

[ChangeLog][QtCore][QLocale] QLocale now has Territory as an alias for
its Country enumeration, and associated territory-based names to match
its country-named methods, to better match the usage in relevant
standards. The country-based names shall in due course be deprecated
in favor of the territory-based names.

Fixes: QTBUG-91686
Change-Id: Ia1ae1ad7323867016186fb775c9600cd5113aa42
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-04-15 20:17:49 +08:00
Volker Hilsheimer
fa3b672f99 Mark variables that might be unused under some configurations
Silence compiler warnings.

Pick-to: 6.1
Change-Id: I857cf2ee177eecf305e4fa14f6b69b019ce85468
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-04-11 19:53:43 +02:00
Andrei Golubev
c34f51d58c Remove meaningless QT_NO_UNICODE_LITERAL check in QString literal tests
This check doesn't really do anything useful anymore: QStringLiteral
is used in Qt without any extra QT_NO_UNICODE_LITERAL #if-ery

Additionally, clean the related (and outdated) comment in
{QString, QByteArray}::literals()

Change-Id: I65b1eac33c5470508997be24f9ba6cf56d8578ea
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2021-03-30 10:37:13 +01:00
Andrei Golubev
b1377ed02d Add literal operators for QString and QByteArray
[ChangeLog][QtCore][QString] Added literal operator
u"..."_qs that converts a char16_t string literal to QString

[ChangeLog][QtCore][QByteArray] Added literal operator
"..."_qba that converts char string literal to QByteArray

Change-Id: I4aa59b28cc17bff346b378eb70008fb8185d21ac
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
2021-03-30 10:37:07 +01:00
Tor Arne Vestbø
d8158c6c93 macOS: Be honest about the system locale
The system locale of a macOS application is not affected by environment
variables like LANG. Yet, we were reporting a name determined from
environment variables as the fallbackUiLocale(), rather than one based
on the language and country of the actual system locale.

This lead, via the usual CLDR likely-subtag fallback, to claiming the
system locale's name, language, script and country were those obtained
from these environment variables, even when they were at odds with the
actual locale being used by the system, which was being used for some
queries.

Worse yet, any data not supplied by these queries was being obtained
from the same CLDR locale as the name, making for an inconsistent mix
of locale data.

While we cannot avoid the likely-subtag fallback step for fallback
data, it is more consistent to use the actual system locale's name
as start-point for that fallback.

At the same time, add support for the language, script and country
queries, so that the QLocale::system() describes itself faithfully,
instead of claiming to be the locale that results from that fallback.

If we want to support LANG or other environment variable overrides,
they should be handled by the layer above the system locale, by
changing the default locale of the Qt application, as if the user
had called QLocale::setDefault().

[ChangeLog][QtCore][QLocale] QLocale::system() on macOS no longer
pretends to support LANG or other environment variables as overrides,
as this is not a feature that the system locale on macOS supports.
To override the locale of an application, use QLocale::setDefault(),
or pass -AppleLocale en_US.

Fixes: QTBUG-90971
Change-Id: Ibdaf5ff9a2050f61233a88eabf3c29094f7757f1
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-02-11 15:01:45 +01:00
Tor Arne Vestbø
f218d89522 Clarify what QSystemLocale::fallbackLocale() is used for
Change-Id: I8f073f996505ccb42020b32fd76520ecef54e628
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-02-09 13:29:22 +01:00
Edward Welbourne
1e3b0d9280 Purge various BLACKLIST entries that are now fixed
Based on BPASS results from recent test-runs.

Task-number: QTBUG-69131
Fixes: QTBUG-87416
Fixes: QTBUG-87662
Task-number: QTBUG-87663
Fixes: QTBUG-88705
Change-Id: I8ebe014517bfa8d124e86b821650b07c591ea16e
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Reviewed-by: Andrei Golubev <andrei.golubev@qt.io>
2021-02-02 10:40:44 +01:00
Ivan Solovev
fe7cbf8fe6 QRegularExpression: add move constructor
- Add move constructors to QRegularExpression, QRegularExpressionMatch
  and QRegularExpressionMatchIterator.
- Update the documentation to explicitly state that only destructor
  and assignment operators can be called for a moved-from object

Task-number: QTBUG-86634
Change-Id: I06b4f54e300541033a9a18339c97338717a06da0
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Alex Blasche <alexander.blasche@qt.io>
2021-01-26 17:20:01 +01:00
Kai Köhne
e7c028bb9c QLocale: Allow direct conversion from language, country, and script codes
This complements patch a148c7b5d71d244, where languageToCode(),
countryToCode() scriptToCode() methods were introduced, with matching
codeToLanguage(), codeToCountry(), and codeToScript() methods.

This allows us to remove the use of private Qt Core API in Qt Linguist.

[ChangeLog][QtCore][QLocale] Added static codeToLanguage(),
codeToCountry(), codeToScript() methods that convert ISO code strings
to the respective enum values.

Change-Id: If5c0843a718c006ade086a6f74ceb86ac6e0fce4
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-01-25 19:56:59 +01:00