Add more tests and DRY the code using the usual row() lambda trick.
Pick-to: 6.6 6.5
Change-Id: I810fba0ab2a96e740e67392155f9507675fe57ae
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
The nameMatch() function has an implicit precondition that neither
argument is nullptr: it immediately dereferences both arguments.
Prevent the crash by checking for name == nullptr early, before
passing to nameMatch().
Add tests for null and empty.
As a drive-by, make variables in the test const (needed for the
QByteArray to avoid detaching, peer pressure for the others).
Amends a639bcda1e.
Pick-to: 6.6 6.5 6.2
Change-Id: I4a30f6c130310eb701ba7c7251168294489c34db
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
By just setting state to nullptr.
Pick-to: 6.6 6.5
Task-number: QTBUG-105105
Change-Id: I6b4f8fe39f1ba51dcfaf98ce7e42c2acd4c4cf98
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The win32 API doesn't give us much choice. _Some_ code pages have
support for returning some error if we pass a specific flag, but not
all of them.
Anyway, since the code pages might not support all that UTF-16 provides,
we can't reasonably make it error out on characters that cannot be
converted.
So, the most reasonable thing we can handle is a unpaired high surrogate
at the end of a string, assume that the rest of the string was fine, and
that the low surrogate will be provided in the next call.
Pick-to: 6.6 6.5
Fixes: QTBUG-118185
Task-number: QTBUG-105105
Change-Id: I1f193c9d8e04bec769d885d32440c759d9dff0c2
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Both to store and to restore.
Without this a 3 or more octet sequence would cause errors or wrong
output. This can be seen with GB 18030.
Pick-to: 6.6 6.5
Fixes: QTBUG-118318
Task-number: QTBUG-105105
Change-Id: Id1f7f5f2fba4633b9f888add2186f4d8d21b7293
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
A few code pages do not support this flag[0]. It's also deprecated[1]
and is what Windows prefers to generate by default. So let's drop it.
[0] https://learn.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-multibytetowidechar
See note at the end for the dwFlags parameter.
[1] It's mentioned in the header files, but not online...
Pick-to: 6.6 6.5
Task-number: QTBUG-118185
Task-number: QTBUG-105105
Change-Id: I798c387170c73a953be874de139868543b2d775e
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The actual formatting of date-time strings is handled by the calendar
backend, but the code's in qlocale.cpp as it uses some of its tools.
When feature timezone is unavailable, we're stuck (as before) with
using QDateTime::timeZoneAbbreviation(), but when it's available we
can use QTimeZone::displayName() to get the localized form of the
abbreviation and offset string.
Make matching changes in QDTP so that it recognizes these localized
abbreviations. We now have another candidate for what local time might
be called, to add to those that must be checked.
This naturally implied some changes to tests. It turns out ICU
believes en_US uses GMT+1/GMT+2 for CET/CEST. Replace some MS
QEXPECT_FAIL()s by including the non-abbreviations we do in fact use
on MS in the lists of "abbreviations" to accept.
[ChangeLog][QtCore][QLocale] When a datetime format includes the
timezone (or offset), the appropriately localised form is (to the
extent the timezone backend in use supports this) used where,
previously, a haphazard choice of system and C locale was used. This
applies to both serialization and parsing.
Task-number: QTBUG-115158
Change-Id: I04f9c1055c3b9008320bb8b758490287fd8be5cd
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Since the TZ backend resorts to ICU for display names, we got
inconsistent results if they disagree about when transitions happened.
Also, ICU uses the current (or only recent history) names for the
zone, so one currently not doing DST (Africa/Tripoli) doesn't get a
report of its DST name at a historical time when it did DST (but ICU
doesn't know about it). Since the ICU backend, in any case, doesn't
override the displayName(qint64, ...), we can simply use the QTZP base
version instead of overloading in the TZ back-end, so we only delegate
to ICU when it might actually help. This also saves duplicating some
locking and lazy-initialization code.
In the process, turn a unique lock into a scoped lock within a
suitable scope. Also, make a comment more grammatical and less
verbose.
Change-Id: Iaaed1fb6a380be8b4258c43cbc4bacd5784345fb
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
The first/last/sliced API may be what we suggest users use, but the vast
majority of the installed codebase uses left/mid/right because they've
been available since time immemorial.
An additional benefit of this is to make left() and right() available as
inline methods.
Change-Id: Ifeb6206a9fa04424964bfffd1788383817ed906c
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Those ought to have been the original implementation, when they were
added in commit 38096a3d70, for Qt 6.0.
Because these classes are exported, we need to provide the previous only
implementations for MSVC. All other compilers would provide inline or
emit local, out-of-line copies.
Change-Id: Ifeb6206a9fa04424964bfffd178836a2ae56157d
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
The test that needs this bool is using 2013, so test that year for a
match. (Africa/Tunis toyed with DST in 1990, the year used before, but
thought better of it.) In the process, move the initialization to the
member-initialization of the class and make the member const.
Change-Id: Ib87636cdb0b038fad0cdef9fbe49e96f7bf79d1f
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Then we can easily test how fromLocal8Bit() and toLocal8Bit() behave
with different code-pages.
Pick-to: 6.6 6.5
Task-number: QTBUG-118318
Task-number: QTBUG-118185
Task-number: QTBUG-105105
Change-Id: Ib1cd3bccd27d598f4c80915557e332befcd96354
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
[ChangeLog][QtCore][QStringList] Added lastIndexOf() overloads that take
a QString/QStringView/QLatin1StringView and a Qt::CaseSenitivity
parameters. Prior to this calling lastIndexOf() would call the methods
inherited from the base class. This change is source compatible and
existing code should continue to work.
Task-number: QTBUG-116918
Change-Id: Ia50c884c00021bf581c23c12e0e0c22700dae446
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
[ChangeLog][QtCore][QStringList] Added filter(QLatin1StringView)
overload, which is more optimized when searching for a Latin-1 string
literal as no conversion to QString is necessary.
Task-number: QTBUG-116918
Change-Id: Ieb92f4cfd545b070258dbc5c701ddfb2e6f3fc64
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
[ChangeLog][QtCore][QStringList] Added indexOf() overloads that take
QString/QStringView/QLatin1StringView, and a Qt::CaseSensitivity
parameter. Prior to this using QStringList::indexOf() called the methods
inherited from the base class.
Task-number: QTBUG-116918
Change-Id: Ibc42130b6509f6ecfe7de0d6be378f226ae61982
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Now that users can pass a QStringMatcher to do the matching, change the
existing overload to not use QStringMatcher.
Thanks to Giuseppe D'Angelo for the idea of passing a QStringMatcher to
filter instead of using a magic number to decide whether to use
QStringMatcher or not.
Results of running filter() and filter_stringMatcher, times are in msecs
and this was compiled with gcc -O3:
Without With QStringMatcher
list10 0.00022 0.000089
list20 0.00040 0.00014
list30 0.00058 0.00018
list40 0.000770 0.00023
list50 0.00094 0.00027
list70 0.0012 0.00037
list80 0.0014 0.00041
list100 0.0018 0.00050
list300 0.0054 0.0014
list500 0.0091 0.0023
list700 0.012 0.0032
list900 0.016 0.0041
list10000 0.17 0.045
Drive-by change: optimize tst_QStringList::populateList().
[ChangeLog][QtCore][QStringList] Added filter(const QStringMatcher &)
overload, which may be faster for large lists and/or lists with very
long strings.
[ChangeLog][Possible Performance Changes][QtCore][QStringList] Changed
the implementation of filter(QStringView) overload to not use
QStringMatcher by default. Using QStringMatcher adds overhead, so it is
beneficial/faster when searching for a pattern in large lists and/or
lists with long strings, otherwise using plain string comparison is
faster. If using QStringMatcher makes a difference in your code, you can
use the newly added filter(QStringMatcher) overload.
Change-Id: I7bb1262706d673f0ce0d9b7699f03c995ce28677
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The existing pattern() method always returns a QString, which means that
if the matcher was constructed using a QStringView, pattern() would
uncoditionally convert it to a QString.
This is useful to check if a match is exact:
auto pattern = matcher.patternView();
if (pattern.size() == needle.size() && matcher.indexIn(needle) == 0)
....
This may be needed for a later change in QStringList::contains();
regardless of that, this change makes sense on its own.
Change-Id: I49018551dd22a8f88cf6b9f878a5166902a26f58
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
- Initialize QStringList with an initializer_list instead of old style
operator <<()
- Use Qt::StringLiterals more, better readability
- Test CaseSensitivity
Change-Id: If7dde14333d54b8c2f682036634ad94d5f9f9c74
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Implement the missing overload to handle UTF-8 specific data types,
including char8_t (C++20), char, uchar and signed char.
Introduce the helper function 'assign_helper_char8' which handles the
non-contiguous_iterator case. The contiguous_iterator case is already
handled by the QAnyStringView overload.
Include 'qstringconverter.h' at the end of the file, since it can't
be included at the top due to diamond dependency conflicts.
QStringDecoder is an implementation detail we don't want users to
depend on when using assign(it, it). It would be unnatural to not
be able to use a function just because we didn't include an
apparently unrelated header.
[ChangeLog][QtCore][QString] Enabled assign() for UTF-8 data types.
Fixes: QTBUG-114208
Change-Id: Ia39bbb70ca105a6bbf1a131b2533f29a919ff66d
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Not only are we subject to Q and P defines, we're also included in the
unnamed namespace now.
Amends df030e06a8.
Pick-to: 6.6
Change-Id: Ie2f4c9f45d9845d8a26140e0e1214e87b615ff02
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
By first checking if the list has any matches before potentially making
it detach.
Change-Id: I7a42c2910ef6efc45033e562573414a3a9ef972e
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QStaticLatin1StringMatcher is a static templated Latin-1 Boyer-Moore
string matcher which can be case sensitive or not. It should be used
when the needle is known at compile time so there is no run-time
overhead when generating the skip table.
The convenience functions qMakeStaticCaseSensitiveLatin1StringMatcher
and qMakeStaticCaseInsensitiveLatin1StringMatcher should be used to
construct the matcher objects.
Green Hills Optimizing Compilers are currently not supported.
[ChangeLog][QtCore] Added QStaticLatin1StringMatcher, which can be used
to create a static constexpr string matcher for Latin-1 content.
Task-number: QTBUG-100236
Change-Id: I8b8eed1e88e152f29cbf8d36d83e410fafc5ca2c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
By saying what's special about some of them
Pick-to: 6.6 6.5
Change-Id: I17bf2e12a27bf55f621020ddf3819ee9e606847d
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The SizeShift was not taken into account when constructing QASV from
QL1SV. This is not an issue in normal Qt builds, because SizeShift == 0
there.
But in bootstrapped case (and in future Qt 7) SizeShift changes to 2,
and the bug becomes visible.
The added test-cases do not really reveal the issue, because we do
not run tests in bootstrapped builds, but at least they will help
to prevent the issues in Qt 7.
Pick-to: 6.6 6.5 6.2
Change-Id: I337b37b5230323a5357f48fd1c9bf799ca507d52
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QString::fromJsString -> QString::fromEcmaString()
QString::toJsString() -> QString::toEcmaString()
For API naming compatibility with QByteArray::fromEcmaUin8Array()
Pick-to: 6.6
Change-Id: If6e2121e31e630d6728ed24e41d14b763f395aaa
Reviewed-by: Piotr Wierciński <piotr.wiercinski@qt.io>
Reviewed-by: Mikołaj Boc <Mikolaj.Boc@qt.io>
Reviewed-by: Lorn Potter <lorn.potter@gmail.com>
When appending to an empty string or byte array, we optimize and
copy the internal pointer. But if the other string/byte array was
created with fromRawData this might be temporary data on the stack/heap
and might be de-allocated or overwritten before the string/byte array
is used or is forced to make a deep-copy. This would lead to incorrect
data being used.
This is easy to overlook if you plan to append multiple strings
together, potentially supplied through an argument. Upon appending a
second string it would make a full copy, but there might not be a
guarantee for that. So, it's hard for users to avoid this pitfall!
Fixes: QTBUG-115752
Pick-to: 6.6 6.5 6.2
Change-Id: Ia9aa5f463121c2ce2e0e8eee8a6c8612b7297f2b
Reviewed-by: Ahmad Samir <a.samirh78@gmail.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Previously name() has always used underscore and bcp47Name() dash; let
the user chose which one best fits their needs.
[ChangeLog][QtCore][QLocale] QLocale's name() and bcp47Name() now let
the caller chose what separator to use between the tags making up the
name, where there is more than one.
Change-Id: Ia689e6a3fb581b42905e7fb1ae7a7b688244d267
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
It has always returned dash-joined forms of the locale names, and
callers who need an underscore-joined form have been obliged to
replace('-', '_') before using them. Given that everything it adds to
the list comes from QLocaleId methods that accept a separator, it's
trivial to let it offer the same choice to its callers and save them
this hassle.
Amended code in QTranslater and QMimeType to save them that hassle.
[ChangeLog][CoreLib][QLocale] QLocale::uiLanguages() now lets the
caller choose what separator to use between the tags that make up each
locale-identifier in the list returned.
Change-Id: I91fcd0b988d9a64e0e9ad9e851f6cb8c1be8ae50
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Various comments need to continue using the enumdata.py names, as they
associate data with particular enum members, but we can now correctly
use the en.xml versions of their names when we report them, rather
than the enum-friendly names we use in the code. Since this now means
the data may stray outside plain ASCII - it'll be UTF-8-encoded - this
implies replacing the QLatin1StringView()s of the code that formerly
read this data with QString::fromUtf8().
Fixes: QTBUG-94460
Change-Id: Id3b08875a46af58c0555c3e303b0e15a19441509
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
We could already use dashes in some, rather than spaces, and now no
longer need to capitalize each word. This changes the *_name_list[]
entries for affected languages to more closely match what CLDR gives
as their names. It also amends various comments. Added tests for the
QLocale::*ToString() functions to cover the entries changed.
Task-number: QTBUG-94460
Change-Id: I0163795cb282881f15a97be00a5311c1936c3a09
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Test names and output need to be UTF-8 for the XML data formats to not
end up malformed - which would upset Coin's testrunner, when it
validates the XML as part of checking - and the few other uses of
toLatin1() were to ASCII content anyway, so can harmlessly (this being
test code, where the slight performance advantage of Latin-1 doesn't
matter) use toUtf8() as well, for the sake of uniformity.
Use of toLatin1() broke an imminent commit in which some territory,
script and language names depart from ASCII, leading to malformed
UTF-8 when they appear in test-data-row names.
Task-number: QTBUG-94460
Change-Id: Ifb826b1e417ba24fd862b93d24d0e7a38858a17f
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
I'd previously understood CLDR's minimumGroupingDigits to mean the
most significant group must have that many digits. It turns out to
mean only that the first grouping separator doesn't get added unless
the more significant group has this many. Once we have one separator,
more can be added that do isolate a single digit.
In the process, I discover some of the prior arithmetic is incorrect;
it is now fixed. Added some basic testing, amended some existing
tests. In the process, fixed naming of some double validator tests.
Pick-to: 6.6 6.5
Fixes: QTBUG-115740
Change-Id: Ia6ce011ba72e72428b015ca22b97d815ebf751b2
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
There was a gap in its numbering, and the quick brown fix could do
with some competition.
Change-Id: I1283bbb6ba321ae2b65b4459327f2428a45f85cc
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
I.e. concatenating a null byte array and an empty-but-not-null byte
array should result in an empty-but-not-null byte array.
This matches the behavior of QString::append(QString) too.
Fixes: QTBUG-114238
Pick-to: 6.6
Change-Id: Id36d10ee09c08041b7dabda102df48ca6d413d8b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
It relied on an implementation detail of operator+=(), that the latter
wouldn't just use assignement (e.g. if `this` is empty/null).
It also had undefined behavior, when the char array used with
fromRawData() went out of, the nested, scope, the code was pointing to a
dangling stack pointer.
Thanks to Thiago for the explanation in code review.
This ties in with further changes in this series, where append() is
changed to preserve null-ness; there is no way to preserve null-ness in
append() while keeping this unittest passing.
Change-Id: I43b9f60db9ce2d471f359f32bcc48e7b4cfceeab
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Everyone must have this by now. This test was 1193 ms of CMake time.
Since this was a PUBLIC feature, I've left it around with a constant
condition.
Change-Id: Ifbf974a4d10745b099b1fffd177754538bbff245
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Ran the scripts, added the new enum members to docs.
Updated tests:
* Two of the new languages are right-to-left,
* Canada has replaced a silly date format with a sensible one.
Fixes: QTBUG-111550
Change-Id: Ie6f1e6e94477167c9e2b5c67e6518ca0f6a7e7fb
Reviewed-by: Mate Barany <mate.barany@qt.io>
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Nothing prevents client code from calling QLocale::setDefault() before
we ever instantiate QLocale::system() - aside from some quirks that
mean setDefault(), currently, does instantiate QLocale::system() to
force initialization of defaultLocalePrivate - so using defaultIndex()
could set the system QLocalePrivate instance's index incorrectly.
In any case, even if the index is initially set correctly, a
subsequent change to the system locale would change the correct index;
and nothing outside QLocale::system() has access to the instance that
would then be remembering an out-of-date index.
Actually tripping over that inconsistency took some deviousness, but
was possible. The index is (currently) only used for month name
lookups and those special-case, for the Roman-derived calendars, the
system locale, to only use the index if the system locale offers no
name for a month. Meanwhile, updateSystemPrivate() uses the fallback
locale's index for its look-up of which CLDR data to copy into the
fallback QLocaleData for the system locale.
None the less, a non-Roman calendar's lookup will go via the index to
get at the CLDR data for that calendar, thereby exposing the system
locale's index to use; and, sure enough, a setDefault() could lead
that to produce wrong answers.
In QLocale::system() there's a cached QLocalePrivate, whose index we
need to ensure stays in sync with the active system locale. So pass
its &m_index to systemData(), which will now (when passed this) ensure
it's up to date. Since we always have called systemData(), to ensure
it is up to date, we can skip that update in the initialization of the
cached private and use m_index = -1 to let systemData() know when it's
in the initial call, thereby making the static cache constinit.
Amended a test to what proved the issue was present.
Change-Id: I8d7ab5830cf0bbb9265c2af2a1edc9396ddef79f
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
- rvalues, they work due to e.g. the implicit conversion to QString, so
the compiler picks the "const String &" overloads. (This may change by
adding rvalue overloads in a later commit, although that will mean
adding many more operator+() overloads to disambiguate the calls)
- QByteArray + QByteArrayView
- More P and more Q (even though in some cases P and Q are defined as
the same operator in some QStringBuilder unittest "scenarios")
Change-Id: I4e7daecdb6887fb52f45732cd32323af20b1b850
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
The same _data() will be re-used with trim().
Change-Id: Ie9b794b7e8d40552d9cacb71df0f8a151d4348a5
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
