For most territories, if we specify only that territory, we should get
a locale specific to that territory. There are exceptions for various
reasons, but check that it's true in most cases, at least. In the
process, convert two QVERIFY(... == ...) into QCOMPARE(..., ...)
Task-number: QTBUG-64940
Change-Id: I7590f20f37b0b459aafb3d1d08f6eb77932fa027
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
tst_qstringapisymmetry needs to use the same collation testability
criteria as tst_qcollator, that is, the locales that may be tested
without ICU and not on Mac nor Windows are only the system locale and
C locale.
Task-number: QTBUG-109954
Change-Id: I69f19ae28b3a16b3827c1eee62ae59fcfdf45209
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
qcollator_posix.cpp is included if ICU is not used and the OS is not
Win nor macOS. Reflect that fact in tst_qcollator instead of using
alternative means which breaks with new platforms that use the
posix collator.
Task-number: QTBUG-109954
Change-Id: I592500ce9626efbcc9377cecf6641967f978c6da
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
There are numerous conflicting symbols in the tst_qstringbuilder\d
variants when batching those together. Remove the linkage from symbols
by putting the common include stringbuilder.cpp in an unnamed
namespace.
Task-number: QTBUG-109954
Change-Id: Ic2a745795b57482c90c9def7667a1145cdb19854
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Not all locales use ASCII hyphen-minus U+002D as minus sign. On macOS
using the nb_NO locale the U+2212 character is used instead when
displaying negative years. Verify that one of the two characters is
found.
Fixes: QTBUG-109853
Change-Id: I424539cc8d427ac199b4528e44bef98e45312d07
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
With the methods that use helpers from qstring.cpp defined in the
latter.
Change-Id: I11d6b0bfb95efe34e56d33d2ecbfe8f4423a9e6c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Turns out we don't support QStringView/QUtf8StringView comparison, and
the only reason the corresponding test succeeded was because it
contained a typo (QStringView instead of QUtf8StringView).
Fix the typo and disable the now-failing test.
Pick-to: 6.5 6.4 6.2
Change-Id: I2210a247aac66743851e53578172a563ee1e96f7
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QtBase didn't contain any checks for QT_RESTRICTED_CAST_FROM_ASCII, so
a recent addition to the QString::append/insert/prepend overload set
made calls with C string literal arguments ambiguous without the CI
noticing. We had a similar problem with QString::multiArg.
To increase test coverage, we now run tst_qstring two times:
- without any define
- with QT_RESTRICTED_CAST_FROM_ASCII (lots of changes necessary)
Most removals are expected, because they disable tests that check the
implicit conversions from QByteArray and const char*, but the
relational operators with QLatin1String objects might warrant fixing.
In some places, when the conversion wasn't the functionality under
test, replaced C string literals or QByteArrays with QLatin1String.
We should also test with QT_NO_CAST_FROM_ASCII, but that's even larger
surgery.
QString doesn't have a ctor from std::nullptr_t, so QString s =
nullptr; doesn't compile in C++17 mode, but does in C++20 mode, due to
the const char8_t* ctor.
Pick-to: 6.5 6.4 6.2 5.15
Change-Id: I0c5a31719a4b8dd585dd748e0ca0d99964866064
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: hjk <hjk@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
... in lieu of <cctype>'s toupper(), which is locale-dependent, and
out-of-line.
The code doesn't run into the toupper(i) issue in the Türkiye locale,
because we don't run tests in that locale and because 'i' is not a
valid format specifier, but don't let the next reader of the code
guess when the use of toAsciiUpper() provides unambiguous guidance.
Task-number: QTBUG-109235
Pick-to: 6.5
Change-Id: I8988f5190441e1ae5cb57370952cda70ca6bb658
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
tests/auto/corelib/text/qbytearrayapisymmetry/tst_qbytearrayapisymmetry.cpp:1174:80:
warning: overflow in expression; result is -9223372036854775808 with
type 'long long' [-Winteger-overflow]
const qlonglong longMaxPlusOne =
static_cast<qlonglong>(Bounds::max()) + 1;
tests/auto/corelib/text/qbytearrayapisymmetry/tst_qbytearrayapisymmetry.cpp:1175:81:
warning: overflow in expression; result is 9223372036854775807 with type
'long long' [-Winteger-overflow]
const qlonglong longMinMinusOne =
static_cast<qlonglong>(Bounds::min()) - 1;
I usually build with GCC, but building with Clang for clazy-standalone,
so I saw these two warnings 500+ times, enough already. :)
Change-Id: Idd86af568ffe89ae49b2a3f9bbeedf312de5e631
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This saves (mostly in corelib/time/) some complications that used to
arise from needing different code-paths for different time-specs.
Task-number: QTBUG-108199
Change-Id: I5dbd09859fce7599f1ba761f8a0bfc4633d0bef9
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Add the full set of substringing operations:
- mid/left/right (old-style)
- sliced/first/last (new style)
- chop/chopped/truncate
The implementation is copied from QUtf8StringView, adjusted to use
sliced() instead of the (ptr, n) ctor, so we need to deal with the tag
twiddling only once, in sliced().
The documentation is also copied from QUtf8StringView.
[ChangeLog][QtCore][QAnyStringView] Added substring functions
sliced(), first(), last(), chop()/chopped(), truncate().
Change-Id: Ief454e9694519e97d9146fa84bc05dda1dded046
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Mate Barany <mate.barany@qt.io>
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Same fix as in tst_qbytearray's QCOMPARE() in
cb9715557c.
Pick-to: 6.4 6.2
Change-Id: I2222d9015ae7121a2fbcf5b936b27de20e873064
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
For both the [4, 7] and [8,15] length cases, we can perform the same
technique: perform two overlapped loads, zero-extend, then perform two
overlapped stores. The 8-character case could be done in a single
load/store pair, but is not worth the extra conditionals. And it should
have the exact same performance numbers whether we use non-overlapping
4-character operations or completely-overlapping 8-character ones (I
*think* the full overlap is actually better).
The 4-character operation is new in this commit. That reduces the
non-vectorized, unrolled to at most 3 characters.
Change-Id: Ib42b3adc93bf4d43bd55fffd16c257ada774236a
Reviewed-by: Lars Knoll <lars@knoll.priv.no>
Because they are too convenient to leave out.
Change-Id: I844cfb794ce0f575c2c65075d9051b0b878a434f
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Overloading insert is a bit tricky since the size might change after
the conversion so either the tail has to be moved twice or a temporary
buffer is needed. For now, add an ineffective but simple overload as in
the case of the const char *s overload, and do the performance
optimization in a follow-up task (QTBUG-108546).
[ChangeLog][QtCore][QString] Added insert(QUtf8StringView) overload.
Task-number: QTBUG-103302
Change-Id: If01c216ff626da29abb43eb68d4de82824f3bfba
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The += operator is already overloaded to handle QStringView and
QLatin1String - add the missing QUtf8StringView overload.
[ChangeLog][QtCore][QString] Added operator+=(QUtf8StringView)
overload.
Task-number: QTBUG-103302
Change-Id: Iec6940bad7866310c826a130b98accebc3c82aa8
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Add the missing overload, among other things it is needed to
implement QTBUG-103302.
[ChangeLog][QtCore][QString] Added append(QUtf8StringView)
overload.
Task-number: QTBUG-103302
Change-Id: I576f73c1919e3a1f1a315d0f82c708e835686eb1
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Also add optimizations for more string comparisons and add tests and
benchmarks.
[ChangeLog][QtCore][QString] Added utf-8 case-insensitive comparisons
Fixes: QTBUG-100235
Change-Id: I7c0809c6d80c00e9a5d0e8ac3ebb045cf7004a30
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
During the implementation of QString::append(QUtf8StringView) it has
become apparent that the testing is insufficient as it did not warn
about an extra growth. The following tests have been added that append:
- y-umlaut and greek letter small theta (2 UTF-8 code units => 1 UTF-16)
- devanagri letter ssa (3 UTF-8 code units => 1 UTF-16)
- chakma digit zero (4 UTF-8 code units => 2 UTF-16 code units)
- some combinations of the above
Pick-to: 6.4 6.2
Task-number: QTBUG-103302
Change-Id: I981213c296bafc81663b08c0f1f339bbd8a96485
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
During the implementation of QString::append(QUtf8StringView) it has
become apparent that the testing is insufficient as it did not warn
about an extra growth. The following tests have been added that append:
- y-umlaut and greek letter small theta (2 UTF-8 code units => 1 UTF-16)
- devanagri letter ssa (3 UTF-8 code units => 1 UTF-16)
- chakma digit zero (4 UTF-8 code units => 2 UTF-16 code units)
- some combinations of the above
Pick-to: 6.4 6.2
Task-number: QTBUG-103302
Change-Id: I3d81cf10b7eb74433ce5bea9b92ce6bce1230dcd
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
During the implementation of QString::append(QUtf8StringView) it has
become apparent that the testing is insufficient as it did not warn
about an extra growth. The following tests have been added that append:
- y-umlaut and greek letter small theta (2 UTF-8 code units => 1 UTF-16)
- devanagri letter ssa (3 UTF-8 code units => 1 UTF-16)
- chakma digit zero (4 UTF-8 code units => 2 UTF-16 code units)
- some combinations of the above
Note that this also affects operator_pluseq_data, which is basically
a wrapper around append_data.
Pick-to: 6.4 6.2
Task-number: QTBUG-103302
Change-Id: I09ed950e3f0e71ae9ae85a455f42e130887f1109
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Both QStringView and QAnyStringView implicitly convert from any
container with a fitting value_type, and working std::data, std::size,
std::begin and std::end.
Add these missing operations (and complementary ones) to XmlStringRef,
so it implicitly converts to QStringView and QAnyStringView, too.
Add a check to that effect and remove the now-superfluous operator
QStringView().
Task-number: QTBUG-103302
Change-Id: I89d586cf64447a82022e06d546d7ee8339fc6dc7
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
... where checks for QByteArray existed before.
The checks we can't add are
- left/right/mid (legacy APIs not implemented in QBAV)
- several relational operators, d/t ambiguities. Created
QTBUG-108805 to track these.
Task-number: QTBUG-108805
Pick-to: 6.4 6.2
Change-Id: I30cc9b29a228d69d32af51234f2c28221478a75c
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
It's not intuitive, so check lest people break it.
Pick-to: 6.4 6.2
Change-Id: I2435cd69be7b77a6ae59cdc7b5fb99658cfc42fd
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Underflows should be treated the same as overflows.
Fixes: QTBUG-108628
Change-Id: I23aa7bbe1d103778cefca08bd3e584e72f306583
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Rework QSimpleParsedNumber to store a qsizetype whose sign serves as
ok flag (positive is ok, zero and negative are not) and magnitude is
the number of characters used. This replaces an endptr that was set to
null to indicate !ok, but that deprived us of end-of-parse
information, which is needed for number-parsing. In particular, JS's
parsing of numbers accepts overflow (where qstrntod() flags it as
invalid) as infinity; so qstrntod() does need to say how long the
overflowing (but JS-valid, none the less) number-text was.
Modify all callers of functions using this (recently-introduced) type
and add tests that fail without this fix.
Fixes: QTBUG-108628
Change-Id: I416cd213e1fb8101b1af5a6d43615b970a5db9b4
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
- If this string is not shared, modify it directly
- If this string is shared, instead of detaching copy the characters
from this string, except the ones that are going to be removed, to a
new string and swap it. This is more efficient than detaching, which
would copy the whole string including the characters that are going
to be removed.
This affects:
remove(const QString &str, Qt::CaseSensitivity cs)
remove(QLatin1StringView str, Qt::CaseSensitivity cs)
Adjust the unittests to test both code paths.
[ChangeLog][QtCore][QString] Improved the performance of
QString::remove() by avoiding unnecessary data copying. Now, if this
string is (implicitly) shared with another, instead of copying
everything and then removing what we don't want, the characters from
this string are copied to the destination, except the ones that need to
be removed.
Task-number: QTBUG-106181
Change-Id: Id8eba59a44bab641cc8aa662eb45063faf201183
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This tests that strings using the first Unicode code-point of such a
multi-character token don't get recognized as "valid" number strings.
This would catch an implementation issue if the parsing code
mistakenly matched against only the first code-point of each "single
character" token.
It also adds tests of integer formatting, with multi-character sign,
and reworks some QStringView().toString()s to use u"..."_s.
Task-number: QTBUG-107801
Change-Id: I7b868ce2955bb322b3ecfc200438a21437090a0c
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Basic unitttest and one to verify erase returns iterator, not
const_iterator.
Change-Id: I44c3b82b4686ff3809648063376f5e36fb7e181d
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
- If the string isn't shared, don't call detach(), instead remove characters
matching ch, and resize()
- If the string is shared, create a new string, and copy all characters
except the ones that would be removed, see task for details
Update unittets so that calls to this overload of remove() test both code
paths (replace() calls remove(QChar, cs) internally).
Drive-by change: use QCOMPARE() instead of QTEST()
Task-number: QTBUG-106181
Change-Id: I1fa08cf29baac2560fca62861fc4a81967b54e92
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
- If this bytearray isn't shared call d->erase() as needed
- if it's shared, instead of detaching, create a new bytearray, and copy
all characters except for the ones that would be removed
See task for details.
Adjust unittest to test both code paths.
Task-number: QTBUG-106182
Change-Id: I806e4d1707004345a2472e056905fbf675f765ab
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This is a semantic patch using ClangTidyTransformator as in
qtbase/df9d882d41b741fef7c5beeddb0abe9d904443d8, but extended to
handle typedefs and accesses through pointers, too:
const std::string o = "object";
auto hasTypeIgnoringPointer = [](auto type) { return anyOf(hasType(type), hasType(pointsTo(type))); };
auto derivedFromAnyOfClasses = [&](ArrayRef<StringRef> classes) {
auto exprOfDeclaredType = [&](auto decl) {
return expr(hasTypeIgnoringPointer(hasUnqualifiedDesugaredType(recordType(hasDeclaration(decl))))).bind(o);
};
return exprOfDeclaredType(cxxRecordDecl(isSameOrDerivedFrom(hasAnyName(classes))));
};
auto renameMethod = [&] (ArrayRef<StringRef> classes,
StringRef from, StringRef to) {
return makeRule(cxxMemberCallExpr(on(derivedFromAnyOfClasses(classes)),
callee(cxxMethodDecl(hasName(from), parameterCountIs(0)))),
changeTo(cat(access(o, cat(to)), "()")),
cat("use '", to, "' instead of '", from, "'"));
};
renameMethod(<classes>, "count", "size");
renameMethod(<classes>, "length", "size");
except that the on() matcher has been replaced by one that doesn't
ignoreParens().
a.k.a qt-port-to-std-compatible-api V5 with config Scope: 'Container'.
Added two NOLINTNEXTLINEs in tst_qbitarray and tst_qcontiguouscache,
to avoid porting calls that explicitly test count().
Change-Id: Icfb8808c2ff4a30187e9935a51cad26987451c22
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
- If this string isn't shared, don't call detach, instead use ->erase() as
needed
- If this string is shared, create a new string, and copy all elements
except the ones that would be removed, see task for details
Update unittest to test both code paths.
Task-number: QTBUG-106181
Change-Id: I4c73ff17a6fa89ddcf6966f9c5bf789753f6d39e
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
With the introduction of QAnyStringView, overloading based on UTF-8
and Latin-1 is becoming more common. Often, the two overloads can
share the processing backend, because we're only interested in the
US-ASCII subset of each.
But if they can't, we need a faster way to convert L1 into UTF-8 than
going via UTF-16. This is where the new private API comes in.
Eventually, we should have the converse operation, too, to complete
the set of direct conversions between the possible three
QAnyStringView encodings L1/U8/U16, but this direction is easier to
code (there are no error cases) and more immediately useful, so
provide L1->U8 alone for now.
Change-Id: I3f7e1a9c89979d0eb604cb9e42dedf3d514fca2c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Some of the entries in QLocale's single_character_data[] table are
not, in fact, single characters; some RTL languages include
bidi-markers in some of the fields, some locales use some denotation
of "times ten to the power" as the exponent separator. There may be
further complications, but let's just get some tests in that verify we
are correctly serializing numbers in these locales. Include some
parsing tests to show that we are indeed failing them.
Task-number: QTBUG-107801
Change-Id: Iab9bfcea5fdcfcb991451920c9531e0e67d02913
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
%.f should be handled like %.0f. You probably don't want it for strings,
though.
Fixes: QTBUG-107991
Pick-to: 6.2 6.4
Change-Id: I07ec23f3cb174fb197c3fffd1721a941fbcf15e1
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
There were two copies of the 0x1D157 row and we can't remember why.
So change one of them to the Chakma digit 3 (a spiral) and annote all
three test-cses with what meaning Unicode assigns to them.
Change-Id: I95837588bd5944f7f2c39c8438d9076e844e4dd0
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
/Zc:lambda seems buggy. Although in my experiments it works well
for 99% Qt repos, it seems some tests will trigger the bug and it
also blocks some new commits. So disable it for now, it's not stable
enough.
Now that this check is disabled, the workaround for tst_qstringapisymmetry
is also not needed anymore, so remove the workaround as well.
Partially reverts commit 8cb832090a
Change-Id: Icf0ecbbaa6262522470e5f5dea05705985ab18f1
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
The zlib convenience API we've been using so far has two problems:
- On Windows-64, where sizeof(long) == 4, the use of ulong for sizes
meant that we could not compress data compressable on other 64-bit
platforms (Unix). While zstream also uses ulong, being a stream API,
it allows feeding data in chunks. The total_in and total_out members
are only required for gzip compression and are otherwise just
informational. They're unsigned, so their overflow does not cause
UB. In summary, using zstream + deflate() allows us to compress more
than 4GiB of data even on Windows-64.
- On all platforms, we always allocated the output buffer in such a
way as to accommodate the pathological case of random, incompressible
data, so the output buffer was larger than the input. Using zstream
+ deflate(), we can start with a smaller buffer, then let zlib pick
up where it left off when it ran out of output buffer space, saving
memory in the common case that compression meaningfully reduces the
size. To avoid the first few rounds of reallocations, we continue to
use zlib's compressBound() for input less than 256KiB.
This completely fixes the compression side of QTBUG-106542 and
QTBUG-104972.
Pick-to: 6.4 6.3 6.2
Fixes: QTBUG-104972
Fixes: QTBUG-106542
Change-Id: Ia7e6c38403906b35462480fd611b482f05a5c59c
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The tests for indexOf() and lastIndexOf() had duplicate data row tags,
due to only using the needle and haystack, although some tests
differed only in start position. Include start position where needed.
Change-Id: I197d415265ab1a805f2d36fb88aec92ea8646f7a
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Enclosing one string in each substring of another does not need to
repeat the empty substring of the latter. Extracting the empty
substring from different positions doesn't get different results.
In the process, tidy up the code a bit.
Change-Id: Ic66febbdadeaac0c466f4f1174d831a991d31e20
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
