Now that QStringConverter can handle non UTF encodings through ICU,
add a way to get a decoder for arbitrary HTML code.
Opposed to QStringConverter::encodingForHtml(), this method will
try to create a valid string decoder also for non unicode codecs.
Pick-to: 6.4
Change-Id: I343584da1b114396c744f482d9b433c9cedcc511
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This adds support for additional codecs to QStringConverter when ICU is
available.
We store the converter in the state (d[0]), and its canonical name in
d[1]. We need the name there, as in the clear function we close the
UConverter, and set the pointer to null. Consequently, the actual
conversion functions might need to re-open the converter again. The
advantage of this approach is that clear is used in the destructor of
State, and with this approach we properly clean up the state.
There is however a disadvantage: The clear function was so far also used
for resetting the state when QStringConverter::resetState . Discarding
the whole Uconverter for that is however rather costly. For that reason
we modify resetState to call a new function, State::reset. For existing
converters, it behaves the same as clear; for the ICU based converter,
we call the more efficient ucnv_reset. Code compiled against Qt 6.4 can
benefit from this more efficient version; code compiled against older Qt
versions will continue to work, as the conversion functions can just
recretate the converter from the name.
We can distinguish between ICU and non-ICU converters by checking if the
UsesIcu flag is set.
QStringConverter::name is changed to return the name stored in d[1]. The
interface of the ICU converter has a dummy name, so code using the old
name function from QT < 6.4 still returns something, namely a message
asking the user to recompile.
The function is moved out of line, as we need to check for the private
ICU feature, and want to avoid having that check in the public header.
As the QStringConverter ctor taking a name now can allocate memory, it
can no longer be noexcept. Removing the noexceptness is safe, as it was
only added after Qt 6.3.
Note that we cannot extend the API consuming or returning Encoding, as
we use Encoding values to index into an array of converter interfaces in
inline API.
Further API to support getting an ICU converter for HTML will be added
in a future commit.
Currently, the code depending on ICU is enabled at compile time if ICU
is found. However, in the future it could be moved into a plugin to
avoid a hard dependency on ICU in Core.
[ChangeLog][Corelib][Text] QStringConverter and API using it now
supports more text codecs if Qt is compiled with ICU support.
Fixes: QTBUG-103375
Change-Id: I7afb92fc68ef994179ebc7a3aa73beebb1386204
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
We implicitly checked it, because, in C++20 builds, the non-equality
relational operators are synthesized from it by the compiler, and we
test those, but we didn't check that <=> returns strong_ordering.
We now do.
Pick-to: 6.4 6.3 6.2
Task-number: QTBUG-104108
Change-Id: Ieb19a2d4cb2d600d884f4e2e89e98c6187e23872
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This allows to defer the toString() invocation until it is really
needed, and so allows to speed-up the test execution.
I was testing a release shared developer build, running
tst_QStringApiSymmetry 10 times before the change, and 10 times after
the change, and then taking the average execution duration, as
reported in the log.
Before the change (using QCOMPARE): 51ms
After the change (using QCOMPARE_EQ): 45ms
As we see from the results, the benefit is around 10%.
Task-number: QTBUG-98873
Task-number: QTBUG-98874
Change-Id: Ifcfbcca1f2c1eaf82c7f1a9098fa1512a269cbf8
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Attempting to use an invalid QStringConverter would so far have resulted
in a crash, as we would dereference the null iface pointer.
Fix this by inserting adequate checks, and ensure that hasError returns
true if we attempt to en/decode with an invalid converter.
Pick-to: 6.2 6.3
Change-Id: Icf74bb88cd8c95685481cc0bd512da99b62f33e6
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
There are no commented out test cases remaining, so the normal
test vectors are identical to full test vectors.
Fixes: QTBUG-97537
Pick-to: 6.3
Change-Id: I987f178f192e1c8e2d998d36499fdce84f237e77
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
UAX #14, revision 45 (Unicode 13) has changed rule LB30 to only
trigger if the open parentheses is non-wide:
(AL | HL | NU) × [OP-[\p{ea=F}\p{ea=W}\p{ea=H}]]
This fixes the remaining 24 line break tests.
Task-number: QTBUG-97537
Pick-to: 6.3
Change-Id: I9870588c04bf0f6ae0a98289739bef8490f67f69
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Implement part of LB30b introduced by UAX #14, revision 47
(Unicode 14.0.0):
[\p{Extended_Pictographic}&\p{Cn}] × EM
This fixes one line breaking test.
Task-number: QTBUG-97537
Pick-to: 6.3
Change-Id: I3fd2372a057b7391d8846e9c146f69a54686ea61
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Word breaking rule WB3d should not be affected by WB4.
This fixes the remaining word break test.
Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: I99aee831d7c54fafcd2a9d526a3e078b12c5bfad
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Disable break between sequences of WSegSpace characters (rule WB3d,
introduced in UAX #29, version 33, Unicode 11.0.0). Also disable breaks
between WSegSpace and (Extend | Format | ZWJ) due to rule WB4.
Adjust "words4" test to take the above changes into account (space
character belongs to WSegSpace).
Mention the full class name in a comment inside the word break table.
This fixes 34 word break tests.
Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: I7dfe8367e45c86913bb7d7fe2adb053711978487
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
This rule was simplified in version UTS #14 version 45 (Unicode 13.0.0)
to read:
× IN
Re-enabled 28 fixed line break tests.
Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: I1c5565a8c1633428c22379917215d4e424ff0055
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Adjust implementation of rule LB8a of UAX #14. The rule was changed
in version 41 (corresponding to Unicode 11.0.0):
ZWJ × (ID | EB | EM) ⇒ ZWJ ×
Fixing this rule fixes 9 line break tests. Those are re-enabled.
Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: I1570719590a46ae28c98ed7d5053e72b12915db7
Reviewed-by: Øystein Heskestad <oystein.heskestad@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Simplifies the test a little.
Pick-to: 6.3
Change-Id: I77c8221eb2824c369feffffd16f0a7fc44215aaf
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
The utf8.txt file was only 21 bytes and contained exactly two non-ASCII
characters. It wasn't very good.
This commit brings back the UTF-8 test rows that existed before commit
18ec53156e deleted tst_Utf8. There's a lot
of overlap with some of the other rows in this test, though.
Pick-to: 6.2 6.3
Change-Id: I77c8221eb2824c369feffffd16f094619b69faef
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
The QLocal8Bit implementation assumes that there's at most one
continuation byte -- that is, that all codecs are either Single or
Double Byte Character Sets (SBCS or DBCS). It appears to be the case for
all Windows default codepages, except for CP_UTF8, which is an opt-in
anyway.
Instead of fixing our codec, let's just use the optimized UTF-8
implementation.
[ChangeLog][Windows] Fixed support for using Qt applications with UTF-8
as the system codepage or by enabling that in the application's
manifest.
Discussed-on: https://lists.qt-project.org/pipermail/interest/2022-May/038241.html
Pick-to: 6.2 6.3
Change-Id: I77c8221eb2824c369feffffd16f0912550a98049
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Amends 2a893db480, which changed the
#ifdef'ery but didn't actually enable the building of the test on
Android.
Task-number: QTBUG-87414
Pick-to: 6.3 6.2
Change-Id: Id944dd3023da40ecbf4b8a324784409a63f94aec
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
We don't support it any more. I don't think it has ever properly
compiled Qt 6 (and it's no longer working for me against GCC 12's
libstdc++ headers). If you report a bug against it, Intel support's
first question is if you can try instead the new Clang/LLVM-based oneAPI
C++ compiler.
So we support only that one, which identifies itself as Q_CC_CLANG.
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16eb57a092c8439e
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
[ChangeLog][QtCore] Deprecated _qs and _qba literal operators
for QString and QByteArray in favor of _s and _ba in the
Qt::Literals::StringLiterals namespace.
Task-number: QTBUG-101408
Change-Id: I26aee0055e3b4c1860de6eda8e0eb857c5b3e11a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
[ChangeLog][QtCore][QByteArray/QByteArrayView/QLatin1String/QString/QStringView]
The string-to-integer conversion functions (toInt() etc) now support
the 0b prefix for binary literals. That means that base = 0 will
recognize 0b to mean base = 2 and an explicit base = 2 argument will
make toInt() (etc) skip an optional 0b.
[ChangeLog][QtCore][Important Behavior Changes] Due to the
newly-introduced support for 0b (binary) prefixes in integer parsing,
some strings that were previously rejected as invalid now parse as
valid. E.g., Qt 6.3 with autodetected bases would have tried to parse
"0b1" as an octal value and fail, whereas 6.4 will parse it as the
binary literal and return 1.
Fixes: QTBUG-85002
Change-Id: Id4eff72d63619080e5afece4d059b6ffd52f28c8
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Some languages have, in the same territory, locales for more than one
script. In such cases, since we ignored the script, we got the one
that is used by default, instead of the one actually asked for. Take
the script into account.
Added TODO comment in test listing the known examples of this;
manually tested before and after the fix to verify the prior code was
indeed getting it wrong and now does do it right.
Change-Id: Iaf9201d6992bc39e6e9346ef8b7c69d418db7253
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
The documentation overtly listed entries in this order but the code
gave less specific entries before more specific (and the tests
verified the same). This is now reversed.
[ChangeLog][QtCore][QLocale] uiLanguages() now prefers more specific
locale names over less specific ones, matching its own documentation,
except where the system backend supplies them in some other order.
This means a translation with the expected script and/or territory as
well as language will be used in preference to a generic one for just
the language, rather than only as a fall-back when the more generic
one is missing.
Fixes: QTBUG-102796
Change-Id: I3c7b3627afb51246df5a6ad0230e23b60af78071
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This shows we're lacking support for the 0b prefix, and, as it turns
out, there's a request for it.
Task-number: QTBUG-85002
Pick-to: 6.3
Change-Id: Ie201c84bf906a7e482b929301699ceb429b53c14
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
When the date-time string falls in a spring-forward (so is invalid)
and one of the fields of the parsed string doesn't match the format
it's meant to (e.g. a single-digit seconds field when format ss was
specified), a check that the current fall-back date-time is between
the minimum and maximum for the parser object failed, triggering an
assertion.
In any case, an invalid default-value wasn't useful to the code that
parsed a single section of the date-time string, so brute-force the
current value to a valid date-time (when possible) using the usual
round-trip via milliseconds since the epoch.
Added the test-case which first revealed the problem, plus a couple
more informed by it, to exercise the same code-paths with fewer things
failing.
Fixes: QTBUG-102199
Pick-to: 6.3 6.2 5.15
Change-Id: I658308614505ef25f4c97d0de6148acb54a65a0f
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
[ChangeLog][QtCore] Added literal operators for _s and _ba for QString
and QByteArray respectively in the Qt::Literals::StringLiterals
namespace.
Task-number: QTBUG-101408
Change-Id: I5cd4e7f36f614ea805cfecc27b91c5d981cd3794
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Andrei Golubev <andrei.golubev@qt.io>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
QList and QString had them, so add them to QByteArray and
QVarLengthArray, too.
In the QVLA case, we need to jump though a hoop or two to avoid having
to duplicate all the reallocation logic. Nothing a few template tricks
cannot solve.
[ChangeLog][QtCore][QByteArray] Added resize(n, ch) overload.
[ChangeLog][QtCore][QVarLengthArray] Added resize(n, v) overload.
Fixes: QTBUG-102270
Change-Id: I0d281ae5b574f440f682e4a62427b434dcf5b687
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Update docs and add tests.
[ChangeLog][QtCore] Documented existing support for 'F' format when
converting floating-point numbers to strings in QLocale::toString(),
hence equally for QString's floating-point formatting. Previously it
was supported but the documentation neglected to mention it; it only
differs from 'f' for infinities and NaN.
Change-Id: Ic946c0f7b9e86fdf512daa3124bea57fc664b34b
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
The testing of infinity and NaN somewhat duplicated existing tests in
tst_QNumeric and, in any case, belongs there.
Change-Id: I6b5d1ff9767daf8e4bbe0025d3efab3d74ed35de
Reviewed-by: Miikka Heikkinen <miikka.heikkinen@qt.io>
Reviewed-by: Sona Kurazyan <sona.kurazyan@qt.io>
Use the modern "portable" _clearfp() and _controlfp(), that work
regardless of the hardware MS is running on, rather than the
x86-specific _clear87() and _control87(); and move some defines into
the #if-ery context in which they're used, rather than defining them
regardless of whether they're used.
Change-Id: I13abf2c85525b1182b99bc2c366aadd6b5fc656f
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
Removed "explicit" keyword from constructors taking
(const char *, qsizetype) and (const char *, const char *).
Switched to using brace initialization for creating QLatin1StringView
in QtCore.
[ChangeLog][QtCore][QLatin1StringView] The (const char *, qsizetype)
and (const char *, const char *) constructors are no longer explicit.
Change-Id: I4f6760692e4df60fe4231e86a25f6ea03cd1bf82
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
When comiling with ubsan, GCC is being too restrictive and ignores that
'static constexpr char []' can never be nullptr. It tries to evaluate
the comparison with nullptr, and fails with:
error: ‘(((const char*)(& hello)) == 0)’ is not a constant
Disable the checks when compiling with GCC.
Fixes: QTBUG-101307
Pick-to: 6.3 6.2
Change-Id: I8322bb7cb326e06cd03b8b107c46a494c825087b
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Do not try to properly specify a path to an external syslocale app,
because the related test can't be executed on Android anyway.
This fixes a failure in initTestCase() and allows to unblock other
test-cases.
Also skip systemLocale() test, because Android platform plugin already
creates a custom QSystemLocale instance, so a hack with MySystemLocale
does not work.
As a drive-by: simplify the #ifery in initTestCase() and remove an
unused second parameter of QSKIP() in emptyCtor_data().
Fixes: QTBUG-87414
Pick-to: 6.3 6.2
Change-Id: Iefc587062362469856fdca77f9ec9d96f5552e45
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
The test was failing because test data was not provided correctly.
That was fixed in 4aea86f5e8 but the
test was never unblacklisted.
Fixes: QTBUG-87418
Pick-to: 6.3 6.2
Change-Id: Ibef7dcfaf59ef50f90d6538a562d03af17f065e0
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
This has performance benefits when doing comparisons.
The check is only performed at compile time.
Task-number: QTBUG-101014
Change-Id: I55694b045fe5e75d9671d0a3a70c80d998cf98c8
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Percent-decoding was previously only present as a static method taking
a QBA parameter; it might as well be an instance method of that
parameter. Change most QBA tests to use it rather the static method.
[ChangeLog][QtCore][QByteArray] percentDecoded() is now available as
an instance method of the byte array to be decoded, equivalent to the
static QByteArray::fromPercentEncoding().
Change-Id: I982101c44bdac5cc4041e85598d52ac101d38fa1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
QString already has a localeAwareCompare(QStringView, QStringView)
static method. Use it in QStringView::localeAwareCompare(QStringView).
[ChangeLog][QtCore][QStringView] Added
QStringView::localeAwareCompare().
Task-number: QTBUG-98431
Change-Id: Iec3865a5439d9fb653cc1150da21042186bdee98
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
The previous constraint called for the value_type of the container to
exactly match the value_type of the tokenizer, which means
toContainer() could only ever work on containers of views. But there
is value in allowing QStringList, even though it works only on QL1S
needles (QStringView -> QString isn't implicit). But users may have
other types that for better or worse implicitly convert from views, so
we shouldn't over-constrain the function.
[ChangeLog][QtCore][QStringTokenizer] toContainer() now works on
containers whose value_type can be constructed from the tokenizer's
value_type. It no longer requires an exact match.
Fixes: QTBUG-101702
Change-Id: Ie384cd1c4b51eaa57675f2a014141ceec8651c81
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
This was not previously tested.
Change-Id: Icd287b519f6bc5d450f4490990ac78b0d06774f6
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
And remove their uses.
[ChangeLog][QtCore][Deprecation Notice] Deprecated QString::count()
and QByteArray::count() that take no parameters, to avoid confusion
with the algorithm overloads of the same name. They can be replaced
by size() or length() methods.
Change-Id: I6541e3235ab58cf750d89568d66d3b1d9bbd4a04
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Required for the API symmetry between QStringView and QLatin1String.
[ChangeLog][QtCore][QStringView] Added an overload of
QStringView::count() for QLatin1String.
Change-Id: Ic49a4b31e8f6f0969eff0f792654d23a60e06c49
Task-numer: QTBUG-98431
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
