Commit Graph

34 Commits

Author SHA1 Message Date
Edward Welbourne
1530731694 tst_QTextBoundaryFinder: clean out the last two foreach uses
Change-Id: I71be0cb59b45fcce438c4a4749ec8b6f9e4f1694
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2023-08-11 19:58:01 +02:00
Alexandru Croitor
d4b7acec4b CMake: Make corelib tests standalone projects
Add the boilerplate standalone test prelude to each test, so that they
can be opened with an IDE without the qt-cmake-standalone-test script,
but directly with qt-cmake or cmake.

Boilerplate was added using the following scripts:
https://git.qt.io/alcroito/cmake_refactor

Manual adjustments were made where the code was inserted in the wrong
location.

Task-number: QTBUG-93020
Change-Id: I28b6d3815c5f43d2c33ea65764f6f3f8f129eaf3
Reviewed-by: Amir Masoud Abdol <amir.abdol@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2023-07-05 15:09:32 +02:00
Friedemann Kleint
97bfacf1e2 tests: Remove remains of qmake conversion from CMakeLists.txt files
Pick-to: 6.5
Change-Id: I8d106554bb86ac1ec9bb7a4083de4c376bcbab1d
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2023-02-17 21:56:49 +01:00
Mikolaj Boc
ae3ffc1d4b Remove tst_QTextBoundaryFinder::thaiLineBreak
The test body has been inside an #if 0 for long, remove the test
altogether. The part that loads libthai does not check anyhing
meaningful.

Task-number: QTBUG-109954
Change-Id: I8ce006a14e4cf926b668e958d4b2343f965a5fb6
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
2023-01-13 17:20:55 +01:00
Ievgenii Meshcheriakov
c4e550703c Update UCD to Revision 30
This corresponds to Unicode version 15.0.0.

Added the following scripts:

    * Kawi
    * Nag Mundari

Full support of these scripts requires harfbuzz version 5.2.0,
this version adds support for Unicode 15.0:

    https://github.com/harfbuzz/harfbuzz/releases/tag/5.2.0

Fixes: QTBUG-106810
Change-Id: Ib06c526e49b0f01ef9f21123bcf875c6b19f2601
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-10-11 14:10:59 +00:00
Marc Mutz
df9d882d41 Port from container.count()/length() to size()
This is semantic patch using ClangTidyTransformator:

  auto QtContainerClass = expr(hasType(namedDecl(hasAnyName(<classes>)))).bind(o)
  makeRule(cxxMemberCallExpr(on(QtContainerClass),
                             callee(cxxMethodDecl(hasAnyName({"count", "length"),
                                                  parameterCountIs(0))))),
           changeTo(cat(access(o, cat("size"), "()"))),
           cat("use 'size()' instead of 'count()/length()'"))

a.k.a qt-port-to-std-compatible-api with config Scope: 'Container'.

<classes> are:

    // sequential:
    "QByteArray",
    "QList",
    "QQueue",
    "QStack",
    "QString",
    "QVarLengthArray",
    "QVector",
    // associative:
    "QHash",
    "QMultiHash",
    "QMap",
    "QMultiMap",
    "QSet",
    // Qt has no QMultiSet

Change-Id: Ibe8837be96e8d30d1846881ecd65180c1bc459af
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2022-10-04 07:40:08 +02:00
Lucie Gérard
32df595275 Change the license of all CMakeLists.txt and *.cmake files to BSD
Task-number: QTBUG-105718
Change-Id: I5d3ef70a31235868b9be6cb479b7621bf2a8ba39
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-08-23 23:58:42 +02:00
Lucie Gérard
fb1b20eab3 Add license headers to cmake files
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.

Existing copyright statements remain intact

Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-08-03 17:14:55 +02:00
Ievgenii Meshcheriakov
0e64d97cc1 tst_qtextboundaryfinder: Remove full copies of data files
There are no commented out test cases remaining, so the normal
test vectors are identical to full test vectors.

Fixes: QTBUG-97537
Pick-to: 6.3
Change-Id: I987f178f192e1c8e2d998d36499fdce84f237e77
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:43 +02:00
Ievgenii Meshcheriakov
4d4d8dac94 QUnicodeTools: Fix line breaking before open parentheses
UAX #14, revision 45 (Unicode 13) has changed rule LB30 to only
trigger if the open parentheses is non-wide:

    (AL | HL | NU) × [OP-[\p{ea=F}\p{ea=W}\p{ea=H}]]

This fixes the remaining 24 line break tests.

Task-number: QTBUG-97537
Pick-to: 6.3
Change-Id: I9870588c04bf0f6ae0a98289739bef8490f67f69
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:43 +02:00
Ievgenii Meshcheriakov
40b4ad1866 QUnicodeTools: Fix line breaking for potential emojis
Implement part of LB30b introduced by UAX #14, revision 47
(Unicode 14.0.0):

    [\p{Extended_Pictographic}&\p{Cn}] × EM

This fixes one line breaking test.

Task-number: QTBUG-97537
Pick-to: 6.3
Change-Id: I3fd2372a057b7391d8846e9c146f69a54686ea61
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:42 +02:00
Ievgenii Meshcheriakov
08d2ae411f QUnicodeTools: Fix interactions of WB3d and WB4 rules
Word breaking rule WB3d should not be affected by WB4.

This fixes the remaining word break test.

Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: I99aee831d7c54fafcd2a9d526a3e078b12c5bfad
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:42 +02:00
Ievgenii Meshcheriakov
c63cdbdc43 QUnicodeTools: Handle WB3c word break rule
Adjust handling of WB3c rule to UAX #29, revision 33 (Unicode 11.0.0).

The rule reads:

    ZWJ × \p{Extended_Pictographic}

This fixes 9 word break tests.

Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: I818d4048828e6663d5c090aa372d83f5099fdffe
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:42 +02:00
Ievgenii Meshcheriakov
79d26ef3ae QUnicodeTools: Adjust properties of WSegSpace word break class
Disable break between sequences of WSegSpace characters (rule WB3d,
introduced in UAX #29, version 33, Unicode 11.0.0). Also disable breaks
between WSegSpace and (Extend | Format | ZWJ) due to rule WB4.

Adjust "words4" test to take the above changes into account (space
character belongs to WSegSpace).

Mention the full class name in a comment inside the word break table.

This fixes 34 word break tests.

Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: I7dfe8367e45c86913bb7d7fe2adb053711978487
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:42 +02:00
Ievgenii Meshcheriakov
9a831bdf54 QUnicodeTools: Fix handling of LB22 line break rule
This rule was simplified in version UTS #14 version 45 (Unicode 13.0.0)
to read:

    × IN

Re-enabled 28 fixed line break tests.

Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: I1c5565a8c1633428c22379917215d4e424ff0055
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:42 +02:00
Ievgenii Meshcheriakov
92d340ed1c QUnicodeTools: Fix handling of ZWJ for line breaks
Adjust implementation of rule LB8a of UAX #14. The rule was changed
in version 41 (corresponding to Unicode 11.0.0):

     ZWJ × (ID | EB | EM) ⇒ ZWJ ×

Fixing this rule fixes 9 line break tests. Those are re-enabled.

Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: I1570719590a46ae28c98ed7d5053e72b12915db7
Reviewed-by: Øystein Heskestad <oystein.heskestad@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:41 +02:00
Lucie Gérard
05fc3aef53 Use SPDX license identifiers
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.

Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-16 16:37:38 +02:00
Alexey Edelev
815bc61626 Cleanup tests that add test data to resources explicitly
Remove Integrity and Android specific code that explicitly adds
test data to the resource files. qt_internal_add_test functions
implicitly adds test data to resources for Android and Integrity
platforms by default.

Change-Id: Ia1d58755b47442e1953462e38606f70fec262368
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-02-11 21:54:44 +01:00
Joerg Bornemann
238e3beb6f Remove unused .qrc files
Task-number: QTBUG-94446
Change-Id: I136d8b4ab070a832866aa50b5701fc6bd863df8a
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-01-17 23:17:01 +01:00
Ievgenii Meshcheriakov
826fc8c9bd Update UCD to Revision 28
This corresponds to Unicode version 14.0.0.

Added the following scripts:

    * CyproMinoan
    * OldUyghur
    * Tangsa
    * Toto
    * Vithkuqi

Full support of these scripts requires harfbuzz version 3.0.0,
this version adds support for Unicode 14.0:

    https://github.com/harfbuzz/harfbuzz/releases/tag/3.0.0

With this release 10 test cases in tst_qurluts46 were fixed, one
additional test case is failing in tst_qtextboundaryfinder and
is commented out. In total 62 line break test cases and 44 word
break test cases are failing.

A comment in src/corelib/text/qt_attribution.json was updated to
include the URL of the page containing UCD version number.

Fixes: QTBUG-94359
Change-Id: Iefc9ff13f3df279f91cbdb1246d56f75b20ecb35
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-18 16:45:10 +00:00
Edward Welbourne
7a4b586f4b Remove conditioning on Android embedded
It is no longer handled separately from Android.
This effectively reverts commit 6d50f746fe

Change-Id: Ic2d75b8c5a09895810913311ab2fe3355d4d2983
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
2021-09-17 17:30:14 +02:00
Giuseppe D'Angelo
d48058f197 Unicode: fix the grapheme clustering algorithm
An oversight in the code kept the algorithm in the GB11 state, even if
the codepoint that is being processed wouldn't allow for that (for
instance a sequence of ExtPic, Ext and Any).

Refactor the code of GB11/GB12/GB13 to deal with code points that break
the sequences (falling back to "normal" handling).

Add some manual tests; interestingly enough, the failing cases are not
covered by Unicode's tests, as we now pass the entire test suite.

Amends a794c5e287.

Fixes: QTBUG-94951
Pick-to: 6.1 5.15
Change-Id: If987d5ccf7c6b13de36d049b1b3d88a3c4b6dd00
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-08-24 12:58:20 +02:00
Giuseppe D'Angelo
a794c5e287 Unicode: fix the extended grapheme cluster algorithm
UAX #29 in Unicode 11 changed the EGC algorithm to its current form.
Although Qt has upgraded the Unicode tables all the way up to
Unicode 13, the algorithm has never been adapted; in other words,
it has been working by chance for years. Luckily, MOST
of the cases were dealt with correctly, but emoji handling
actually manages to break it.

This commit:

* Adds parsing of emoji-data.txt into the unicode table generator.
  That is necessary to extract the Extended_Pictographic property,
  which is used by the EGC algorithm.

* Regenerates the tables.

* Removes some obsoleted grapheme cluster break properties, and
  adds the ones added in the meanwhile.

* Rewrites the EGC algorithm according to Unicode 13. This is
  done by simplifying a lot the lookup table. Some rules (GB11,
  GB12, GB13) can't be done by the table alone so some hand-rolled
  code is necessary in that case.

* Thanks to these fixes, the complete upstream GraphemeBreakTest
  now passes. Remove the "edited" version that ignored some rows
  (because they were failing).

Change-Id: Iaa07cb2e6d0ab9deac28397f46d9af189d2edf8b
Pick-to: 6.1 6.0 5.15
Fixes: QTBUG-92822
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Konstantin Ritt <ritt.ks@gmail.com>
2021-04-16 20:31:39 +02:00
Joerg Bornemann
ad2da2d27a Remove the qmake project files
Remove the qmake project files for most of Qt.

Leave the qmake project files for examples, because we still test those
in the CI to ensure qmake does not regress.

Also leave the qmake project files for utils and other minor parts that
lack CMake project files.

Task-number: QTBUG-88742
Change-Id: I6cdf059e6204816f617f9624f3ea9822703f73cc
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
2021-01-07 15:32:28 +01:00
David Skoland
27d96b4789 Replace QtTest headers with QTest
Complete search and replace of QtTest and QtTest/QtTest with QTest, as
QtTest includes the whole module. Replace all such instances with
correct header includes. See Jira task for more discussion.

Fixes: QTBUG-88831
Change-Id: I981cfae18a1cabcabcabee376016b086d9d01f44
Pick-to: 6.0
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2020-12-22 15:20:30 +01:00
Giuseppe D'Angelo
1869615fc9 QChar: make construction from integral explicit
QChar should not be convertible from any integral type except from
char16_t, short and possibly char (since it's a direct superset).

David provided the perfect example:

  if (str == 123) { ~~~ }

compiles, with 123 implicitly converted to QChar (str == "123"
was meant instead). But similarly one can construct other
scenarios where QString(123) gets accidentally used (instead of
QString::number(123)), like QString s; s += 123;.

Add a macro to revert to the implicit constructors, for backwards
compatibility.

The breaks are mostly in tests that "abuse" of integers (arithmetic,
etc.). Maybe it's time for user-defined literals for QChar/QString,
but that is left for another commit.

[ChangeLog][Potentially Source-Incompatible Changes][QChar] QChar
constructors from integral types are now by default explicit.
It is recommended to use explicit conversions, QLatin1Char,
QChar::fromUcs4 instead of implicit conversions. The old behavior
can be restored by defining the QT_IMPLICIT_QCHAR_CONSTRUCTION
macro.

Change-Id: I6175f6ab9bcf1956f6f97ab0c9d9d5aaf777296d
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2020-11-15 14:41:05 +01:00
Alexandru Croitor
403213240c CMake: Regenerate projects to use new qt_internal_ API
Modify special case locations to use the new API as well.
Clean up some stale .prev files that are not needed anymore.
Clean up some project files that are not used anymore.

Task-number: QTBUG-86815
Change-Id: I9947da921f98686023c6bb053dfcc101851276b5
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2020-09-23 16:59:06 +02:00
Lars Knoll
0ae5b8af9c Clean up QTextBoundaryFinder and qunicodetools
Make QTBF ready for Qt6 by using qsizetype in the API and use
QStringView where it makes sense.

Change the exported API of qunicodetools to use QStringView as
well and use char16_t internally.

Change-Id: I853537bcabf40546a8e60fdf2ee7d751bc371761
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2020-09-07 09:09:55 +02:00
Lars Knoll
9515d45f88 Remove QTextCodec dependency from test
Change-Id: Ie546065c3179d475df46b284ca7df502c4465b93
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2020-05-14 07:50:49 +02:00
Sona Kurazyan
7e1dacc27a Port qtbase/tests/auto/corelib/text tests to CMake
Task-number: QTBUG-78220
Change-Id: I497da6ed489854bdee5a1ead9a3f34118c78d001
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-04-27 14:34:51 +02:00
Edward Welbourne
54f8be6cc0 Update UCD to Revision 26
Include WordBreakTest.html, since a test uses sample strings from it,
albeit without actually reading the file.

Had to comment out more of the new tests, as at Revision 24, pending
an update to harfbuzz and the text boundary detection code.

Task-number: QTBUG-79631
Task-number: QTBUG-79418
Task-number: QTBUG-82747
Change-Id: I0082294b09d67ffdc6a9b5c15acf77ad3b86f65f
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2020-03-14 11:26:59 +01:00
Edward Welbourne
00f0863cbe Core: Use Qt::SplitBehavior in preference to QString::SplitBehavior
The Qt version was added in 5.14 "for use as eventual replacement for
QString::SplitBehavior." Move another step closer to that goal.

Change-Id: I446f9ddc8f8de4a0b79b09edb44f7c1496fbc33f
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2020-02-28 15:37:11 +01:00
Edward Welbourne
c3eb521a0f Update UCD data to Unicode 12.1.0's Revision 24
Had to teach the update program to accept category Lm as for
Joining_Transparent, for the sake of a new ArabicShaping.txt entry.
Added three new Unicode versions, several new scripts and a new
word-break class.

Updated UCD's test data for tst_QTextBoundaryFinder.  This left 57
tests failing; I have commented out the data rows for those tests,
pending someone with more knowledge addressing this.

Task-number: QTBUG-79631
Task-number: QTBUG-79418
Change-Id: Ic33d3b3551195d47a84d98e84020f57a68f0b201
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
2019-10-30 17:38:02 +01:00
Edward Welbourne
a9aa206b7b Move text-related code out of corelib/tools/ to corelib/text/
This includes byte array, string, char, unicode, locale, collation and
regular expressions.

Change-Id: I8b125fa52c8c513eb57a0f1298b91910e5a0d786
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2019-07-10 17:05:30 +02:00