Commit Graph

1186 Commits

Author SHA1 Message Date
Mikolaj Boc
ad1980cd43 Create a driver for running batched tests on WASM
A driver application has been prepared in js for running batched tests.
There is a convenient public API defined for reading the current test
status & subscribing to changes thereof.
The solution is modular - the module qwasmjsruntime can be used for any
wasm instantiation, e.g. in the next iteration of qtloader.

Change-Id: I00df88188c46a42f86d431285ca96d60d89b3f05
Pick-to: 6.4
Reviewed-by: David Skoland <david.skoland@qt.io>
2022-08-24 19:08:58 +02:00
Alexandru Croitor
a310319a04 CMake: Rewrite double-conversion find module
Rename FindWrapDoubleConversion.cmake into
FindWrapSystemDoubleConversion.cmake.
Merge contents of Finddouble-conversion.cmake into the one above.
This allows users to provide their own Finddouble-conversion.cmake
file (Conan can do it).
Don't mark the system package as required, because we have a bundled
one too.
Add link to upstream.
Make sure to show either Config file or library path when one is
found.

Pick-to: 6.2 6.3 6.4
Fixes: QTBUG-104541
Change-Id: I9ea2330697c6fc280328849ca11522291c4073d8
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2022-07-02 00:11:12 +02:00
Dimitrios Apostolou
b152c3b50d sanitizer-testrunner: enforce line buffering
Under coin-agent, the stdout/err are not a tty, so flushing needs to be
forced.

Change-Id: I06de43328a4f4d1c17df7188f31b5f7bc63e3335
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-06-16 21:04:48 +02:00
Dimitrios Apostolou
1ab354b245 qt-testrunner: Commit forgotten README file
Pick-to: 6.4 6.3 6.2
Change-Id: Ie307b95947a6910bd65bb22ecf938fae9de22f9d
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-06-16 21:04:48 +02:00
Dimitrios Apostolou
fe0e87c9a6 Add ASAN build for qtbase
The test run is wrapped with a special TESTRUNNER script that ignores
failing tests (there are several tests failing when built with ASAN) and
also ignores LSAN errors (memory leaks - but still visible in the
output).

The test run only fails if a test reports ASAN errors or if it
crashes (or times out, which is like a crash caused by qtestlib's
watchdog timer).

Fixes: QTQAINFRA-5025
Change-Id: I861756ab49388ac4a52409d3a780684244e469b1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-06-15 23:58:16 +02:00
Kai Köhne
04cc705947 CMake: Fix typos
Found by codespell

Pick-to: 6.4
Change-Id: I4907e423b6b345acf82f2d7e0ed62479719d694e
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-06-13 15:37:36 +02:00
Jani Heikkinen
0cbb471b1b Bump version to 6.5.0
Change-Id: I83a4f915a914bdc18f6706bb902f3e3b13da074f
Reviewed-by: Qt Submodule Update Bot <qt_submodule_update_bot@qt-project.org>
2022-06-06 13:23:59 +03:00
David Skoland
fd4f218c1e Expose the qtloader object globally
When testing, we need to query the state of the Qt application,
so change the scope of qtloader from inside the init function
to global scope.

Additionally, adjust the test script accordingly to query and use this
state to make good decisions on how to terminate.

Change-Id: I6264ba20843716eb87340b160680617b718f6bd9
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2022-06-03 05:30:46 +02:00
Yuhang Zhao
42fe677584 Core: make CLDR data constexpr
Task-number: QTBUG-100485
Pick-to: 6.3 6.2
Change-Id: Ib8c5160ca0994662a5fdc2293dc734c1bdcac4f2
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
2022-05-27 09:33:28 +08:00
Yuhang Zhao
34c21d0407 Core: make Unicode Database constexpr
Task-number: QTBUG-100485
Pick-to: 6.3 6.2
Change-Id: I41480a34b14fd86a68a5c10b7e0f3d250e785d0f
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-26 12:42:36 +08:00
Ievgenii Meshcheriakov
838a7a01f3 Unicode: Extract EastAsianWidth property
This property is needed to properly implement the line breaking
algorithm from UAX #14.

Task-number: QTBUG-97537
Pick-to: 6.3
Change-Id: Ia83cc553c9ef19fae33560721630849d2a95af84
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:43 +02:00
Ievgenii Meshcheriakov
1a26719c54 Unicode: Remove obsolete word break classes
Remove E_Base, Glue_After_Zwj, E_Base_GAZ, and E_Modifier obsoleted by
UTS #29, version 33 (Unicode 11.0.0).

Task-number: QTBUG-97537
Pick-to: 6.2 6.3
Change-Id: If5dc36ae17cd8746bbe81b73bbcc0863181e4a7a
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2022-05-24 23:07:42 +02:00
Assam Boudjelthia
53604df833 Android: fix default name of the CI emulator
This name was changed in qt5/coin but wasn't reflected here. Currently,
things works correctly because the test target provide the correct name
as an env var.

Pick-to: 6.2 6.3
Change-Id: I8968285de2c7759d16f303c48d6295295dffbef6
Reviewed-by: Simo Fält <simo.falt@qt.io>
2022-05-19 15:43:38 +03:00
Luca Di Sera
e5a5c099af Add an exception in testrunner for QDoc
QDoc recently added some tests that use Catch2 instead of testlib.
Similarly to selftests, the normal behavior of `qt-testrunner.py` will
fail, trying to pass an argument that is not supported by Catch2 when
running the tests in CI.

To avoid the issue, an exception for the test executables that QDoc uses
was added to `qt-testrunner.py`, using the already available mechanism
that is used for selftests.

Change-Id: I5eb4b75f239eda0c493fb06ece164545d159372b
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
2022-05-19 12:51:46 +02:00
Lucie Gérard
05fc3aef53 Use SPDX license identifiers
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.

Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-16 16:37:38 +02:00
Thiago Macieira
3bcd6026ed util/x86simdgen: update with the copyright header
Matching update done in https://github.com/opendcdiag/opendcdiag/pull/78

Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16ec02591cb5ed70
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-06 10:27:18 -07:00
Joerg Bornemann
6608e63298 Fix build of util/glgen
Don't use QSet::toList(), and fix an obvious typo.
Also, core5compat is now needed due to the use of QRegExp.

Change-Id: I766455996c07d354e97a4ed4939f1774fc449331
Reviewed-by: Laszlo Agocs <laszlo.agocs@qt.io>
2022-05-06 03:48:09 +02:00
Marc Mutz
dffca8bb0e Add a simple Qt 6/CMake-compatible script a la includemocs.py
Perl is dead and I don't speak Python, so if someone wants to port it,
be my guest. I needed something that I can use to roll out includemocs
across all Qt modules, and this is it. It works for me™, so I don't
expect to do much development with it.

This is an automated committer script I've been using so far:

    # SRCDIR=~~~
    # BUILDDIR=~~~~
    cd $BUILDDIR
    find */src -name mocs_compilation.cpp | while read FILE; do
        if grep -qsE '^#include' "$FILE"; then
            DIR="$(dirname "$FILE")"
            LIB="$(basename "$DIR")"
            case "$LIB" in
            Q*)
                LIB="${LIB%%_autogen}"
                ;;
            *)
                LIB="Qt${LIB%%_autogen}"
                ;;
            esac
            DIR="${DIR%/*}"
            path/to/includemocs6.sh "$SRCROOT/$DIR" "$DIR" "$FILE"
            (cd "$SRCROOT/$DIR" && git commit -am "$LIB: includemocs

$(cat "$SRCROOT/commit-msg.txt")" --no-edit)
        fi
    done

If the script cannot associate a moc file with a cpp file, it will
print a warning and continue.

The script tries to include the moc-file right after the
QT_END_NAMESPACE to work around many TUs ending in an #endif from some
#if QT_CONFIG or other. If there's no QT_END_NAMESPACE, it appends
the include and prints a warning.

Fixes: QTBUG-102886
Pick-to: 6.3 6.2
Change-Id: I16c5a9f845777ea2e82f15611b4fdd32f98ce0bb
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-05-05 04:26:19 +00:00
Dimitrios Apostolou
72793ef31d Generalize exclusion list
It seems there are some tests in the various Qt modules that skip
writing the XML testlog file. If there are good reasons for that (for
example test is a custom shellscript) then it should be added to the
DUMB_TESTS list.

Change-Id: Iffb00e1fda42bb8e7338a94c9320e45900db1bc1
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
2022-04-22 15:39:03 +02:00
Dimitrios Apostolou
399dbba775 qt-testrunner: improve error logging
Do not print full stack traces when a controlled exception occurs,
instead print just the exception name and the message.

Change-Id: I5669862d2e98c550d88d1d11fb2bf49f197cfc17
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-04-22 15:38:59 +02:00
Joerg Bornemann
2c364aef4c Remove the qmake2cmake wrapper scripts and document where they are
Pick-to: 6.3
Change-Id: Ib36f4e614845a3b375f4a86239fa7e6e26d7adea
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-04-12 10:34:40 +02:00
Edward Welbourne
681bef22a7 Fix generation of qsimd_x86 files to require no hand-editing
Recent fixes to include the "We mean it" comment in the header also
lead to that comment appearing in the generated .cpp file, which also
lacked the "This is a generated file. DO NOT EDIT." comment. The
generated header also lacked a blank line after the "We mean it"
comment, so include that (and take it out, too, in the .cpp). The
"Please see" line of the "DO NOT EDIT" comment also used the name of
the generator script as seen from the Makefile that drives the
regeneration; replace this with the README.md file that actually
explains how to regenerate the files in corelib/global/.

This amends commit 71af0d7059
and commit b852584556

Change-Id: I4b5b4dbef5954819632bb625d1914a9ec46e15d9
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-04-07 23:04:04 +02:00
Dimitrios Apostolou
0b4446e16f qt-testrunner: be more strict if bad XML log files are written
If a test returns 0 but writes an XML logfile that contains FAIL or a
corrupted XML file, then qt-testrunner considers it a CRASH and exits
with 3.

Previously any test execution returning 0 (success) was considered
a PASS. Changing this behavior with this patch finds a lot of test
crashes on Android (QTBUG-100470), because the tests are run indirectly
on the emulator and the test wrapper process could not detect the crash,
thus returning 0 to qt-testrunner. But the corrupt XML file is caught
now.

Likewise, if a test returns != 0 but the XML logfile contains only PASS,
qt-testrunner considers it a FAIL. This used to be the case but now
tests are added.

Finally changed logging for such cases from INFO to WARNING.

Task-number: QTBUG-100470
Change-Id: I404c9d2211c7de027bf776d1914519d37f513ca1
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
2022-04-05 12:06:23 +03:00
David Skoland
b01ee1d44c Add WASM testrunner
Add a python script that allows us to run wasm tests in CI, along with
the necessary cmake logic to install the script and execute tests
accordingly.

Change-Id: I93b95c115538c4e27b2b833405acab8162be2a8a
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2022-03-31 16:50:25 +02:00
Joerg Bornemann
bac56fd4d0 pro2cmake: Introduce qmake2cmake convenience scripts
Add qmake2cmake[.bat] wrapper scripts that can be used to convert user
projects to CMake.  For now, user projects are internally handled as Qt
examples with one difference: the generation of example-specific
installation code is suppressed.

Fixes: QTBUG-98655
Change-Id: I1a57f6d12efe0bdf383592ab33682a611692db80
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:30 +01:00
Joerg Bornemann
928828b549 pro2cmake: Handle QT_(MAJOR_|MINOR_|PATCH_)VERSION conditions
Task-number: QTBUG-98852
Change-Id: I4c86fff7bbcc6c42cd04094f2409c3d04779597c
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:30 +01:00
Joerg Bornemann
0dc374c9d8 pro2cmake: Remove INSTALL_LOCATION from qt_add_qml_module calls
This function does not handle installation.

Change-Id: I02fdc244f49b5935aa1ac51bff8a25970ad3a335
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:30 +01:00
Joerg Bornemann
3954b7385e pro2cmake: Generate find_package call for Qt5/Qt6 for examples
This makes the QT_VERSION_(MAJOR|MINOR|PATH) variables available.

Task-number: QTBUG-98852
Change-Id: I7e40f2a7ac09975ce21e45cda384af928e1fa629
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:30 +01:00
Joerg Bornemann
eab2a57b37 pro2cmake: Handle optional Qt modules
...and write a separate find_package(Qt6 OPTIONAL_COMPONENTS Foo Bar)
call for those.

Task-number: QTBUG-96799
Change-Id: I3386487774c386edde6767dca92ce433bfed906e
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:30 +01:00
Joerg Bornemann
50f3b1c519 pro2cmake: Add a blackbox test for converting from qmake to CMake
If you can set the environment variable DEBUG_PRO2CMAKE_TEST_CONVERSION
to 1, the output of pro2cmake is written to the temp directory.  This
helps analysing test failures.

Change-Id: Ida42c5b76a67172d00ce0d2488adc7fb376c6b11
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:30 +01:00
Joerg Bornemann
e60c41ac0a pro2cmake: Enable AUTOUIC only if the project has FORMS
Change-Id: I1bf232fa3c389eb86707d3af04de6cf9eb09451a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:29 +01:00
Joerg Bornemann
1c2f6d2ace pro2cmake: Write only one find_package call for Qt packages
Task-number: QTBUG-96799
Change-Id: I1eb8ac05f360b74e5ae1217b5535a33227b5084b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:29 +01:00
Joerg Bornemann
f5a8d70dee pro2cmake: Add LibraryMapping.components
LibraryMapping.components is a list that holds the sub-components of a
CMake package.  Before, we held this information in
LibraryMapping.extra as find_package argument.

A subsequent patch will make use of LibraryMapping.components and
doesn't have to do find_package argument parsing.

Change-Id: Ie0d317245fb6ec1511e06b2e14c364292fced63a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:29 +01:00
Joerg Bornemann
b28f436cd4 pro2cmake: Make generated examples use qt_add_qml_module()
We prefer it when the CMakeLists.txt project files for examples use
qt_add_qml_module() when adding resource files, rather than
qt6_add_resource().

A bit of refactoring was needed to re-use the code that extracts
resource information from the .pro file.

The new function write_qml_module is now responsible for writing
qt_add_qml_module calls.

Task-number: QTBUG-96799
Change-Id: I74dc2d681dcf4fc848e1147b3232ce9a9e0946c9
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-08 03:45:29 +01:00
Kai Köhne
71af0d7059 Re-add missing 'we mean it' header
This got lost in commit b852584556 .
Also augment util/x86simdgen/header so that the header is
automatically included when qsimd_x86_p.h is re-generated.

Change-Id: I3e59b983f78b8c1aced3757e1aa5dceb6d653d97
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-03-07 14:04:25 +01:00
Joerg Bornemann
f56c068ee0 pro2cmake: Fix indentation of INSTALL_EXAMPLESDIR assignment
Change-Id: If1f5d8b5afc76fac62fdf9836a17c4ab87ddc1b1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-03 15:56:08 +01:00
Joerg Bornemann
3d6c86ec57 pro2cmake: Fix static type errors in condition_simplifier_cache.py
Mypy reported the following:

condition_simplifier_cache.py:111: error: Argument 1 has incompatible type "*List[str]"; expected
"Optional[float]"
            file_handle = file_open_func(*file_open_args, **file_open_kwargs)
                                          ^
condition_simplifier_cache.py:111: error: Argument 2 has incompatible type "**Dict[str, object]";
expected "Union[str, Path]"
            file_handle = file_open_func(*file_open_args, **file_open_kwargs)

Fix by calling portalocker.Lock directly.

Change-Id: I538319791b1a4a1b6dffac0544d87c705e8809a7
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-03-03 15:56:08 +01:00
Alexandru Croitor
34a4fe0166 CMake: Rename FindZSTD to FindWrapZSTD
And the target ZSTD::ZSTD to WrapZSTD::WrapZSTD.
This should allow building Qt with the
-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON set.

Pick-to: 6.2 6.3
Fixes: QTBUG-100537
Change-Id: I748601e4ad6f518323bf1034d6fc1de582c815e1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-02-28 23:08:20 +01:00
Joerg Bornemann
268767e9a1 pro2cmake: Update README.md
Mention that flake8 and black can be run via Makefile targets.

Remove the outdated list of flake8 warnings.  The Makefile ignores more
warnings/errors, and it doesn't seem to be useful to duplicate this list
here.

Also remove the description of the black tool's arguments for the same
reason.

Change-Id: I941c3ab68b7a3d2477f7fbb5d28603987d0b2cab
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-02-28 15:41:22 +01:00
Joerg Bornemann
dece6f17e6 pro2cmake: Set a project version for examples
For example projects, change the generated project() call to
   project(foo VERSION 1.0 LANGUAGES CXX)

Some CMake API derives default values from the project version, and it's
generally advisable to set a project version number.

The version number is read from qmake's VERSION variable.  That's
actually a target version number, but it might be the right thing in
most cases.  Fall back to version 1.0 if VERSION is not set.

Task-number: QTBUG-96799
Change-Id: Ia0c551cf62621eb217e1dd541dcbd8945f78138e
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-02-28 15:41:18 +01:00
Alexandru Croitor
2389aaf8c7 pro2cmake: Handle qmake condition operator precedence
Unfortunately qmake does not have operator precedence in conditions,
and each sub-expression is simply evaluated left to right.

So c1|c2:c3 is evaluated as (c1|c2):c3 and not c1|(c2:c3). To handle
that in pro2cmake, wrap each condition sub-expression in parentheses.

It's ugly, but there doesn't seem to be another way of handling it,
because SymPy uses Python operator precedence for condition operators,
and it's not possible to change the precendece.

Fixes: QTBUG-78929
Change-Id: I6ab767c4243e3f2d0fea1c36cd004409faba3a53
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-02-28 15:41:11 +01:00
Joerg Bornemann
6708fad936 pro2cmake: Fix test_realworld_comment_scope
Change the expected value from None to [], because that's the value of
the if branch.  It has been like that since v6.0.0 at least.

Change-Id: Iefdb22a772fc5540ad5a38566be5a7f529e00cb1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-02-28 15:40:43 +01:00
Joerg Bornemann
7b8913b1ba pro2cmake: Remove superfluous call to _simplify_flavors_in_condition
If the flavors argument is an empty list, this function becomes the
identity function.

Change-Id: I534df079578ff27d24ae15760ea12464e3961f93
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-02-28 15:40:39 +01:00
Joerg Bornemann
e33c88a36d pro2cmake: Fix test_scope_handling.py
When the android-embedded scope was removed in
7a4b586f4b, the conditions in
test_scope_handling.py were adjusted following the laws of logic.
However, the scope handling code does not follow the same laws.

Effectively revert the part of said commit in test_scope_handling.py but
use "UNKNOWN_PLATFORM" instead of "ANDROID_EMBEDDED".

Change-Id: Ic090451e2a28b50f5be5668503e216cbe3871633
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-02-28 15:40:36 +01:00
Joerg Bornemann
8a6899239a pro2cmake: Re-format sources with black
...to have a consistent baseline for further improvements.

Change-Id: Iba8e83a7a5cf5ca0cdf509f79e7d2dc2d8f42fec
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-02-28 15:40:32 +01:00
Thiago Macieira
d581778251 qedidvendortable.py: include something before using macros
Since it's a private header, qglobal_p.h makes sense.

Change-Id: Ibf4acec0f166495998f7fffd16d5d8a38ee8f1f8
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2022-02-24 12:07:00 -08:00
Ievgenii Meshcheriakov
7a58ca2d3a Remove util/publicSuffix
This program was replaced by psl-make-dafsa.

Task-number: QTBUG-95889
Change-Id: If52d92734ad362364c6250473281886e1ea5545d
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2022-02-24 19:36:26 +01:00
Thiago Macieira
b2298b7e94 qsimd_p.h: let the generator script generate the ARCH target strings
__attribute__((target("arch=xxxx"))) does not work because the compilers
(GCC at least) don't test the CPU features that they are targeting, so
we keep getting "inline failed" compiler errors.

GCC bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90129

Upstream patch: https://github.com/opendcdiag/opendcdiag/pull/59

Change-Id: I6fcda969a9e9427198bffffd16cea09fda4406d2
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
2022-02-19 13:40:21 -08:00
Thiago Macieira
b852584556 qsimd: update the generator script from OpenDCDiag
I'd been making changes to that and improving it for the past 2 years
without bringing it back into Qt.

The list of features is mostly the same, except:
- removed TSX features
- removed features specific to Xeon Phi processors
- added CET and AVX512FP16 features
- added the bit for hybrid CPU detection

See matching update at https://github.com/opendcdiag/opendcdiag/pull/49

Change-Id: I6fcda969a9e9427198bffffd16ce860b5a38aece
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
2022-02-18 15:46:41 -08:00
Dimitrios Apostolou
c37d0e84bc Clean up QNX platform configs and improve testrunner scripts
QNX requires its own TESTRUNNER script in order to SSH to the QEMU guest
and run the tests. Simplify it a lot and fix the way it passes arguments
so that they retain spaces and quotation.

Also wrap it with the generic TESTRUNNER script for CI, which is
qt-testrunner.py, so that tests can re-run in case of flakiness.

Delete prefix.sh as the environment variables are now properly set in
the Coin platform config files.

Avoid executing extra chmod commands to make scripts executable,
but use the right Coin instruction for that. For reference, we
use 493 for file mode which equals to 755 in octal, as Coin expects
the file mode in decimal.

Change-Id: Ife4d1caef606f48b92ba1da1cfb14ec0dea11ef2
Reviewed-by: Pasi Petäjäjärvi <pasi.petajajarvi@qt.io>
2022-02-14 12:56:06 +01:00
Jani Heikkinen
994fd04b2b Bump version to 6.4.0
Change-Id: Ie0e2133d6c9125b901364c979c60b6efd585f026
Reviewed-by: Qt Submodule Update Bot <qt_submodule_update_bot@qt-project.org>
2022-01-31 17:20:06 +02:00
Dimitrios Apostolou
b013f94086 Write XML test log files even for the flaky test re-runs
Previously the qt-testrunner script was avoiding to write XML log files
for the flaky test re-runs, since parsing them always provides limited
information about what went wrong (for example unparseable files) and
might miss issues completely (for example a crashed test might not even
write a log file), or even mislead if multiple different data points
about a single testcase end up into the database..

But currently Coin is the system that uploads test information to the
testresults database and expects to find such log files in an attempt to
send as much information as possible regarding failures.

Re-enabling these log files will allow Coin to deduce as much
information as possible about the test runs.

This is a temporary step at least, until the test-uploading logic is
separated from Coin. Then it will be easier to find the full information
about what happened with flaky or crashed tests by special logs from
qt-testrunner.py, which is the test re-running orchestrator and
therefore has more knowledge about what went wrong.

Fixes: QTQAINFRA-4754
Change-Id: I69e4469cbe6c0a6ca5b2473eaf8d034632564342
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
2022-01-30 01:00:35 +01:00
Dimitrios Apostolou
b4d9d5c89c qt-testrunner: do not try to re-run initTestCase and others
Some function names are special for qtestlib, in the sense that they can
not be specified as a command line argument to run individually.

In such cases qt-testrunner treats the failure specially and tries once
to re-run the full test executable.

Fixes: QTBUG-89011
Change-Id: I0cc25f91c57374e5ac65ade10e2e223fe969f211
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
2022-01-30 01:00:35 +01:00
Joerg Bornemann
eced23b6a8 Don't set CMAKE_AUTORCC in pro2cmake.py
To tool generates calls to qt6_add_resources instead, and we don't want
to promote the usage of CMAKE_AUTORCC.

Task-number: QTBUG-87643
Change-Id: I58458416514949a6ffc9b93c7eb6d24bc2ed01ca
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-01-21 19:54:13 +01:00
Joerg Bornemann
dc2c590b1a pro2cmake: Fix exception with newer pyparsing module
The pyparsing module's debug action properties have been renamed.
Check the existence of the attributes before assigning.

Change-Id: I8fff652304a0af215c56f54b63d613a1f6a5207a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-01-21 19:54:13 +01:00
Dimitrios Apostolou
5180e70dbe qt-testrunner: be more clear that it was the test that crashed
Task-number: QTBUG-99970
Change-Id: Id2d01b6ab7d428356d9dfc953107014791393d35
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2022-01-18 19:43:54 +01:00
Edward Welbourne
755b0aa681 Turn a condition into a lambda to save needless evaluation
The public suffix list scanner's utf8encode()'s main loop always
worked out whether a character is a hex digit, even when it didn't
need to know. Package the computation in a lambda and only test it
when it is needed. Also assert non-empty input generates non-empty
output.

Change-Id: Iaf48aad382624e421cea9c9cdb8bba5fc47b1596
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
2022-01-18 16:23:56 +01:00
Dimitrios Apostolou
3acfd39653 qt-testrunner: disable JUnit XML test logs
It causes crashes because of interleaved execution of Quick tests.

Task-number: QTBUG-98350
Change-Id: I61e26f5da5617e06810db8eb7991f17eecdd3acb
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-01-12 16:14:18 +01:00
Dimitrios Apostolou
98f1034f96 Properly wait for the Android emulator to fully boot
It had been noticed that sometimes the Android emulator in our CI took
several minutes to fully boot. It turns out that this behavior can be
reproduced locally by removing the image files under $HOME/.android
especially the "userdata-qemu.img.qcow2" file.

Then the emulator goes through several reboots until fully booted with a
full set of packages installed. We discovered that the property that
signifies it is finished is dev.bootcomplete=1. So we now check for this
flag and remove the other heuristics we had.

We also disable the debug output to avoid hundreds lines of
logs. Instead we selectively print the values that the emulator returns,
every second until full boot is detected.

We increase the Coin timeout for the shell script, from 5min to 10min,
since it has been measured that it takes about 2.5min on a good day, and
the script itself retries several times to restart the emulator in case
of failure.

Finally we adjust the coding style a bit to be more consistent.

Pick-to: 6.3 6.2
Fixes: QTQAINFRA-4681
Change-Id: I77062dceb91477e957696c89bfacb4ebabc34c1f
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2022-01-07 19:34:50 +01:00
Dimitrios Apostolou
c8e0a742fb Wait for boot to be fully complete with all packages installed
It happens that the android emulator supposedly "finishes" booting and
getprop shows bootanim=stopped and boot_completed=1. But sometimes not
all packages have been installed (`pm list packages` shows only 16
packages installed).

After around half a minute the boot animation starts spinning
(bootanim=running) again despite boot_completed=1 all the time. After
some minutes the boot animation stops again and the list of packages
contains 80 packages, among which is also the "development" package.
Only then is the device ready for `adb install` of custom packages.

This should be the last time we see the message:

Error: Could not access the Package Manager.  Is the system running?

Just for completeness, we also properly disown the emulator process and
its file descriptors, since the parent shell dies but the process stays.

Pick-to: 6.3 6.2
Fixes: QTQAINFRA-4681
Change-Id: I2b9d4bdc3dac0f10d09d4f09c83b4028ebc31f48
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
2022-01-06 17:14:58 +01:00
Dimitrios Apostolou
8bf1beb9bd Do not hide android emulator output
It might print useful messages, even though they might come mixed with
other test messages.

However we can't leave stdout/stderr as is, because Coin agent hangs at
the end of the shell script, waiting for them to close. So we just
redirect them to a file.

Pick-to: 6.3 6.2
Change-Id: I797af78786b7df31131b3c3261e1c1fc00e5d460
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
2022-01-06 14:34:15 +01:00
Dimitrios Apostolou
0aaf514042 Wait for android emulator to be up and running properly
Pick-to: 6.3 6.2
Fixes: QTQAINFRA-4681
Change-Id: Icfa75b6982964970172726379e5a2a2bb640f8bf
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
2022-01-06 14:34:09 +01:00
Dimitrios Apostolou
24572fccae Enable verbose logging in the last re-run of a failing test
Fixes: COIN-728
Change-Id: I08802e377e26e5dd7f7d1c44b5efe4280b09f957
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
2022-01-03 17:40:35 +01:00
Dimitrios Apostolou
63ffae3fa3 Introduce qt-testrunner.py
Script that wraps Qt test execution in order to iron-out flakiness.

Task-number: QTBUG-96353
Change-Id: Ie8ee780e3f4f8d086c080a7784f9f68fd1867be2
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
2022-01-03 16:32:56 +01:00
Ievgenii Meshcheriakov
4f53c703e4 QLocale: Extend support for language codes
This commit extends functionality for QLocale::codeToLanguage()
and QLocale::languageToCode() by adding an additional argument
that allows selection of the ISO 639 code-set to consider for
those operations.

The following ISO 639 codes are supported:
    * Part 1
    * Part 2 bibliographic
    * Part 2 terminological
    * Part 3

As a result of this change the codeToLanguage() overload without
the additional argument now returns a Language value if it matches
any know code. Previously a valid language was returned only if
the function argument matched the first code defined for that
language from the above list.

[ChangeLog][QtCore][QLocale] Added overloads for codeToLanguage()
and languageToCode() that support specifying which ISO 639 codes
to consider.

Fixes: QTBUG-98129
Change-Id: I4da8a89e2e68a673cf63a621359cded609873fa2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-12-09 03:45:08 +01:00
Morten Johan Sørvig
d3646d2b05 Say hello to qtwasmserver
Development web server for web applications. Supports
http and https. Sets COOP and COEP headers.

The web server script supports certificate generation using
mkcert (github.com/FiloSottile/mkcert). Briefly, mkcert
supports generating server certificates for the current
local ip address(es), using a certificate authority
which can be installed on devices where the app should
run.

The COOP and COEP headers are required to enable the
SharedArrayBuffer API, which is required for Emscripten’s
pthreads implementation.

The server serves the current directory on localhost
by default. Use the “-a” argument to bind to an additional
address, or "--all” to bind to all IPv4 addresses found on
local network interfaces. Change the directory by passing
it as a positional argument.

Task-number: QTBUG-79087
Change-Id: Id0cba649e42af53ed8106e336a77e78398bcf901
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: David Skoland <david.skoland@qt.io>
2021-12-06 16:36:09 +01:00
Ievgenii Meshcheriakov
aee0021fc8 QLocale: Add support for Kaingang and Nheengatu languages
Update the locale generation script to support Kaingang and
Nheengatu languages. These are new in CLDR v40. Regenerate
the locale data.

Task-number: QTBUG-94358
Change-Id: I5195d5161d8c4d9f17129bbcfde39dfd3fcf1cd5
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-11-10 00:36:12 +01:00
Ievgenii Meshcheriakov
2c505eea01 locale_database: Add entry for 'South Sudan Standard Time'
This timezone is new in CLDR 40, Olson database calls it Africa/Juba.
The offset is UTC+2.

Reference: https://techcommunity.microsoft.com/t5/daylight-saving-time-time-zone/2021-time-zone-updates-for-republic-of-south-sudan-now-available/ba-p/2234981

Task-number: QTBUG-94358
Pick-to: 6.2
Change-Id: Ib70dbd9b472eb9cf8cb62a0eb5e241199148c077
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-11-10 00:36:11 +01:00
Ievgenii Meshcheriakov
7d7663cfc4 Use a HTTPS URL for the CLDR download link
FTP is insecure and is not supported by modern browsers anymore.

See also: https://mywiki.wooledge.org/FtpMustDie

Change-Id: Iad65d29912e79a4f3fadb9317bb5d9c5fe9b68d7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-11-05 19:32:23 +01:00
Joerg Bornemann
6ab16d52d2 configure: Remove unused facility to define variable assignments
The function qt_commandline_assignment and everything related to it is
removed from configure.  It was only used in qtbase, and all usage has
been removed.

More general variable assignments will be added in a subsequent commit.

Task-number: QTBUG-88210
Change-Id: I7cfa782e89914f2b0dc0277c46e425c8a825557e
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-11-01 22:32:01 +01:00
Assam Boudjelthia
a81f24ada5 Rename deprecated ANDROID_SDK_HOME to ANDROID_SDK_ROOT
Pick-to: 6.2
Task-number: QTBUG-97002
Change-Id: I777491f542e5388e04ed4c1b7ff5d9db75f9e778
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2021-10-22 17:21:10 +02:00
Ievgenii Meshcheriakov
826fc8c9bd Update UCD to Revision 28
This corresponds to Unicode version 14.0.0.

Added the following scripts:

    * CyproMinoan
    * OldUyghur
    * Tangsa
    * Toto
    * Vithkuqi

Full support of these scripts requires harfbuzz version 3.0.0,
this version adds support for Unicode 14.0:

    https://github.com/harfbuzz/harfbuzz/releases/tag/3.0.0

With this release 10 test cases in tst_qurluts46 were fixed, one
additional test case is failing in tst_qtextboundaryfinder and
is commented out. In total 62 line break test cases and 44 word
break test cases are failing.

A comment in src/corelib/text/qt_attribution.json was updated to
include the URL of the page containing UCD version number.

Fixes: QTBUG-94359
Change-Id: Iefc9ff13f3df279f91cbdb1246d56f75b20ecb35
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-18 16:45:10 +00:00
Joerg Bornemann
f19668ce65 Don't use libs in /usr/local for configure tests by default on macOS
Since commit f3c7d22dd0 we do not use
libraries from /usr/local and other non-system locations on macOS.  But
our configure tests still did.  This led to discrepancies between
find_package calls in configure tests and the Qt project itself.

Mentioned commit removed /usr/local and friends from
CMAKE_SYSTEM_PREFIX_PATH.  But we can't pass this variable to the
configure tests, because CMake sets it up and overwrites our value.

Pass CMAKE_SYSTEM_PREFIX_PATH and CMAKE_SYSTEM_FRAMEWORK_PATH as
QT_CONFIGURE_TEST_CMAKE_SYSTEM_{PREFIX|FRAMEWORK}_PATH variables to
tests.

Tests with separate project files that call find_package() must add code
like this after the project() command:

if(DEFINED QT_CONFIGURE_TEST_CMAKE_SYSTEM_PREFIX_PATH)
    set(CMAKE_SYSTEM_PREFIX_PATH
        "${QT_CONFIGURE_TEST_CMAKE_SYSTEM_PREFIX_PATH}")
endif()
if(DEFINED QT_CONFIGURE_TEST_CMAKE_SYSTEM_FRAMEWORK_PATH)
    set(CMAKE_SYSTEM_FRAMEWORK_PATH
        "${QT_CONFIGURE_TEST_CMAKE_SYSTEM_FRAMEWORK_PATH}")
endif()

Adjust pro2cmake accordingly.

Task-number: QTBUG-97076
Change-Id: Iac1622768d1200e6ea63be569eef12b7eada6c76
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-10-08 23:15:31 +02:00
Ievgenii Meshcheriakov
906cb8d4ef Remove util/harfbuzz/update-harfbuzz
This is not the correct script to update harfbuzz. The correct one is
src/3rdparty/harfbuzz-ng/import_from_tarball.sh

Change-Id: Ic9ed43cae591cd6957effb10f557166c43498ad9
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
2021-10-05 21:53:10 +02:00
Ievgenii Meshcheriakov
394f8a9c4b Unicode: Add script to facilitate UCD update
Add a script that downloads UCD data for a given Unicode version,
unpacks it, and copies the used files to appropriate locations inside
the Qt source code.

Also update the README and use an HTTPS link for the UCD data file.
FTP links are no longer supported by some browsers.

Task-number: QTBUG-94359
Change-Id: I2aa70a588f675e411fa6b3ce5b4444a7c07ed707
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-05 20:38:02 +02:00
Alexandru Croitor
9665929706 pro2cmake: Generate public CMake API code for projects by default
Previously pro2cmake assumed that all .pro were internal Qt projects
like qtbase/src/corelib.pro and generated private api calls like
qt_internal_add_module().

Public CMake API calls (like qt_add_executable) were only generated if
an --is-example command line flag was passed to the script or if the
.pro file was located under the examples subfolder of a qt repo source
directory (indicated by the presence of a .qmake.conf file).

Change the logic to always generate public CMake API code by default,
unless a .cmake.conf/.qmake.conf file is encountered with a valid repo
module version inside, in which case apply the old heuristic of
checking for the ./examples subfolder.

The intention is to have a sensible default so that Qt users can use
the script to more easily migrate their qmake projects without having
to explicitly specify the --is-example flag.

Pick-to: 6.2
Task-number: QTBUG-96799
Change-Id: I38f4f1b349a5b9688cf7bc5914d4fde72e660a98
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-09-27 21:36:36 +02:00
Alexandru Croitor
124f7c681f pro2cmake: Augment searching of .qmake.conf with .cmake.conf
Adapt to fb656c036d which removes
.qmake.conf files in repos by searching for either .qmake.conf or
.cmake.conf.

The locations of those files are only used to compute relative paths
based on the directory where the conf file is.

For now, remove the warning if a conf file is not found, because we
intend to change the script to generate user project code by default,
instead of Qt internal project code and the warning looks scary.

Pick-to: 6.2
Task-number: QTBUG-96799
Change-Id: I301e93d2ce5c05a52877eb6ed9eb9cee54fc5b15
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-09-27 21:36:33 +02:00
Joerg Bornemann
a0e56294c1 Skip unnecessary commands when cross-building tools
Introduce a new macro qt_internal_return_unless_building_tools which
simply calls return() if tools are not built.  This macro is supposed to
be called after qt_internal_add_tool().

Using this macro avoids having to special-case code for when
qt_internal_add_tool() creates imported targets in cross-builds.

Adjust pro2cmake accordingly.

Task-number: QTBUG-85084
Change-Id: I9e1c455c29535dd8c318efa890ebd739c42effc1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-09-27 16:34:23 +02:00
Alexandru Croitor
0da123d67b CMake: Bump almost all cmake_minimum_required calls to 3.16
Needed for subsequent change that will check and error out if the
version is lower than 3.16. We do that to ensure all policies
introduced by CMake up to version 3.16 have their behavior set to
NEW.

Pick-to: 6.2
Task-number: QTBUG-95018
Change-Id: Ieaf82c10987dd797d86a3fd4a986a67e72de486a
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
2021-09-22 19:36:49 +02:00
Edward Welbourne
7a4b586f4b Remove conditioning on Android embedded
It is no longer handled separately from Android.
This effectively reverts commit 6d50f746fe

Change-Id: Ic2d75b8c5a09895810913311ab2fe3355d4d2983
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
2021-09-17 17:30:14 +02:00
Ievgenii Meshcheriakov
8be862687d unicode: Fix typo s/supersting/superstring/
Thanks to Konstantin Ritt for spotting it.

Change-Id: Ia3b5c4103b315cdb690fcd8b42239f000acdbef0
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-09-15 21:58:12 +02:00
Edward Welbourne
ce868ffb7c Remove trailing zero rows from qtimezoneprivate_data_p.h's tables
They are not needed. Iterations over the table track their sizes.
The size-of-table constants just needed their -1s removed.
Incidentally use std::size() rather than sizeof(array)/sizeof(element).

Change-Id: Ie20eef9f6f5786d93c10b830a87e006d3c5bcc1a
Reviewed-by: Ievgenii Meshcheriakov <ievgenii.meshcheriakov@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
2021-09-10 19:33:06 +02:00
Alexandru Croitor
8a66e3cfe5 pro2cmake: Fix reported make flake8 issues
pro2cmake.py:458:5: F841 local variable 'is_parent_path' is assigned
to but never used

Pick-to: 6.2
Change-Id: I1c61efe5cf568fa0c99b07eed28b93d3444a5fac
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2021-09-09 10:21:27 +02:00
Alexandru Croitor
21a281d962 pro2cmake: run make format to fix styling issues
Pick-to: 6.2
Change-Id: I718e34d628576bb709096c0672da3a69b71ae8df
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2021-09-09 10:21:27 +02:00
Oliver Eftevaag
8337493301 pro2cmake.py: generate examples with properties in qt_add_executable()
This patch will slightly change the output of CMakeLists.txt files that
are generated for examples.

* set_target_properties() will no longer be added to the top-level
  scope. If the WIN32 and MACOSX_BUNDLE properties should be added,
  they will instead be added to qt_add_executable().

* The version in cmake_minimum_required() will now be 3.16, rather than
  3.14.

Pick-to: 6.2
Change-Id: I79e1865dace5538d2b7ff264da02f9e28a655ae9
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-09-08 15:43:16 +02:00
Ievgenii Meshcheriakov
5f3f073126 unicode: Build IDNA map superstring greedily
This slightly reduces memory required for mapping tables:

    uncompressed size: 1146 characters
    consolidated size: 904 characters
    memory usage: 47856 bytes

Task-number: QTBUG-85323
Change-Id: Ic960789e433e80acf1a4e36791533a1c55a735c8
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-09-03 14:43:16 +02:00
Ievgenii Meshcheriakov
2d78b71fc6 unicode: Pack 2 QChar's into one entry for IDNA mapping
Store up to 2 QChar's for mapping values inside the mapping
table itself. This reduces the size of the superstring for
other mapping values.

results:
    uncompressed size: 1146 characters
    consolidated size: 1001 characters
    memory usage: 48050 bytes

Task-number: QTBUG-85323
Change-Id: I922a6d2037551d0532ddae1a032ec1a9890f40a7
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-09-03 14:43:16 +02:00
Ievgenii Meshcheriakov
ca1eeb23fa unicode: More compact IDNA mapping tables implementation
This implementation stores mapping that are 1 QChar long
inside the mapping tables, the longer mappings are stored as
an offset and length pairs pointing into the common superstring
of all such mapping values.

Size comparison with the existing implementation follows.

old:
    max mapping length: 6
    memory usage: 103608 bytes

new:
    uncompressed size: 3250 characters
    consolidated size: 2367 characters
    memory usage: 50782 bytes

Task-number: QTBUG-85323
Change-Id: I9f2e32438dd463457e0fcd783136bb17145e27a8
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-09-03 14:43:16 +02:00
Ievgenii Meshcheriakov
4e460aa3f7 tests: Add test for UTS #46 implementation using Unicode data
The new test is called tst_qurluts46. It verifies QUrl::{to,from}Ace()
functionality using the data from IdnaTestV2.txt supplied by Unicode.

The file was downloaded from
https://www.unicode.org/Public/idna/13.0.0/IdnaTestV2.txt

Task-Id: QTBUG-85371
Change-Id: I4c6a4942ef6018dafc90cb84ef73f6b2614566d7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-09-01 14:15:45 +02:00
Ievgenii Meshcheriakov
2afe1a3c19 unicode: Generate tables for IDNA/UTS #46
Update the Unicode data processing tool to generate properties
and mapping tables needed to implement UTS #46
(https://unicode.org/reports/tr46/). The implementation extends
the standard to allow usage of underscores in URLs. This is done
for compatibility with DNS-SD and SMB protocols.

The data file needed to generate the new properties was taken from
https://www.unicode.org/Public/idna/13.0.0/IdnaMappingTable.txt

Task-number: QTBUG-85323
Change-Id: I2c303bf8a08aefb18a7491fb9b55385563bfa219
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-08-26 16:55:05 +02:00
Alexandru Croitor
c13b98d9bc CMake: Bump project versions
Fixes: QTBUG-95454
Change-Id: I2467d3ae27b54424e59a7a4ab00d364eaec517d5
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
2021-08-02 12:42:29 +02:00
Ievgenii Meshcheriakov
e9519d207e locale_database: Use context manager interface to update source files
Use context manager interface (with statement) to atomically update source
files. This ensures that all files are properly closed and the temporary
file is removed even in case of errors.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I18cd96f1d03e467dea6212f6576a41e65f414ce1
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
2021-07-20 16:51:51 +02:00
Ievgenii Meshcheriakov
8f06d3f938 locale_database: Use NamedTemporaryFile for temporary files
Using NamedTemporaryFile instead mkstemp + fdopen simplifies the code.
It also makes it easier to switch to using context managers for handling
source file modification.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ibeae840ac6dde3d0b49cd7f985cfa6cd775b7f47
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-19 22:05:54 +02:00
Ievgenii Meshcheriakov
2d0c2c9f1c locale_database: Use pathlib to manipulate paths in Python code
pathlib's API is more modern and easier to use than os.path. It
also allows to distinguish between paths and other strings in type
annotations.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ie6d9b4e35596f7f6befa4c9635f4a65ea3b20025
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-19 22:05:54 +02:00
Ievgenii Meshcheriakov
5ef5dce53b locale_database: Use argparse module to parse command line arguments
arparse is the standard way to parse command line arguments in Python.
It provides help and usage information for free and is easier to extend
than a custom argument parser.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I1e4c9cd914449e083d01932bc871ef10d26f0bc2
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-16 19:04:20 +02:00
Ievgenii Meshcheriakov
41458fafa0 locale_database: Use f-strings in Python code
Replace most uses of str.format() and string arithmetic by f-strings.
This results in more compact code and the code is easier to read
when using an appropriate editor.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I3409f745b5d0324985cbd5690f5eda8d09b869ca
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-16 19:04:20 +02:00
Ievgenii Meshcheriakov
65a0e04072 locale_database: Add schema for intermediate locale data files
The schema is in RelaxNG Compact syntax. It can be used to validate
files produced by the cldr2qlocalexml.py script and also gives an
overview of the file format.

Change-Id: I344978f2201c5e67e236ab580a12ad33262f33cb
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-16 18:27:28 +02:00
Ievgenii Meshcheriakov
2f4868548b locale_database: Use super() to call base class methods
This is the standard way to call base class methods in Python 3 and
it is shorter than the custom one used now.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ifaff591a46e92148fbf514856109ff794a50c9f7
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-15 17:06:54 +02:00
Ievgenii Meshcheriakov
53382b7b07 locale_database: Don't use u prefix for strings in python files
This prefix is useless with Python 3.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: Ic008d53fe506865759e9a5003f439f7ac107b9e6
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-15 17:06:53 +02:00
Ievgenii Meshcheriakov
b02d17c5c0 Convert CLDR scripts to Python 3
The convertion is moslty done using 2to3 script with manual cleanup
afterwards.

Task-number: QTBUG-83488
Pick-to: 6.2
Change-Id: I4d33b04e7269c55a83ff2deb876a23a78a89f39d
Reviewed-by: Cristian Maureira-Fredes <cristian.maureira-fredes@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-07-15 17:06:53 +02:00