Commit Graph

97 Commits

Author SHA1 Message Date
Pasi Petäjäjärvi
11999a0aaf CI: Enable building autotests for QNX QEMU
Task-number: QTBUG-87628
Pick-to: 6.2
Change-Id: Id88daa2a08491b71d9d113ab02d0ca7806872a0e
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Marianne Yrjänä <marianne.yrjana@qt.io>
2021-10-27 10:30:02 +00:00
Pasi Petäjäjärvi
6c22f7e39c CI: Add QNX qemu support
* Script used to parse env variables and run test over ssh on qemu
* Script to start qnx qemu
* Script used to set env variables for ssh so they are correct in qemu
* Add b2qt sdk host toolchain to path for actual qemu binaries

Task-number: QTBUG-87628
Pick-to: 6.2
Change-Id: I951172211c2963654899491568771310a410e616
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
2021-10-27 13:30:01 +03:00
Topi Reinio
439f6c9576 coin: Add instructions to generate docs during testing
A single configuration provides qdoc and qtattributionsscanner via
provisioning. The environment variables instruct QDoc to run in
testing mode; it ignores linking warnings (since we don't have .index
files for external documentation modules) and terminates with non-zero
exit code if documentation warnings exceed the limit set in the
documentation config.

The provisioned binaries are portable, packaged together with their
dependencies. As they are installed under /opt, the instructions
create symlinks to the binaries in the install directory before
building the 'generate_docs' target.

Pick-to: 6.2
Task-number: QTBUG-78069
Fixes: QTBUG-95554
Change-Id: I4a4765250247e2fc6eddd153c36cf8126ddd10ae
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
2021-10-18 16:23:13 +00:00
Heikki Halmet
755f7c7ff2 Add support for Microsoft Visual Studio 2022 Preview version
When official version is available variableValue path need to be
changed.

Task-number: QTQAINFRA-4540
Change-Id: Ie5fe676962297aa5f18f225eb4b8dfe0957671c8
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
2021-09-30 09:32:11 +00:00
Tor Arne Vestbø
8dd14a956c Revert "coin: Generate JUnit result files"
This reverts commit c7ddaa9f58.

Reason for revert: Broke integrations

Change-Id: I21ffb3b16e361a48a5d9b7a528d150e51519623c
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-16 17:02:44 +00:00
Tor Arne Vestbø
c7ddaa9f58 coin: Generate JUnit result files
The JUnit format is easier to feed to tools that don't support
the native Qt Test XML format.

Pick-to: 6.2
Change-Id: Iad677b98953275fd70765acb039f1fb3d0f44dab
Reviewed-by: Simo Fält <simo.falt@qt.io>
2021-09-11 17:32:04 +02:00
Assam Boudjelthia
d9381bc7c3 Enable building and uploading tests for Android targets if tests enabled
Enable building and uploading test artefacts for all modules in case
tests are enabled.

Pick-to: 6.2
Task-number: QTBUG-88846
Change-Id: I30832820dea348187b517e170fa1c5266db85604
Reviewed-by: Toni Saario <toni.saario@qt.io>
2021-08-30 20:51:47 +03:00
Alexandru Croitor
80705298ca CMake: Build a subset of tests when targeting iOS in the CI
Add infrastructure to build cmake auto tests in the CI when targeting
iOS.
Currently the are only CI instructions for qtbase.
More work is needed to make it work for other repos.

With this change, we will build a single Widgets application targeting
the iOS simulator. We can't target the device SDK in the CI because
signing fails due to a missing signing certificate and provisioning
profile.

The Coin instructions will now set a QT_BUILD_ENVIRONMENT=ci env var
whose value will be checked in _qt_internal_test_expect_pass, to
ensure we build for the simulator SDK when using a universal Qt.
Without this, xcodebuild will try to build with the device SDK and
fail to build the project.

Pick-to: 6.2
Task-number: QTBUG-95839
Change-Id: Ib39c9527b107b2004746ccbdc9d9d1d658f88c76
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2021-08-26 16:40:15 +02:00
Toni Saario
2f3b234d6c Apply custom timeouts to cross compilations
Pick-to: 6.2
Change-Id: I93ce6b6a6011a0883820f291121d7374391494c6
Reviewed-by: Michal Klocek <michal.klocek@qt.io>
2021-08-10 12:21:58 +00:00
Simo Fält
aa1433fd94 Make Integrity to use cross compile instructions
Change-Id: I22617a5f7b8971eae679058feaf41ba811e0a3dd
Reviewed-by: Kimmo Ollila <kimmo.ollila@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-07-15 18:56:23 +03:00
Simo Fält
c1b6fdd0d9 Add instructions to collect results from COIN test run
Change adds coin_ctest_runner.py which is used to pass test arguments
for testcase. Results are stored to xml files, which are then parsed by
COIN. Data that we are interested in, is injected to influx db.

Task-number: COIN-643
Change-Id: Ib1c99fc8aacc6a6b5863d282f3435fe1cfdcd013
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-07-01 08:46:57 +00:00
Assam Boudjelthia
c6e13056b5 Increase the timeout for test targets for enforce exit code yaml
Amends 01e35a2145.

Task-number: QTBUG-94127
Change-Id: Ic464b2df32bee045f30cf6532c672a6450c433fe
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-06-04 18:16:09 +00:00
Heikki Halmet
2a8cc9926e Avoid setting DCMAKE_C_COMPILER and DCMAKE_CXX_COMPILER twice
If these values already exist do not add them to configure. This also
allows setting these values from platform configurations

Change-Id: I2f2aea90d91fa3436953e0c4103f0ea204b3564b
Reviewed-by: Toni Saario <toni.saario@qt.io>
2021-06-02 21:02:45 +00:00
Assam Boudjelthia
01e35a2145 Increase the timeout for test targets
The Android test VMs take almost the same amount of time of that set to
the test timeout, this is causing it to be killed by COIN in some cases.

This is for example a stat from a successful test instance:
   Total Test time (real) = 7110.20 sec

It's very close to 7200 which is the timeout value. The VM is taking
that long because the tests are only half built, the projects' shared
libs are built during the Build step of the integration, and the actual
apk (Gradle build) is done later when doing a ninja tst_name_check.
This latter behavior should be changed so that the apks are built before
going to testing.

So it makes sense for now to increase the timeout and avoid having to
restage multiple times.

Task-number: QTBUG-94127
Change-Id: I31783bcc1255c7821134e174ab6b0fb199000272
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2021-06-02 21:02:41 +03:00
Nicholas Bennett
190ef1c23b Update the COIN build instructions to also build creator help file
Changed the cmake build target from "generate_docs" to "docs"

Pick-to: 6.1
Fixes: QTBUG-93174
Change-Id: I244290888f578ccfee4c733cd8f93bb6faed4fda
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
2021-06-01 11:58:50 +00:00
Toni Saario
0ed6fd77a0 Allow overriding CMake build timeout values
This allows modules to set custom timeouts for the builds.
Works by modules setting the CMAKE_BUILD_TIMEOUT and
CMAKE_BUILD_OUTPUT_TIMEOUT in the module's module_config.

Change-Id: I6f0170d77e9a962fb37e171d1c0d8c7b2277bb96
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-05-18 07:02:52 +03:00
Simo Fält
3969ef40cc Add instructions for Windows 10 on Arm config
Task-number: QTQAINFRA-3966
Change-Id: Ib8ac80a5b8aa5c88d9e5d0eaad9ba550cc2442f8
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Fathi Boudra
2021-05-05 16:28:27 +03:00
Alexandru Croitor
6e378b6efd coin: Simplify build conditions and allow WebAssembly non-qtbase repos
Rather than using multiple layers of negation, we can just specify a
list of allowed configs.

Allow cross-building other repos when targeting WebAssembly.

Be explicit about not building Android tests for repos other than
qtbase (until that is not longer desired).
Currently the qt5 platform configuration only has qtbase marked
with the AndroidTestRun feature.

Amends fd16c65b7e
Amends 13c460d0ff
Superseeds 60a5b7be03f5800caa8a117c3b07b60d8fb208ec

Change-Id: If8de002e0c9f748b43e3d2271ac283462603b2e6
Reviewed-by: Toni Saario <toni.saario@qt.io>
2021-04-12 09:51:27 +02:00
Joerg Bornemann
fd16c65b7e CI: Add WebAssembly to cross-compiling configs
Change-Id: Ic6fd236623c53ec691cf8c84bb1d385e9b1ac9d2
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2021-04-08 10:12:37 +02:00
Alexandru Croitor
0e6c4224f0 CMake: Build minimal subset of tests in desktop static builds
Add new configure option -make minimal-static-tests and CMake option
QT_BUILD_MINIMAL_STATIC_TESTS. In conjunction with QT_BUILD_TESTS
it will enable building a minimal subset of tests when targeting
a static desktop Qt build.

In qtbase the minimal subset includes all the auto tests of testlib,
tools, corelib and cmake. In particular this will also do cmake build
tests and qmake build tests (tst_qmake)

Adjust CI instructions to enable building a minimal subset of static
tests when a platform configuration is tagged with the
MinimalStaticTests feature.

Fix and skip a few tests that were failing.

Pick-to: 6.1
Task-number: QTBUG-87580
Task-number: QTBUG-91869
Change-Id: I1fc311b8d5e743ccf05047fb9a7fdb813a645206
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-04-06 09:15:37 +01:00
Joerg Bornemann
4ae9922166 Move bin/qt-internal-configure-tests to libexec
Pick-to: 6.1
Task-number: QTBUG-88791
Change-Id: I78613128760b5820e6db17b0c09462e34f7c7bb2
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-03-30 14:39:12 +01:00
Toni Saario
6172f3101e Allow disabling module build and building of tests per module
This patch enables per module control of the build instructions.
This enables documentation building in modules that are not capable
of building the module itself on linux.

Change-Id: I72b7931c5ffda9bf437a99c27c0bb340665ce927
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
2021-02-19 16:28:06 +02:00
Toni Saario
22d692e9e0 Doc Build: Do not fail if CMakeCache.txt does not exist
If we skip module build the CMakeCache.txt does not exist.

Change-Id: Ic44bddf29f1eef1e6ffc568c871d9d7e7de71f48
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
2021-02-19 16:27:47 +02:00
Assam Boudjelthia
136324ad62 Coin: fix android emulator script to use bash instead of sh
Task-number: QTBUG-91180
Pick-to: 6.0 6.1
Change-Id: I410c6769aac3b3b760ff35aa71d52c8a5d6c1143
Reviewed-by: Toni Saario <toni.saario@qt.io>
2021-02-19 10:31:44 +02:00
Assam Boudjelthia
bcbdbd50fe Coin: remove workarounds to allow running Android tests
We had two workarounds:
* script that adds Gui to tests
* create a symbolic link for the qt install dir to fake_prefix which
androiddelployqt was expecting them to be under

Both issues are fixed, thus removing the workarounds.

Pick-to: 6.1 6.0
Change-Id: Ic022bece15afe92c693d573893d260b13b4227ed
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
2021-02-18 13:42:19 +02:00
Marianne Yrjänä
4d9a5f28ee Add QNX7.1 to CI
Change-Id: Ib7288844f3474632d70b60beadfb2b06379855c6
Reviewed-by: Simo Fält <simo.falt@qt.io>
2021-01-19 15:17:55 +02:00
Marianne Yrjänä
2481d92c6e Prepare QNX env for cmake build
QTQAINFRA-3768

Change-Id: I5993acd282843095deba3d34006b7aa3e23db448
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
2021-01-11 15:02:09 +02:00
Assam Boudjelthia
f3cfdb905b Android: install android test scripts to libexec to test on all modules
The scripts that are used by Android test VMs are now located in qtbase
and are not easily usable by other modules. To fix that and allow other
modules to use those scripts, we install them with cmake into libexec.

Task-number: QTQAINFRA-4052
Pick-to: 6.0
Change-Id: Ibdd3658fd9fe7e007104a85d9999028a2de99a33
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-12-11 20:28:50 +00:00
Assam Boudjelthia
86a655387c Android: restart emulator if it gets stuck during testing
Task-number: QTQAINFRA-4052
Pick-to: 6.0
Change-Id: Ie1a0d1c6b6c5cd0425b7ccf195ac57408142f3c3
Reviewed-by: Simo Fält <simo.falt@qt.io>
2020-12-11 11:09:22 +02:00
Alexandru Croitor
24f12d0cef CMake: Build examples with qmake against a CMake built Qt
We want to remove the Qt .pro files for projects, except examples,
because examples are still meant to build with qmake.

To not lose coverage on examples built with qmake, add instructions that
will build the qtrepo/examples folder with qmake when the CMake
configuration has -DQT_BUILD_EXAMPLES=ON.

This means that such configurations will build examples both with CMake
and qmake.

Aside from making sure that our examples will still build with qmake, it
will gives us some some coverage that a CMake-built qmake works
correctly.

Implementation-wise, add new instructions files that can call qmake and
make depending on configuration and target type.

Pick-to: 6.0
Fixes: QTBUG-85986
Change-Id: Ie8f4cbcda03c94da2aef455e32f48dad41a4bdb0
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-12-04 15:39:32 +01:00
Toni Saario
7fd89ed63b Add instructions to build documentation
This enables documentation building in add-ons and other modules that
need to build docs.

Task-number: QTQAINFRA-3972
Change-Id: Ic2763d6d36d26e5a1267b312727c3c268ad6d114
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
(cherry picked from commit e84e85dd4c59f2c79f81ba4d8d7f04397eb0003c)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
2020-12-01 08:12:09 +00:00
Joerg Bornemann
fb21a5ce1a CMake: Name QT_NO_MAKE_*/BUILD_* variables consistently
For consistency, apply the following renamings:
QT_NO_MAKE_EXAMPLES -> QT_BUILD_EXAMPLES_BY_DEFAULT
QT_NO_MAKE_TESTS    -> QT_BUILD_TESTS_BY_DEFAULT
QT_NO_MAKE_TOOLS    -> QT_BUILD_TOOLS_BY_DEFAULT
BUILD_EXAMPLES      -> QT_BUILD_EXAMPLES
BUILD_TESTING       -> QT_BUILD_TESTS

This should help to better convey the difference between "BUILD" and
"NO_MAKE".

To configure tests, but not to build them by default, pass the
following to CMake:
-DQT_BUILD_TESTS=ON -DQT_BUILD_TESTS_BY_DEFAULT=OFF

Analoguous for examples:
-DQT_BUILD_EXAMPLES=ON -DQT_BUILD_EXAMPLES_BY_DEFAULT=OFF

Tools can be excluded from the default build with:
-DBUILD_TOOLS_BY_DEFAULT=OFF

The variable BUILD_TESTING is still available and initialized with the
value of QT_BUILD_TESTS.

Pick-to: 6.0 6.0.0
Change-Id: Ie5f29dfbdca8bfa8d687981dfe8c19c0397ca080
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-11-20 20:38:03 +01:00
Simo Fält
13c460d0ff Add coin instructions to run Android test in emulator
Task-number: QTQAINFRA-3867
Pick-to: 6.0
Change-Id: Ie6dd9c2dfeeccd526c2133d7ac03efce5b7ed091
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
2020-11-20 13:28:31 +00:00
Alexandru Croitor
bcc7d34540 CMake: Provide a qt-internal-configure-tests script
This is meant to be called by our CI instructions to build standalone
tests of a Qt repository.

Currently it just calls qt-cmake with
-DQT_BUILD_STANDALONE_TESTS=ON, but it might contain more things in
the future.

The script also simplifies configuring standalone tests locally, due
to not having to remember the name of the magical variable.

Change our CI instructions to use the new script.

Change-Id: I6bc02b4e94adc9d0d05fecb0fe70a561043271f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-10-26 15:23:25 +01:00
Alexandru Croitor
054c1aaf79 CMake: Don't use mkspec set by sourcing b2qt environment script
qmake sees the QMAKESPEC variable and uses it when building tests,
instead of using the target mkspec written in the target_qt.conf file
by the build system. Unset the environment variable to force usage
of the mkspec specified when confugring Qt.

Task-number: QTBUG-86053
Change-Id: I24ceddd1436393b496665e439dd1441ecbd36df7
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2020-10-26 13:46:49 +01:00
Toni Saario
a5b963c638 Set MSVC environment script specifically for MSVC
Current clause is too broad and new compilers do not work with it
properly. E.g. Clang.

Change-Id: I7afadfec07935e76882d27ec7ab408b0e597654b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-10-01 22:54:22 +03:00
Toni Saario
f26d4def67 Read test specific configure arguments from environment
Reading configure arguments for tests from environment allows control
of CMake args from outside of the module itself. TEST_CONFIGURE_ARGS is
read and appended to host tests build and TARGET_TEST_CONFIGURE_ARGS
respectively for target's tests.

Change-Id: I8270b0254525aec24f7614cba2b90b291e5eb3d2
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-09-28 05:28:22 +00:00
Toni Saario
19d2e61f87 Respect abort testing on failure flag
On regular commits we do not want to waste time running all test if
there is failures.

Change-Id: I050d191058293f4311268169eb26754349930129
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2020-09-25 23:31:02 +03:00
Toni Saario
716e41e86f CMake: Use CMAKE_AUTOGEN_VERBOSE to see moc invocations
In qmake builds we can see the moc invocations, whereas in CMake builds
by default we don't.
Modify the Coin instructions to pass the CMAKE_AUTOGEN_VERBOSE cache
variable so that AUTOMOC prints the moc invocations.

Change-Id: I50be13224839fbbdece3c9e8a4935a72aba91a8e
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-07-24 17:33:17 +02:00
Toni Saario
684fea1314 CMake: Add instructions to re-use host artifacts for target builds
We don't want to build the host Qt configuration every single time we
do a cross-build in Coin.
Coin learned some new qt5.git platform configuration options, which
allow specifying a dependency between a host Coin configuration and a
target one.

This means we can specify a host macOS config as a dependency when
building iOS, and the host artifacts for all dependent repos
(qtbase, qtsvg, qtdeclarative, etc) will be installed into
/home/qt/work/install (as specified by the
{{.AgentWorkingDir}}/install location).

Same for Linux + Android and Linux + qemu.

Modify the qtbase Coin instructions to use these installed host
artifacts if such a dependency is present (platformDependency is not
null). The target artifacts will be installed into a subfolder of the
host installation, called 'target', e.g. /home/qt/work/install/target.

If a dependency is not present, continue to build a host Qt before
doing a target build. This allows gradual conversion of configurations
in case if some of them don't work, or if the dependencies.yaml have
not propagated far enough for all new repos to use the new
instructions.
In this case the locations of the artifacts are different
host - /home/qt/work/install/host
target - /home/qt/work/install/target

Task-number: QTBUG-85623
Change-Id: I33539f4376034539fb7db80293dc4d39dcb9539b
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-07-24 17:33:13 +02:00
Alexandru Croitor
70f7155fcc CMake: Propagate non-qtbase configure args to other repositories
Currently configure arguments specified in
qt5.git/coin/platform_configs are only used when configuring
qtbase itself. Sadly, Coin lacks support for specifying per-repo
configure flags.

Until Coin gets the feature, the best we can do is to pass
all the configure flags to each repo we are configuring.

Unfortunately if we pass all of qtbase's configure flags to other
repos it will break cross-compiling configurations (because the
toolchain argument would be overridden with the initial qtbase
provided toolchain instead of the Qt generated chainloading one).

We thus have to separate flags that should and shouldn't be passed
to other repos.

Introduce usage of two new environment variables called
NON_QTBASE_CONFIGURE_ARGS and NON_QTBASE_TARGET_CONFIGURE_ARGS.
Additional configure arguments can be specified in these variables,
and they will be used when configuring repositories, with the
exception of qtbase.

Ultimately, this is needed to disable detection / usage of certain
strawberry perl system libraries in qtimageformats on MinGW.
Doing that prevents crashes of the tst_qtiff test.

Minor implementation note is that the environment variables have to be
explicitly set in qt5.git platform_configs otherwise it breaks the
Windows Ninja Multi-Config configuration with the error
"The system cannot find the file specified".

Supplements 4b4f0be08debcfde62caf4066222b348ce65dea1 from qt5.git.
Amends previous revert 946fc92d20

Task-number: QTBUG-84886
Task-number: COIN-601
Change-Id: Ie4305c5d799bf4a29da5032a691e797b49c3cf33
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-07-13 21:01:59 +02:00
Alexandru Croitor
946fc92d20 Revert "CMake: Propagate configure args to all repositories"
This reverts commit adbadc57df.

The change breaks building of repos other than qtbase when
cross-compiling to Android. Not sure yet why.

Change-Id: I6a6efef5360ed141c0c5056afb03b1a0d90439ae
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2020-07-08 21:14:18 +02:00
Alexandru Croitor
adbadc57df CMake: Propagate configure args to all repositories
Currently the configure arguments specified in qt5.git/coin are only
used when building qtbase itself. None of them are used when
configuring other repos like qtsvg.

While in principle passing all the configure arguments to other repos
is not a good thing, we need to do it to circumvent the lack of
support for specifying per-repo configure flags in Coin.

It's needed to disable detection / usage of certain strawberry perl
system libraries in qtimageformats on MinGW. Doing that prevents
crashes of the tst_qtiff test.

Supplements 4b4f0be08debcfde62caf4066222b348ce65dea1 from qt5.git.

Task-number: QTBUG-84886
Task-number: COIN-601
Change-Id: I59be8c62ebb10282db1b0a9da2d3e3e5595c925f
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-07-08 11:01:38 +02:00
Joerg Bornemann
860220bd33 CMake: Re-run tests in CI like we do for the qmake build
The test runner re-runs failing tests up to 5 times. We did not do this
for the CMake build, and now we're getting lots of flaky tests failing.
Fix this by passing the -repeat until-pass:5 parameter to ctest.

Change-Id: I6ff3c1e7901bc22cafec87b15d087eab2a565c65
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2020-07-06 12:17:51 +02:00
Alexandru Croitor
7c51f7ccc5 CMake: Enable enforcing CI tests in qtbase
Any failing tests in CMake configurations will now prevent
integrations to go through in the qtbase repository.

Task-number: QTBUG-84886
Change-Id: I8c963cb2540a29a9c8702acd3d282da82fb521e7
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2020-07-03 11:14:11 +02:00
Alexandru Croitor
0423814a2f CMake: Add support for enforcing passing CI tests per repository
Create a new test instructions yaml template that does not ignore the
exit code of running tests. This template can be included by
repositories where tests pass in all configurations tested by the CI.

The template should be included in module_config.yaml via

Test:
    - !include "{{qt/qtbase}}/coin_module_test_template_v3.yaml"

This allows us to opt into enforcing test runs per repository,
rather than waiting to fix all tests in all qt5.git repositories.

Implementation notes

Try to extract the common parts of the instructions as much as
possible to avoid duplication. Unfortunately some duplication still
remains due to the restricted yaml language supported by Coin.

Add a short README.md file that describes the differences between the
existing templates.

The v2 non-enforcing test template should probably be removed in
the future.

Task-number: QTBUG-84886
Task-number: QTBUG-85364
Change-Id: I718fc3cb44d6aefdbebc2fd2088a910a8095a375
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-07-03 11:14:11 +02:00
Alexandru Croitor
5d6705c567 CMake: Skip / ignore failing tests on CMake platforms
Skip crashing tests and ignore failing tests on CMake platforms.

Add missing QTEST_ENVIRONMENT=ci env var assignment to Coin test
instructions. This was hardcoded by the Coin code for qmake
configurations.

Task-number: QTBUG-85364
Change-Id: Id2312e504a0d36b8f8596d4cebaa49c63731406e
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-07-01 14:55:29 +02:00
Alexandru Croitor
167741717f CMake: Fix tst_qmake not to hang and re-enable it
We don't really know why, but using the ctest
--force-new-ctest-process flag stops the test from hanging.

Also re-enable the test.

Task-number: QTBUG-78449
Task-number: QTBUG-81365
Change-Id: I33094696dfe3f610dc257089074b1c2a9926f651
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-06-26 22:14:09 +02:00
Alexandru Croitor
a2c772dc58 CMake: Use build environment prefix.bat when running tests
Tests like tst_qmake need the build environment to be able to build
apps / libraries. This is mostly needed for MSVC.

Set the TESTS_ENV_PREFIX env var to point to the proper prefix.bat
(host or target) and use that when running ctest.

Task-number: QTBUG-85240
Task-number: QTBUG-78449
Task-number: QTBUG-81365
Change-Id: I6fa68714202ac7fc703973fc772e03b84790a043
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
2020-06-26 22:14:04 +02:00
Alexandru Croitor
d8f814cdb7 CMake: Add instructions for building and testing qemu configs
Before this change, tests were never built for cross-compiling
configurations.

Add instructions that build and run tests for qemu configurations.
This required a bit of abstraction / indirection to find out what's
the path to the correct qt-cmake call, as well as prepending the
correct env prefix to each call (either the host one or the target
one).

Also, the QEMU configuration requires a few environment variables
to pick up the correct runtime linker and which qpa plugin to use
for running tests.

Finally, make sure the tests are not built and run if the DisableTests
feature is set in the configuration.

Task-number: QTBUG-84423
Change-Id: I0cec28c801a657e67cfa48f3c61cfe487109946b
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2020-06-25 14:11:56 +02:00