The JUnit format is easier to feed to tools that don't support
the native Qt Test XML format.
Pick-to: 6.2
Change-Id: Iad677b98953275fd70765acb039f1fb3d0f44dab
Reviewed-by: Simo Fält <simo.falt@qt.io>
Enable building and uploading test artefacts for all modules in case
tests are enabled.
Pick-to: 6.2
Task-number: QTBUG-88846
Change-Id: I30832820dea348187b517e170fa1c5266db85604
Reviewed-by: Toni Saario <toni.saario@qt.io>
Add infrastructure to build cmake auto tests in the CI when targeting
iOS.
Currently the are only CI instructions for qtbase.
More work is needed to make it work for other repos.
With this change, we will build a single Widgets application targeting
the iOS simulator. We can't target the device SDK in the CI because
signing fails due to a missing signing certificate and provisioning
profile.
The Coin instructions will now set a QT_BUILD_ENVIRONMENT=ci env var
whose value will be checked in _qt_internal_test_expect_pass, to
ensure we build for the simulator SDK when using a universal Qt.
Without this, xcodebuild will try to build with the device SDK and
fail to build the project.
Pick-to: 6.2
Task-number: QTBUG-95839
Change-Id: Ib39c9527b107b2004746ccbdc9d9d1d658f88c76
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Change-Id: I22617a5f7b8971eae679058feaf41ba811e0a3dd
Reviewed-by: Kimmo Ollila <kimmo.ollila@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Change adds coin_ctest_runner.py which is used to pass test arguments
for testcase. Results are stored to xml files, which are then parsed by
COIN. Data that we are interested in, is injected to influx db.
Task-number: COIN-643
Change-Id: Ib1c99fc8aacc6a6b5863d282f3435fe1cfdcd013
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
If these values already exist do not add them to configure. This also
allows setting these values from platform configurations
Change-Id: I2f2aea90d91fa3436953e0c4103f0ea204b3564b
Reviewed-by: Toni Saario <toni.saario@qt.io>
The Android test VMs take almost the same amount of time of that set to
the test timeout, this is causing it to be killed by COIN in some cases.
This is for example a stat from a successful test instance:
Total Test time (real) = 7110.20 sec
It's very close to 7200 which is the timeout value. The VM is taking
that long because the tests are only half built, the projects' shared
libs are built during the Build step of the integration, and the actual
apk (Gradle build) is done later when doing a ninja tst_name_check.
This latter behavior should be changed so that the apks are built before
going to testing.
So it makes sense for now to increase the timeout and avoid having to
restage multiple times.
Task-number: QTBUG-94127
Change-Id: I31783bcc1255c7821134e174ab6b0fb199000272
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
This allows modules to set custom timeouts for the builds.
Works by modules setting the CMAKE_BUILD_TIMEOUT and
CMAKE_BUILD_OUTPUT_TIMEOUT in the module's module_config.
Change-Id: I6f0170d77e9a962fb37e171d1c0d8c7b2277bb96
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Rather than using multiple layers of negation, we can just specify a
list of allowed configs.
Allow cross-building other repos when targeting WebAssembly.
Be explicit about not building Android tests for repos other than
qtbase (until that is not longer desired).
Currently the qt5 platform configuration only has qtbase marked
with the AndroidTestRun feature.
Amends fd16c65b7e
Amends 13c460d0ff
Superseeds 60a5b7be03f5800caa8a117c3b07b60d8fb208ec
Change-Id: If8de002e0c9f748b43e3d2271ac283462603b2e6
Reviewed-by: Toni Saario <toni.saario@qt.io>
Add new configure option -make minimal-static-tests and CMake option
QT_BUILD_MINIMAL_STATIC_TESTS. In conjunction with QT_BUILD_TESTS
it will enable building a minimal subset of tests when targeting
a static desktop Qt build.
In qtbase the minimal subset includes all the auto tests of testlib,
tools, corelib and cmake. In particular this will also do cmake build
tests and qmake build tests (tst_qmake)
Adjust CI instructions to enable building a minimal subset of static
tests when a platform configuration is tagged with the
MinimalStaticTests feature.
Fix and skip a few tests that were failing.
Pick-to: 6.1
Task-number: QTBUG-87580
Task-number: QTBUG-91869
Change-Id: I1fc311b8d5e743ccf05047fb9a7fdb813a645206
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
This patch enables per module control of the build instructions.
This enables documentation building in modules that are not capable
of building the module itself on linux.
Change-Id: I72b7931c5ffda9bf437a99c27c0bb340665ce927
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
If we skip module build the CMakeCache.txt does not exist.
Change-Id: Ic44bddf29f1eef1e6ffc568c871d9d7e7de71f48
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
We had two workarounds:
* script that adds Gui to tests
* create a symbolic link for the qt install dir to fake_prefix which
androiddelployqt was expecting them to be under
Both issues are fixed, thus removing the workarounds.
Pick-to: 6.1 6.0
Change-Id: Ic022bece15afe92c693d573893d260b13b4227ed
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
The scripts that are used by Android test VMs are now located in qtbase
and are not easily usable by other modules. To fix that and allow other
modules to use those scripts, we install them with cmake into libexec.
Task-number: QTQAINFRA-4052
Pick-to: 6.0
Change-Id: Ibdd3658fd9fe7e007104a85d9999028a2de99a33
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
We want to remove the Qt .pro files for projects, except examples,
because examples are still meant to build with qmake.
To not lose coverage on examples built with qmake, add instructions that
will build the qtrepo/examples folder with qmake when the CMake
configuration has -DQT_BUILD_EXAMPLES=ON.
This means that such configurations will build examples both with CMake
and qmake.
Aside from making sure that our examples will still build with qmake, it
will gives us some some coverage that a CMake-built qmake works
correctly.
Implementation-wise, add new instructions files that can call qmake and
make depending on configuration and target type.
Pick-to: 6.0
Fixes: QTBUG-85986
Change-Id: Ie8f4cbcda03c94da2aef455e32f48dad41a4bdb0
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This enables documentation building in add-ons and other modules that
need to build docs.
Task-number: QTQAINFRA-3972
Change-Id: Ic2763d6d36d26e5a1267b312727c3c268ad6d114
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
(cherry picked from commit e84e85dd4c59f2c79f81ba4d8d7f04397eb0003c)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
For consistency, apply the following renamings:
QT_NO_MAKE_EXAMPLES -> QT_BUILD_EXAMPLES_BY_DEFAULT
QT_NO_MAKE_TESTS -> QT_BUILD_TESTS_BY_DEFAULT
QT_NO_MAKE_TOOLS -> QT_BUILD_TOOLS_BY_DEFAULT
BUILD_EXAMPLES -> QT_BUILD_EXAMPLES
BUILD_TESTING -> QT_BUILD_TESTS
This should help to better convey the difference between "BUILD" and
"NO_MAKE".
To configure tests, but not to build them by default, pass the
following to CMake:
-DQT_BUILD_TESTS=ON -DQT_BUILD_TESTS_BY_DEFAULT=OFF
Analoguous for examples:
-DQT_BUILD_EXAMPLES=ON -DQT_BUILD_EXAMPLES_BY_DEFAULT=OFF
Tools can be excluded from the default build with:
-DBUILD_TOOLS_BY_DEFAULT=OFF
The variable BUILD_TESTING is still available and initialized with the
value of QT_BUILD_TESTS.
Pick-to: 6.0 6.0.0
Change-Id: Ie5f29dfbdca8bfa8d687981dfe8c19c0397ca080
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This is meant to be called by our CI instructions to build standalone
tests of a Qt repository.
Currently it just calls qt-cmake with
-DQT_BUILD_STANDALONE_TESTS=ON, but it might contain more things in
the future.
The script also simplifies configuring standalone tests locally, due
to not having to remember the name of the magical variable.
Change our CI instructions to use the new script.
Change-Id: I6bc02b4e94adc9d0d05fecb0fe70a561043271f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
qmake sees the QMAKESPEC variable and uses it when building tests,
instead of using the target mkspec written in the target_qt.conf file
by the build system. Unset the environment variable to force usage
of the mkspec specified when confugring Qt.
Task-number: QTBUG-86053
Change-Id: I24ceddd1436393b496665e439dd1441ecbd36df7
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Current clause is too broad and new compilers do not work with it
properly. E.g. Clang.
Change-Id: I7afadfec07935e76882d27ec7ab408b0e597654b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reading configure arguments for tests from environment allows control
of CMake args from outside of the module itself. TEST_CONFIGURE_ARGS is
read and appended to host tests build and TARGET_TEST_CONFIGURE_ARGS
respectively for target's tests.
Change-Id: I8270b0254525aec24f7614cba2b90b291e5eb3d2
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
On regular commits we do not want to waste time running all test if
there is failures.
Change-Id: I050d191058293f4311268169eb26754349930129
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
In qmake builds we can see the moc invocations, whereas in CMake builds
by default we don't.
Modify the Coin instructions to pass the CMAKE_AUTOGEN_VERBOSE cache
variable so that AUTOMOC prints the moc invocations.
Change-Id: I50be13224839fbbdece3c9e8a4935a72aba91a8e
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
We don't want to build the host Qt configuration every single time we
do a cross-build in Coin.
Coin learned some new qt5.git platform configuration options, which
allow specifying a dependency between a host Coin configuration and a
target one.
This means we can specify a host macOS config as a dependency when
building iOS, and the host artifacts for all dependent repos
(qtbase, qtsvg, qtdeclarative, etc) will be installed into
/home/qt/work/install (as specified by the
{{.AgentWorkingDir}}/install location).
Same for Linux + Android and Linux + qemu.
Modify the qtbase Coin instructions to use these installed host
artifacts if such a dependency is present (platformDependency is not
null). The target artifacts will be installed into a subfolder of the
host installation, called 'target', e.g. /home/qt/work/install/target.
If a dependency is not present, continue to build a host Qt before
doing a target build. This allows gradual conversion of configurations
in case if some of them don't work, or if the dependencies.yaml have
not propagated far enough for all new repos to use the new
instructions.
In this case the locations of the artifacts are different
host - /home/qt/work/install/host
target - /home/qt/work/install/target
Task-number: QTBUG-85623
Change-Id: I33539f4376034539fb7db80293dc4d39dcb9539b
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
Currently configure arguments specified in
qt5.git/coin/platform_configs are only used when configuring
qtbase itself. Sadly, Coin lacks support for specifying per-repo
configure flags.
Until Coin gets the feature, the best we can do is to pass
all the configure flags to each repo we are configuring.
Unfortunately if we pass all of qtbase's configure flags to other
repos it will break cross-compiling configurations (because the
toolchain argument would be overridden with the initial qtbase
provided toolchain instead of the Qt generated chainloading one).
We thus have to separate flags that should and shouldn't be passed
to other repos.
Introduce usage of two new environment variables called
NON_QTBASE_CONFIGURE_ARGS and NON_QTBASE_TARGET_CONFIGURE_ARGS.
Additional configure arguments can be specified in these variables,
and they will be used when configuring repositories, with the
exception of qtbase.
Ultimately, this is needed to disable detection / usage of certain
strawberry perl system libraries in qtimageformats on MinGW.
Doing that prevents crashes of the tst_qtiff test.
Minor implementation note is that the environment variables have to be
explicitly set in qt5.git platform_configs otherwise it breaks the
Windows Ninja Multi-Config configuration with the error
"The system cannot find the file specified".
Supplements 4b4f0be08debcfde62caf4066222b348ce65dea1 from qt5.git.
Amends previous revert 946fc92d20
Task-number: QTBUG-84886
Task-number: COIN-601
Change-Id: Ie4305c5d799bf4a29da5032a691e797b49c3cf33
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
This reverts commit adbadc57df.
The change breaks building of repos other than qtbase when
cross-compiling to Android. Not sure yet why.
Change-Id: I6a6efef5360ed141c0c5056afb03b1a0d90439ae
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Currently the configure arguments specified in qt5.git/coin are only
used when building qtbase itself. None of them are used when
configuring other repos like qtsvg.
While in principle passing all the configure arguments to other repos
is not a good thing, we need to do it to circumvent the lack of
support for specifying per-repo configure flags in Coin.
It's needed to disable detection / usage of certain strawberry perl
system libraries in qtimageformats on MinGW. Doing that prevents
crashes of the tst_qtiff test.
Supplements 4b4f0be08debcfde62caf4066222b348ce65dea1 from qt5.git.
Task-number: QTBUG-84886
Task-number: COIN-601
Change-Id: I59be8c62ebb10282db1b0a9da2d3e3e5595c925f
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
The test runner re-runs failing tests up to 5 times. We did not do this
for the CMake build, and now we're getting lots of flaky tests failing.
Fix this by passing the -repeat until-pass:5 parameter to ctest.
Change-Id: I6ff3c1e7901bc22cafec87b15d087eab2a565c65
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Any failing tests in CMake configurations will now prevent
integrations to go through in the qtbase repository.
Task-number: QTBUG-84886
Change-Id: I8c963cb2540a29a9c8702acd3d282da82fb521e7
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Create a new test instructions yaml template that does not ignore the
exit code of running tests. This template can be included by
repositories where tests pass in all configurations tested by the CI.
The template should be included in module_config.yaml via
Test:
- !include "{{qt/qtbase}}/coin_module_test_template_v3.yaml"
This allows us to opt into enforcing test runs per repository,
rather than waiting to fix all tests in all qt5.git repositories.
Implementation notes
Try to extract the common parts of the instructions as much as
possible to avoid duplication. Unfortunately some duplication still
remains due to the restricted yaml language supported by Coin.
Add a short README.md file that describes the differences between the
existing templates.
The v2 non-enforcing test template should probably be removed in
the future.
Task-number: QTBUG-84886
Task-number: QTBUG-85364
Change-Id: I718fc3cb44d6aefdbebc2fd2088a910a8095a375
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Skip crashing tests and ignore failing tests on CMake platforms.
Add missing QTEST_ENVIRONMENT=ci env var assignment to Coin test
instructions. This was hardcoded by the Coin code for qmake
configurations.
Task-number: QTBUG-85364
Change-Id: Id2312e504a0d36b8f8596d4cebaa49c63731406e
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
We don't really know why, but using the ctest
--force-new-ctest-process flag stops the test from hanging.
Also re-enable the test.
Task-number: QTBUG-78449
Task-number: QTBUG-81365
Change-Id: I33094696dfe3f610dc257089074b1c2a9926f651
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
Tests like tst_qmake need the build environment to be able to build
apps / libraries. This is mostly needed for MSVC.
Set the TESTS_ENV_PREFIX env var to point to the proper prefix.bat
(host or target) and use that when running ctest.
Task-number: QTBUG-85240
Task-number: QTBUG-78449
Task-number: QTBUG-81365
Change-Id: I6fa68714202ac7fc703973fc772e03b84790a043
Reviewed-by: Cristian Adam <cristian.adam@qt.io>
Before this change, tests were never built for cross-compiling
configurations.
Add instructions that build and run tests for qemu configurations.
This required a bit of abstraction / indirection to find out what's
the path to the correct qt-cmake call, as well as prepending the
correct env prefix to each call (either the host one or the target
one).
Also, the QEMU configuration requires a few environment variables
to pick up the correct runtime linker and which qpa plugin to use
for running tests.
Finally, make sure the tests are not built and run if the DisableTests
feature is set in the configuration.
Task-number: QTBUG-84423
Change-Id: I0cec28c801a657e67cfa48f3c61cfe487109946b
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
If we configure qtbase with Ninja Multi-Config, we should use the same
generator when building other repositories as well, to ensure that
all repositories have the same set of configurations (debug and
release). To do that, the Coin instructions will call the
qt-cmake-private wrapper which records the generator used.
For standalone tests continue to use qt-cmake, so that only a single
configuration is built (no need to build tests in both debug and
release mode). The configuration built will be the first one from
the initial list with which qtbase was configured (usually
RelWithDebiInfo). This is ensured by the
QtBuildInternalsExtraConfig.cmake file.
Task-number: QTBUG-80900
Change-Id: I701b2f652a22d51e640a6fdf19c3b2d2dfb34d5c
Reviewed-by: Simon Hausmann <hausmann@gmail.com>
Calling cmake --install . only installs a single configuration.
To install both debug and release artifacts, the install
invocation needs to be done for each configuration.
To keep the Coin instruction code simpler, delegate the looping
over configurations to a custom CMake script, and use it in the
Coin instructions.
Replace all cmake --install calls in the instructions with calls
to either call_host_install.yaml or call_target_install.yaml.
The path to the script depends on whether we are building
qtbase or another module. In the former case the script should
be called from the build dir, otherwise from the install dir.
The other distinction is whether the host or target env prefix
needs to be added.
Task-number: QTBUG-80900
Change-Id: Ied4bf739e2b1a2307f22fc79c1cfad746c8cbc44
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Don't do any of the MSVC-related instructions when targeting MinGW.
Make sure to pass gcc and g++ as compilers when targeting MinGW.
Don't use any ENV_PREFIX that sets up the MSVC environment.
Task-number: QTBUG-75578
Change-Id: Icf5c39b58391d473d914fe8dc17d062812b07df1
Reviewed-by: Simon Hausmann <hausmann@gmail.com>
Using an environment prefix wrapper shell script that sources the SDK's
env setup. The script also ensure that we don't loose cmake on the way.
Change-Id: I9d08bc58f0efaf688512ab26a7ddb800309a5015
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
When building for qemu, the target "os" is still set to Linux, so try to
detect qemu by looking at the version field.
Change-Id: I7c66cdb29a47a44d5b8a394977136139fc646155
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
