The test run is wrapped with a special TESTRUNNER script that ignores
failing tests (there are several tests failing when built with ASAN) and
also ignores LSAN errors (memory leaks - but still visible in the
output).
The test run only fails if a test reports ASAN errors or if it
crashes (or times out, which is like a crash caused by qtestlib's
watchdog timer).
Fixes: QTQAINFRA-5025
Change-Id: I861756ab49388ac4a52409d3a780684244e469b1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Cross-builds already were out-of-source.
Move non-cross-builds to out-of-source as well.
Fixes: QTBUG-82820
Fixes: QTBUG-96513
Change-Id: I5bef08f18a16e51fe2c501565699494b46546f84
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Add required environment variable configurations to COIN for
building Qt SW in Windows host for QNX OS.
Task-number: QTBUG-101292
Pick-to: 6.3 6.2
Change-Id: I3ea4cbaaebb693a0a46b5f9cbd08209805366ca7
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
To ensure that both Linux and Windows host builds are equivalent
the Linux host builds need to use the common QNX SDP file that
support both host environments. This new QNX SDP does not include
QNX license files like previously, instead they are separately
installed by the Ansible scripts of the CI system to the $HOME/.qnx
directory. That directory is also supposed to be used for any possible
binaries found via PATH environment variable.
Task-number: QTBUG-101292
Pick-to: 6.3 6.2
Change-Id: I97ad71f00e8289e708c51565f86c6d0c84a9ae84
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
To do that, we install the example binaries into a new directory
rather than the main install dir.
The contents of the dir is not archived by the Coin agent.
This decreases the size of the binary artifacts.
This will be especially important when the examples contain deploy logic
which will install the shared Qt libraries next to each example binary.
Pick-to: 6.2 6.3
Task-number: QTBUG-90820
Change-Id: I0b112698514309bb9b0794ce4117e8bf30c3a819
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Can be useful in the future for path construction, to avoid adding
two different instructions where the only difference is the path
style.
Pick-to: 6.2 6.3
Change-Id: Id67b287a2d9f67889dcc5c4fc7781d9ef60a56be
Reviewed-by: Toni Saario <toni.saario@qt.io>
For some reason starting of qemu randomly fails in the given timeout.
Set it to reasonably large amount so that we can rule out issues with
qemu itself. Also increase script timeout less than what coin gives
for the script to run se we should see if script bails out and is not
stuck.
Pick-to: 6.2 6.3
Change-Id: I37c7ec636f154955c6472d5e9d0fa92bb5bb457f
Reviewed-by: Toni Saario <toni.saario@qt.io>
Re-apply 9a9b253b68 to use
CMAKE_STAGING_PREFIX in cross-compilation builds.
Amends b7986a8f6e9df3727f433a0df0ba56a3355153d0.
Fixes: QTBUG-102108
Pick-to: 6.3
Change-Id: I8875da2df1427f02be68dd737168d76a8dc4ed2a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Makes WarningsAreErrors feature meaningful on cross-compilations.
Change-Id: I777e44a64757d1fcd5f8a49eae0e5322b92ac53b
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
We want to use configure and qt-configure-module when building in
Coin, rather than pure cmake and qt-cmake.
There are a few benefits:
- CI tests the scripts, making sure we don't introduce regressions
- CI uses the same scripts that we mention in our documentation for
developers to use
- The platform configurations become a bit less verbose and easier to
read due to less shouty-case CMake options
To ensure a more gradual porting, Coin will only use the new
instructions if the 'UseConfigure' feature is set on a platform
configuration in qt5.git. This allows going back to the old
instructions in case if something isn't working properly.
Due the opt-in, we need to support both old and new instructions in
the implementation.
The change strives to remove as much duplicate code as possible,
by moving it into common includes.
The README.md is updated to mention the overview of how the different
environment variables are used.
There are a few important things to point out.
1) Because during the porting we have to allow mixing of the old style
and new style, platform configs have to separate CMake-style options
from configure-style options in different environment variables.
Otherwise the instructions wouldn't be able to create a valid
configure call, where all CMake-style options have to go at the end
after a double dash --.
After all platform configs are ported to the new style, it should be
possible to combine all the options in a single environment variable
if that is desired, but it will require another round of porting to
remove all the '-D' prefixes in CMake-style options, and just use
regular variable assignment which configure supports.
e.g. -DQT_BUILD_EXAMPLES=ON becomes QT_BUILD_EXAMPLES=ON, which can be
mixed in-between configure-style args.
2) Configure is more strict in that it doesn't allow passing
unknown options. Due to that, we can't pass non-qtbase configure
options via NON_QTBASE_CONFIGURE_ARGS. qt-configure-module would
error out in the repos where the configure option is unknown.
Because we don't have a Coin configure variable for each repo,
we circumvent the issue by continuing to pass CMake-style options via
NON_QTBASE_CMAKE_ARGS instead, which does not do validation checks.
In the future, we could introduce a configure flag that disables
the validation checks.
Pick-to: 6.2 6.3
Task-number: QTQAINFRA-4357
Task-number: QTQAINFRA-4815
Change-Id: I72d8ba0b3a543b42982e22ae8d6566c0e885c446
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
This part originates from when the CMake was being introduced and
now things are stable enough that the bug rarely is everywhere.
Pick-to: 6.2 6.3
Task-number: COIN-828
Change-Id: Ifb7b5ce82740f5ae49f712f9666870993ce00b15
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Previously we used sh $* for passing parameters of various calls like
cmake / configure. This did not properly preserve quoted arguments,
leading to potential issues.
For example when configuring qtbase with
cmake -DQT_QMAKE_DEVICE_OPTIONS="DISTRO_OPTS=hard-float boot2qt"
the 'boot2qt' option would be passed as a separate argument to cmake,
leading to the QT_QMAKE_DEVICE_OPTIONS cache var containing only
'DISTRO_OPTS=hard-float'.
Use "$@" to preserve the quotes.
Inspired by
https://wiki.bash-hackers.org/scripting/posparams#all_positional_parameters
Amends e9b8837c31
Pick-to: 6.2 6.3
Change-Id: Ib67d4bc138c65f7430457cb2a2baa55bff72e549
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
There seems to be lot of timeouts on testing where qemu is not able to
start at current 60 seconds period.
Pick-to: 6.2 6.3
Change-Id: Icbc47b75a8288e672237789cb273ef5a644c08ba
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
We set the default verbosity of configure output to NOTICE in
regular Qt builds.
To preserve the STATUS verbosity in CI runs, we now pass it
explicitly when configuring all repos / tests.
Pick-to: 6.2 6.3
Change-Id: Ib63739c05855cfd5951d38ce3f8a7fc8d1218cd3
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
QNX requires its own TESTRUNNER script in order to SSH to the QEMU guest
and run the tests. Simplify it a lot and fix the way it passes arguments
so that they retain spaces and quotation.
Also wrap it with the generic TESTRUNNER script for CI, which is
qt-testrunner.py, so that tests can re-run in case of flakiness.
Delete prefix.sh as the environment variables are now properly set in
the Coin platform config files.
Avoid executing extra chmod commands to make scripts executable,
but use the right Coin instruction for that. For reference, we
use 493 for file mode which equals to 755 in octal, as Coin expects
the file mode in decimal.
Change-Id: Ife4d1caef606f48b92ba1da1cfb14ec0dea11ef2
Reviewed-by: Pasi Petäjäjärvi <pasi.petajajarvi@qt.io>
Use CMAKE_STAGING_PREFIX instead of CMAKE_INSTALL_PREFIX when
cross-compiling. This separates the host path used in staging
prefix and the target path used in the install prefix for the
device. This prevents for example Windows paths from being used
in a device that does not support those. It also tells qmake
not to sysrootify paths when building with it.
Embedded linux and QNX builds are mostly affected and need this
to use correct RPATHs and to unsysrootify qmake. Mobile platforms
(Android and iOS) are not affected since they package binaries
separately. WASM and INTEGRITY are static builds and device paths
are not used.
Cross-compiled auto tests keep staging prefix in RPATHs due to
the behavior implemented in commit 20292250d4
which keeps the QEMU test runs working as before.
Pick-to: 6.3 6.2
Change-Id: If464ccd8cd9318a853df9afcb2aa709fbb2c1837
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
As a drive-by this fixes the qtbase build to also have the
CMAKE_AUTOGEN_VERBOSE option set.
Pick-to: 6.2 6.3
Change-Id: I32324fb1e8e16299c5f34517edbc7ff335d84e14
Reviewed-by: Toni Saario <toni.saario@qt.io>
It had been noticed that sometimes the Android emulator in our CI took
several minutes to fully boot. It turns out that this behavior can be
reproduced locally by removing the image files under $HOME/.android
especially the "userdata-qemu.img.qcow2" file.
Then the emulator goes through several reboots until fully booted with a
full set of packages installed. We discovered that the property that
signifies it is finished is dev.bootcomplete=1. So we now check for this
flag and remove the other heuristics we had.
We also disable the debug output to avoid hundreds lines of
logs. Instead we selectively print the values that the emulator returns,
every second until full boot is detected.
We increase the Coin timeout for the shell script, from 5min to 10min,
since it has been measured that it takes about 2.5min on a good day, and
the script itself retries several times to restart the emulator in case
of failure.
Finally we adjust the coding style a bit to be more consistent.
Pick-to: 6.3 6.2
Fixes: QTQAINFRA-4681
Change-Id: I77062dceb91477e957696c89bfacb4ebabc34c1f
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Script that wraps Qt test execution in order to iron-out flakiness.
Task-number: QTBUG-96353
Change-Id: Ie8ee780e3f4f8d086c080a7784f9f68fd1867be2
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
Was likely an oversight.
Amends 9f559699cc
Pick-to: 6.2 6.3
Change-Id: I13c2e4cc32e68c1ce98f59a932e3448250b0679b
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Hostname is already set in options file, lets use that instead of
setting it again manually.
Task-number: QTBUG-99168
Pick-to: 6.2 6.3
Change-Id: I31c42a597ff45a94a2e36aa6f5002d171ad448ee
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
Currently only qtbase is being run Qt autotests
Task-number: QTBUG-99168
Pick-to: 6.2 6.3
Change-Id: Ib9678c937c91946f07b78f8141df4837fa5d4111
Reviewed-by: Toni Saario <toni.saario@qt.io>
- there is no possibilities for running autotests on CI
Pick-to: 6.2
Change-Id: I15f0c600172ec4962edf3716a651483982c4f8c6
Reviewed-by: Toni Saario <toni.saario@qt.io>
Bic tests do not work due to bic tests being broken.
A feature enables easier testing of the bic tests.
Change-Id: I15b400ee8f0f877ac2c6c71fc50d51c5e11b330d
(cherry picked from commit 31910cbc09ff219b332aac94afe78c99d34274b3)
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
MSVC2022_PREVIEW can be removed later when MSVC2022 official
release has been merged
Task-number: QTQAINFRA-4540
Pick-to: 6.2
Change-Id: I7756b53b1cd5863d21c1c1e3bb19373f6f0cf8fa
Reviewed-by: Ville-Pekka Karhu <ville-pekka.karhu@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
The actual module build is done in-tree and after this attempting
to do shadow build of the dirty source directory causes issues in
some modules.
Pick-to: 6.2
Change-Id: I74bca186b238c5c1f10576b4f821efdf66b827ed
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Allows us to remove redundant information in the platform configs.
Pick-to: 6.2 5.15
Change-Id: Id0c09ce6f22f3bd52ef426c14a1b2806559b2321
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* Script used to parse env variables and run test over ssh on qemu
* Script to start qnx qemu
* Script used to set env variables for ssh so they are correct in qemu
* Add b2qt sdk host toolchain to path for actual qemu binaries
Task-number: QTBUG-87628
Pick-to: 6.2
Change-Id: I951172211c2963654899491568771310a410e616
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
A single configuration provides qdoc and qtattributionsscanner via
provisioning. The environment variables instruct QDoc to run in
testing mode; it ignores linking warnings (since we don't have .index
files for external documentation modules) and terminates with non-zero
exit code if documentation warnings exceed the limit set in the
documentation config.
The provisioned binaries are portable, packaged together with their
dependencies. As they are installed under /opt, the instructions
create symlinks to the binaries in the install directory before
building the 'generate_docs' target.
Pick-to: 6.2
Task-number: QTBUG-78069
Fixes: QTBUG-95554
Change-Id: I4a4765250247e2fc6eddd153c36cf8126ddd10ae
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
When official version is available variableValue path need to be
changed.
Task-number: QTQAINFRA-4540
Change-Id: Ie5fe676962297aa5f18f225eb4b8dfe0957671c8
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
This reverts commit c7ddaa9f58.
Reason for revert: Broke integrations
Change-Id: I21ffb3b16e361a48a5d9b7a528d150e51519623c
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
The JUnit format is easier to feed to tools that don't support
the native Qt Test XML format.
Pick-to: 6.2
Change-Id: Iad677b98953275fd70765acb039f1fb3d0f44dab
Reviewed-by: Simo Fält <simo.falt@qt.io>
Enable building and uploading test artefacts for all modules in case
tests are enabled.
Pick-to: 6.2
Task-number: QTBUG-88846
Change-Id: I30832820dea348187b517e170fa1c5266db85604
Reviewed-by: Toni Saario <toni.saario@qt.io>
Add infrastructure to build cmake auto tests in the CI when targeting
iOS.
Currently the are only CI instructions for qtbase.
More work is needed to make it work for other repos.
With this change, we will build a single Widgets application targeting
the iOS simulator. We can't target the device SDK in the CI because
signing fails due to a missing signing certificate and provisioning
profile.
The Coin instructions will now set a QT_BUILD_ENVIRONMENT=ci env var
whose value will be checked in _qt_internal_test_expect_pass, to
ensure we build for the simulator SDK when using a universal Qt.
Without this, xcodebuild will try to build with the device SDK and
fail to build the project.
Pick-to: 6.2
Task-number: QTBUG-95839
Change-Id: Ib39c9527b107b2004746ccbdc9d9d1d658f88c76
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Change-Id: I22617a5f7b8971eae679058feaf41ba811e0a3dd
Reviewed-by: Kimmo Ollila <kimmo.ollila@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Change adds coin_ctest_runner.py which is used to pass test arguments
for testcase. Results are stored to xml files, which are then parsed by
COIN. Data that we are interested in, is injected to influx db.
Task-number: COIN-643
Change-Id: Ib1c99fc8aacc6a6b5863d282f3435fe1cfdcd013
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
If these values already exist do not add them to configure. This also
allows setting these values from platform configurations
Change-Id: I2f2aea90d91fa3436953e0c4103f0ea204b3564b
Reviewed-by: Toni Saario <toni.saario@qt.io>
The Android test VMs take almost the same amount of time of that set to
the test timeout, this is causing it to be killed by COIN in some cases.
This is for example a stat from a successful test instance:
Total Test time (real) = 7110.20 sec
It's very close to 7200 which is the timeout value. The VM is taking
that long because the tests are only half built, the projects' shared
libs are built during the Build step of the integration, and the actual
apk (Gradle build) is done later when doing a ninja tst_name_check.
This latter behavior should be changed so that the apks are built before
going to testing.
So it makes sense for now to increase the timeout and avoid having to
restage multiple times.
Task-number: QTBUG-94127
Change-Id: I31783bcc1255c7821134e174ab6b0fb199000272
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Heikki Halmet <heikki.halmet@qt.io>
This allows modules to set custom timeouts for the builds.
Works by modules setting the CMAKE_BUILD_TIMEOUT and
CMAKE_BUILD_OUTPUT_TIMEOUT in the module's module_config.
Change-Id: I6f0170d77e9a962fb37e171d1c0d8c7b2277bb96
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
