Our tests have the unfortunate tendency to not clean up their resources;
as a consequence the leaksan check creates ton of (non-actionable)
warnings.
Our main focus for now are catching non-leak related issues via ASAN
(e.g. use-after-free).
The special casing for the cmake API tests stays in place in case we
want to re-enable leaksan checks for most tests again.
Task-number: QTQAINFRA-5315
Change-Id: I3e50ebf06ebd373f0ce0a4bb790ec8d38c344515
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
This patchset adds the rules to utilize build artifacts from multiple
android abi builds and to use them for building and running tests.
If target.arch has the value 'Multi', coin will now skip building Qt
itself, since this means that all Qt versions were build previously.
It only builds test using the pre-built Qt for Android archives and
then runs the regular Android tests using multi-ABI apks. The rules
are implemented for qtbase only.
Change-Id: I265c70c45caa2155c75cd15c7e3a4f6a863252b8
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
We need to make a test for wasm run in CI, and have to start small
with qtbase. This lets the tests compile and run.
Additionally, reordered the platforms for full consistency.
Apply the same change to the qtbase config.
Change-Id: If3cedffdd7f1b21215c05b9b9302df8234a47a0c
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
qttestrunner.py needs special logic to handle wasm
tests, which is not implemented yet. Disable the
coin testrunner until that is implemented.
Change-Id: Idfd001df5f6dcf44904940ace9adef5a355886cf
Pick-to: 6.4
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Mikołaj Boc <Mikolaj.Boc@qt.io>
Reflects what has been configured in the Coin. Allows removing
the rules from Coin and brings them visible.
Pick-to: 6.3 6.4
Change-Id: I8b3567bca3ac040725253764775249a0b0dee8eb
Reviewed-by: Simo Fält <simo.falt@qt.io>
These cause Coin to spawn VM for setting one env variable.
Change-Id: I3ef87092107394b1d985e7549c3d3ca7dc509156
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Allow building iOS tests in other repos, if DisableTests is not
specified.
The reordering in coin_module_build_template_v2.yaml is done to match
the same order that's in coin_qtbase_build_template_v2.yaml.
The change in coin_module_test_template_v3.yaml is just cosmetic.
Amends 80705298ca
Pick-to: 6.4
Task-number: QTBUG-96056
Change-Id: Ic69ef938cb9319fa9a0eb063046b009de34bebc1
Reviewed-by: Toni Saario <toni.saario@qt.io>
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.
Existing copyright statements remain intact
Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Add an option to limit the number of tests for building and testing
Android multi-ABI configurations in CI. Currently only Core tests
supposed to run.
Change-Id: Ibb8a41d60d108259ef2675ec54bde2482f87c8b2
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
As the build dir was changed the location of CMake build files changed,
which resulted in "could not load cache" error.
Change-Id: I97d55d060bfab08ab54d1b15d9eabbede1776155
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The test run is wrapped with a special TESTRUNNER script that ignores
failing tests (there are several tests failing when built with ASAN) and
also ignores LSAN errors (memory leaks - but still visible in the
output).
The test run only fails if a test reports ASAN errors or if it
crashes (or times out, which is like a crash caused by qtestlib's
watchdog timer).
Fixes: QTQAINFRA-5025
Change-Id: I861756ab49388ac4a52409d3a780684244e469b1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Cross-builds already were out-of-source.
Move non-cross-builds to out-of-source as well.
Fixes: QTBUG-82820
Fixes: QTBUG-96513
Change-Id: I5bef08f18a16e51fe2c501565699494b46546f84
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Add required environment variable configurations to COIN for
building Qt SW in Windows host for QNX OS.
Task-number: QTBUG-101292
Pick-to: 6.3 6.2
Change-Id: I3ea4cbaaebb693a0a46b5f9cbd08209805366ca7
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
To ensure that both Linux and Windows host builds are equivalent
the Linux host builds need to use the common QNX SDP file that
support both host environments. This new QNX SDP does not include
QNX license files like previously, instead they are separately
installed by the Ansible scripts of the CI system to the $HOME/.qnx
directory. That directory is also supposed to be used for any possible
binaries found via PATH environment variable.
Task-number: QTBUG-101292
Pick-to: 6.3 6.2
Change-Id: I97ad71f00e8289e708c51565f86c6d0c84a9ae84
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
To do that, we install the example binaries into a new directory
rather than the main install dir.
The contents of the dir is not archived by the Coin agent.
This decreases the size of the binary artifacts.
This will be especially important when the examples contain deploy logic
which will install the shared Qt libraries next to each example binary.
Pick-to: 6.2 6.3
Task-number: QTBUG-90820
Change-Id: I0b112698514309bb9b0794ce4117e8bf30c3a819
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Can be useful in the future for path construction, to avoid adding
two different instructions where the only difference is the path
style.
Pick-to: 6.2 6.3
Change-Id: Id67b287a2d9f67889dcc5c4fc7781d9ef60a56be
Reviewed-by: Toni Saario <toni.saario@qt.io>
For some reason starting of qemu randomly fails in the given timeout.
Set it to reasonably large amount so that we can rule out issues with
qemu itself. Also increase script timeout less than what coin gives
for the script to run se we should see if script bails out and is not
stuck.
Pick-to: 6.2 6.3
Change-Id: I37c7ec636f154955c6472d5e9d0fa92bb5bb457f
Reviewed-by: Toni Saario <toni.saario@qt.io>
Re-apply 9a9b253b68 to use
CMAKE_STAGING_PREFIX in cross-compilation builds.
Amends b7986a8f6e9df3727f433a0df0ba56a3355153d0.
Fixes: QTBUG-102108
Pick-to: 6.3
Change-Id: I8875da2df1427f02be68dd737168d76a8dc4ed2a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Makes WarningsAreErrors feature meaningful on cross-compilations.
Change-Id: I777e44a64757d1fcd5f8a49eae0e5322b92ac53b
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
We want to use configure and qt-configure-module when building in
Coin, rather than pure cmake and qt-cmake.
There are a few benefits:
- CI tests the scripts, making sure we don't introduce regressions
- CI uses the same scripts that we mention in our documentation for
developers to use
- The platform configurations become a bit less verbose and easier to
read due to less shouty-case CMake options
To ensure a more gradual porting, Coin will only use the new
instructions if the 'UseConfigure' feature is set on a platform
configuration in qt5.git. This allows going back to the old
instructions in case if something isn't working properly.
Due the opt-in, we need to support both old and new instructions in
the implementation.
The change strives to remove as much duplicate code as possible,
by moving it into common includes.
The README.md is updated to mention the overview of how the different
environment variables are used.
There are a few important things to point out.
1) Because during the porting we have to allow mixing of the old style
and new style, platform configs have to separate CMake-style options
from configure-style options in different environment variables.
Otherwise the instructions wouldn't be able to create a valid
configure call, where all CMake-style options have to go at the end
after a double dash --.
After all platform configs are ported to the new style, it should be
possible to combine all the options in a single environment variable
if that is desired, but it will require another round of porting to
remove all the '-D' prefixes in CMake-style options, and just use
regular variable assignment which configure supports.
e.g. -DQT_BUILD_EXAMPLES=ON becomes QT_BUILD_EXAMPLES=ON, which can be
mixed in-between configure-style args.
2) Configure is more strict in that it doesn't allow passing
unknown options. Due to that, we can't pass non-qtbase configure
options via NON_QTBASE_CONFIGURE_ARGS. qt-configure-module would
error out in the repos where the configure option is unknown.
Because we don't have a Coin configure variable for each repo,
we circumvent the issue by continuing to pass CMake-style options via
NON_QTBASE_CMAKE_ARGS instead, which does not do validation checks.
In the future, we could introduce a configure flag that disables
the validation checks.
Pick-to: 6.2 6.3
Task-number: QTQAINFRA-4357
Task-number: QTQAINFRA-4815
Change-Id: I72d8ba0b3a543b42982e22ae8d6566c0e885c446
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
This part originates from when the CMake was being introduced and
now things are stable enough that the bug rarely is everywhere.
Pick-to: 6.2 6.3
Task-number: COIN-828
Change-Id: Ifb7b5ce82740f5ae49f712f9666870993ce00b15
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Previously we used sh $* for passing parameters of various calls like
cmake / configure. This did not properly preserve quoted arguments,
leading to potential issues.
For example when configuring qtbase with
cmake -DQT_QMAKE_DEVICE_OPTIONS="DISTRO_OPTS=hard-float boot2qt"
the 'boot2qt' option would be passed as a separate argument to cmake,
leading to the QT_QMAKE_DEVICE_OPTIONS cache var containing only
'DISTRO_OPTS=hard-float'.
Use "$@" to preserve the quotes.
Inspired by
https://wiki.bash-hackers.org/scripting/posparams#all_positional_parameters
Amends e9b8837c31
Pick-to: 6.2 6.3
Change-Id: Ib67d4bc138c65f7430457cb2a2baa55bff72e549
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
There seems to be lot of timeouts on testing where qemu is not able to
start at current 60 seconds period.
Pick-to: 6.2 6.3
Change-Id: Icbc47b75a8288e672237789cb273ef5a644c08ba
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
We set the default verbosity of configure output to NOTICE in
regular Qt builds.
To preserve the STATUS verbosity in CI runs, we now pass it
explicitly when configuring all repos / tests.
Pick-to: 6.2 6.3
Change-Id: Ib63739c05855cfd5951d38ce3f8a7fc8d1218cd3
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
QNX requires its own TESTRUNNER script in order to SSH to the QEMU guest
and run the tests. Simplify it a lot and fix the way it passes arguments
so that they retain spaces and quotation.
Also wrap it with the generic TESTRUNNER script for CI, which is
qt-testrunner.py, so that tests can re-run in case of flakiness.
Delete prefix.sh as the environment variables are now properly set in
the Coin platform config files.
Avoid executing extra chmod commands to make scripts executable,
but use the right Coin instruction for that. For reference, we
use 493 for file mode which equals to 755 in octal, as Coin expects
the file mode in decimal.
Change-Id: Ife4d1caef606f48b92ba1da1cfb14ec0dea11ef2
Reviewed-by: Pasi Petäjäjärvi <pasi.petajajarvi@qt.io>
Use CMAKE_STAGING_PREFIX instead of CMAKE_INSTALL_PREFIX when
cross-compiling. This separates the host path used in staging
prefix and the target path used in the install prefix for the
device. This prevents for example Windows paths from being used
in a device that does not support those. It also tells qmake
not to sysrootify paths when building with it.
Embedded linux and QNX builds are mostly affected and need this
to use correct RPATHs and to unsysrootify qmake. Mobile platforms
(Android and iOS) are not affected since they package binaries
separately. WASM and INTEGRITY are static builds and device paths
are not used.
Cross-compiled auto tests keep staging prefix in RPATHs due to
the behavior implemented in commit 20292250d4
which keeps the QEMU test runs working as before.
Pick-to: 6.3 6.2
Change-Id: If464ccd8cd9318a853df9afcb2aa709fbb2c1837
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
As a drive-by this fixes the qtbase build to also have the
CMAKE_AUTOGEN_VERBOSE option set.
Pick-to: 6.2 6.3
Change-Id: I32324fb1e8e16299c5f34517edbc7ff335d84e14
Reviewed-by: Toni Saario <toni.saario@qt.io>
It had been noticed that sometimes the Android emulator in our CI took
several minutes to fully boot. It turns out that this behavior can be
reproduced locally by removing the image files under $HOME/.android
especially the "userdata-qemu.img.qcow2" file.
Then the emulator goes through several reboots until fully booted with a
full set of packages installed. We discovered that the property that
signifies it is finished is dev.bootcomplete=1. So we now check for this
flag and remove the other heuristics we had.
We also disable the debug output to avoid hundreds lines of
logs. Instead we selectively print the values that the emulator returns,
every second until full boot is detected.
We increase the Coin timeout for the shell script, from 5min to 10min,
since it has been measured that it takes about 2.5min on a good day, and
the script itself retries several times to restart the emulator in case
of failure.
Finally we adjust the coding style a bit to be more consistent.
Pick-to: 6.3 6.2
Fixes: QTQAINFRA-4681
Change-Id: I77062dceb91477e957696c89bfacb4ebabc34c1f
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Script that wraps Qt test execution in order to iron-out flakiness.
Task-number: QTBUG-96353
Change-Id: Ie8ee780e3f4f8d086c080a7784f9f68fd1867be2
Reviewed-by: Daniel Smith <Daniel.Smith@qt.io>
Was likely an oversight.
Amends 9f559699cc
Pick-to: 6.2 6.3
Change-Id: I13c2e4cc32e68c1ce98f59a932e3448250b0679b
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Hostname is already set in options file, lets use that instead of
setting it again manually.
Task-number: QTBUG-99168
Pick-to: 6.2 6.3
Change-Id: I31c42a597ff45a94a2e36aa6f5002d171ad448ee
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
Currently only qtbase is being run Qt autotests
Task-number: QTBUG-99168
Pick-to: 6.2 6.3
Change-Id: Ib9678c937c91946f07b78f8141df4837fa5d4111
Reviewed-by: Toni Saario <toni.saario@qt.io>
- there is no possibilities for running autotests on CI
Pick-to: 6.2
Change-Id: I15f0c600172ec4962edf3716a651483982c4f8c6
Reviewed-by: Toni Saario <toni.saario@qt.io>
Bic tests do not work due to bic tests being broken.
A feature enables easier testing of the bic tests.
Change-Id: I15b400ee8f0f877ac2c6c71fc50d51c5e11b330d
(cherry picked from commit 31910cbc09ff219b332aac94afe78c99d34274b3)
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
MSVC2022_PREVIEW can be removed later when MSVC2022 official
release has been merged
Task-number: QTQAINFRA-4540
Pick-to: 6.2
Change-Id: I7756b53b1cd5863d21c1c1e3bb19373f6f0cf8fa
Reviewed-by: Ville-Pekka Karhu <ville-pekka.karhu@qt.io>
Reviewed-by: Toni Saario <toni.saario@qt.io>
The actual module build is done in-tree and after this attempting
to do shadow build of the dirty source directory causes issues in
some modules.
Pick-to: 6.2
Change-Id: I74bca186b238c5c1f10576b4f821efdf66b827ed
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Allows us to remove redundant information in the platform configs.
Pick-to: 6.2 5.15
Change-Id: Id0c09ce6f22f3bd52ef426c14a1b2806559b2321
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* Script used to parse env variables and run test over ssh on qemu
* Script to start qnx qemu
* Script used to set env variables for ssh so they are correct in qemu
* Add b2qt sdk host toolchain to path for actual qemu binaries
Task-number: QTBUG-87628
Pick-to: 6.2
Change-Id: I951172211c2963654899491568771310a410e616
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
A single configuration provides qdoc and qtattributionsscanner via
provisioning. The environment variables instruct QDoc to run in
testing mode; it ignores linking warnings (since we don't have .index
files for external documentation modules) and terminates with non-zero
exit code if documentation warnings exceed the limit set in the
documentation config.
The provisioned binaries are portable, packaged together with their
dependencies. As they are installed under /opt, the instructions
create symlinks to the binaries in the install directory before
building the 'generate_docs' target.
Pick-to: 6.2
Task-number: QTBUG-78069
Fixes: QTBUG-95554
Change-Id: I4a4765250247e2fc6eddd153c36cf8126ddd10ae
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
When official version is available variableValue path need to be
changed.
Task-number: QTQAINFRA-4540
Change-Id: Ie5fe676962297aa5f18f225eb4b8dfe0957671c8
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
This reverts commit c7ddaa9f58.
Reason for revert: Broke integrations
Change-Id: I21ffb3b16e361a48a5d9b7a528d150e51519623c
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
The JUnit format is easier to feed to tools that don't support
the native Qt Test XML format.
Pick-to: 6.2
Change-Id: Iad677b98953275fd70765acb039f1fb3d0f44dab
Reviewed-by: Simo Fält <simo.falt@qt.io>
