The batcher does not currently work properly with non-qtbase
submodules. Disable them temporarily so that at least the
tests in qtbase are batched. Do this from qtbase so that changes
to each and every submodule are not necessary.
Also, maintain a list of tests that were thus skipped. On any
qt cmake internal function call that refers to such a target,
identify the target as skipped and make the call a no-op.
Fixes: QTBUG-109785
Change-Id: Ib0aa5d39eee8315ffd4ac62f6d1f44fe9bbf7a2f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This replaces the qt_parse_all_arguments macro with the built-in
`cmake_parse_arguments(PARSE_ARGV`. In addition, a new function,
_qt_internal_validate_all_args_are_parsed, can be used to check whether
any _UNPARSED_ARGUMENTS have been passed to the function.
Fixes: QTBUG-99238
Change-Id: I8cee83dc92dc6acdaaf747ea6ff9084c11dc649b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This works by collecting the paths of all blacklist files
and deferring a call which ultimately reads all of the files and
glues them together to form a master blacklist file for the batch.
There might be conflicting function names inside the batch. For now
we ignore the problem, while keeping in mind that it exists.
Fixes: QTBUG-110016
Change-Id: I9c8412097418c6e93297ab89af718d7466e2e451
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Otherwise CI hangs if no existing browser can be reused (typical
case)
Fixes: QTBUG-109876
Change-Id: I63436a23c23c4b74c27c18effafde53bb6a4a34e
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
This is needed for static plugins to work correctly as currently
the only place where plugin logic is handled for non-standalone
builds is in qt_internal_add_executable - adding the library later
when encountered in one tests that get batched does not trigger
plugin handling logic.
Fixes: QTBUG-109812
Change-Id: Idcc2bc6b42b2b7ad6afb9796f554b0529de3de5f
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
When encountering a batchable target, only add its defines to the
sources that the target contains. Otherwise, the defines interfere
with other tests in the batch.
Fixes: QTBUG-109848
Change-Id: I35d1665d29bb0ce93b82059f7f3b715070539d21
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
This makes sure the browser only uses a basic password store and
doesn't try to do anything fancy with the OS pass store, since it
causes an issue on opensuse (kwallet isn't set up in CI).
Change-Id: Ib48a15f4834fa9bb26cf8cf3fff9bb4d0fd0238a
Reviewed-by: Mikołaj Boc <Mikolaj.Boc@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
By adding a CMake option() that is OFF by default.
Change-Id: Ic66cae5bb9739ae1db3e534f671cc1e9e4b09647
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This is done to avoid cmake syntax errors when adding new tests.
Change-Id: I8106b13a865b990c52e09573488881bd18e64c7f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
I keep getting this warning because docker-compose is not installed.
Change-Id: Id8e48e8f498c4a029619fffd172932bd86ed03ba
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Since a good number of tests use exec and friends, we want to use
asyncify for them. Asyncify is still possible to enable universally
with device options.
Change-Id: I0916f55328c98f2ba179cda3757eeee9d27f3e47
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Currently, emrun will pick the browser to run tests on somewhat
arbitrarily. For predictability and stability, we should use one
browser as a basis, and Chrome seems to be a good fit as wasm features
are generally in good shape on it and it has good support for launching
from command-line.
Change-Id: I1d06a5916ad24ab9df9b0826c0773c652e6d3fcd
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Make cmake changes that have ctest run the emrun test runner for
all tests that are build for wasm.
Change-Id: I8c07068d79cfd0d745dbcc3d3f025c7c48fe1069
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
_qt_is_test_executable can only be set on a test that is backed by a
real target. QMLTESTs might not backed by an executable target, if
they are missing cpp SOURCES.
Which led to errors like
CMake Error at
cmake/QtTestHelpers.cmake:510
(set_target_properties):
set_target_properties Can not find target to add properties to:
textedit
Call Stack (most recent call first):
tests/auto/qmltest/textedit/CMakeLists.txt:10 (qt_internal_add_test)
Amends 62c681a599
Change-Id: Ie66fd3e94484562061f851c0a034629959d091da
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
This is needed to later determine what kind of runner should be
selected on WASM for the executable. Tests use the test runner,
whereas other executables use qtLoader.
Change-Id: I75aa361403b72f8e82a288967b8a81b8232d68dc
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
If Qt itself is built without the deprecated APIs, but the tests are
not, tests will fail to compile and/or link.
This patch makes sure that the specified QT_DISABLE_DEPRECATED_UP_TO
and QT_WARN_DEPRECATED_UP_TO values are correctly used in the tests.
The definitions are propagated to tests, batched tests, manual tests
and benchmarks.
Fixes: QTBUG-104858
Change-Id: Idf15accaf96c47599084426ba625b985f507ca8b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
DISABLE_EXCEPTION_CATCHING is added unconditionally on WASM. Add
NO_EXCEPTIONS for all WASM tests since those are never supported.
Change-Id: I2ee10779e7ae0d285494ad650be52dee3099915a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Local file system files are difficult to reach on WASM. Build the test
data in so that it's reachable to WASM tests nevertheless.
Change-Id: I65022db7645248d040434759fb60c3076fd65bd8
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Extend the support of pointing at the right test from batch from WASM
to all supported platforms.
Change-Id: I9b6342ed0954f0fdc2f08c76f6b3f24f44f0388a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This is useful in some cases where extra parameters have to be set on
the test target.
Change-Id: Ic727fdaf0bc9c2e1c684d1a86352d31c0f974477
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The batch test target name is a useful thing to obtain in various
target-generating scripts.
Change-Id: I9605cf860fe1485e48108eba7e93f9064209d8fb
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
An approach of test batching (joining multiple tests into a single
binary) has been taken, due to long linking times/binary size on certain
platforms, including WASM. This change adds a new feature
'batch_test_support' in Qt testlib. Based on the value of the feature,
test batching may become enabled with the -batch-tests switch.
Batching works for every target added via qt_internal_add_test. When
first such target is being processed, a new combined target for all of
the future test sources is created under the name of 'test_batch'.
CMake attempts to merge the parameters of each of the tests, and some
basic checks are run for parameter differences that are impossible to
reconcile.
On the C++ level, convenience macros instantiating the tests are
redefined when batch_tests is on. The new, changed behavior triggered
by the changes in the macros registers the tests in a central test
registry, where they are available for execution based solely on their
test name. The test name is interoperable with the names CMake is aware
of, so CTest is able to run the tests one by one in the combined binary.
Task-number: QTBUG-105273
Change-Id: I2b6071d58be16979bd967eab2d405249f5a4e658
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
It's needed for creating qmake build tests.
CMake / CTest has a limitation of not allowing to create single-config
tests when using a multi-config generator using the add_test(NAME)
signature.
Using add_test(NAME) forcefully creates per-config tests, which means
that it's not possible to just run ctest to execute tests, without
specifying a -C parameter, which we do in the CI.
qmake tests need to use the add_test(NAME) signature
to specify the WORKING_DIRECTORY option.
Because of the above limitation, a work around is to not use the
add_test(NAME) signature, but instead delegate the working directory
assignment to a generated cmake script, which
_qt_internal_create_command_script can already do.
Pick-to: 6.4
Task-number: QTBUG-96058
Change-Id: I6f439165994671724157f0edb7a71e351271e329
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
This ensures that tests can be executed on the simulator or device, by
doing the necessary steps like setting a bundle identifier, Info.plist
file, launch screen, etc.
This is done by calling _qt_internal_finalize_executable in the
implementation of all internal test adding functions.
The finalizers are limited only to iOS for now, as an incremental
step, and to ensure we don't accidentally break tests on other
platforms.
At least WebAssembly uses its own finalizers which would likely cause
duplicate calls if the _qt_internal_finalize_executable was
unconditional.
Pick-to: 6.4
Fixes: QTBUG-104754
Change-Id: I729d56385dd206b22c975fc2ce4e2c683e6e4e2c
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.
Existing copyright statements remain intact
Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Warn projects not to use it because PUBLIC_LIBRARIES don't make
sense for executable targets and it also led to some issues in the
internal functions where some of them did not expect to receive
PUBLIC_LIBRARIES.
To ensure builds don't needlessly break, treat PUBLIC_LIBRARIES values
as regular LIBRARIES. In the future we might add an error instead.
Using PUBLIC_LIBRARIES in qt_internal_add_app, etc, accidentally
worked because the option name and the values following it were
parsed as values of the "previous" option, like SOURCES or
INCLUDE_DIRECTORIES or LIBRARIES, and when those got
passed through to qt_internal_extend_target, things magically worked.
We have a lot of projects using PUBLIC_LIBRARIES, mostly due to the
way qmake pro files were written and how pro2cmake converted them.
We'll have to clean up each repo.
Change-Id: I69e09d34afdf98f0d47c08d324643fc986f8131c
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Added a small helper function that also installs the script to
prefix builds. Additionally, leveraging the cmake option
CROSSCOMPILING_EMULATOR for the testrunner fits neatly in the rest
of our cmake code.
Change-Id: I75288e97c81b250ac3997f2e7a22bc7bd82b7b69
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Add a python script that allows us to run wasm tests in CI, along with
the necessary cmake logic to install the script and execute tests
accordingly.
Change-Id: I93b95c115538c4e27b2b833405acab8162be2a8a
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
CROSSCOMPILING_EMULATOR is a target property, not a test property.
Fixes: QTBUG-87864
Pick-to: 6.2 6.3
Change-Id: Icb07e9ed71a6bcbfceb7aa2116bf56eaa0a545c6
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Network autotests cannot connect to services on docker containers from
QNX qemu without all ports explicitly defined (both host and container)
side.
Pick-to: 6.2 6.3
Task-number: QTQAINFRA-4684
Change-Id: Iba44cfa17d42d43ecec3e29985e404f75d5fc232
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
This introduces a new helper function,
qt_internal_add_repo_local_defines and makes use of it in
qt_internal_add_{module,test,executable,benchmark,plugin}. That function
checks whether QT_EXTRA_INTERNAL_TARGET_DEFINES is set. If it is, the
defines listed in there will be aded to all targets passed to the
functions mentioned above.
The intended usage is that QT_EXTRA_INTERNAL_TARGET_DEFINES gets set
in the repository local .cmake.conf. This allows e.g. opting in to
source incompatible changes in leaf modules (as long as those are
guarded by some define).
Pick-to: 6.2 6.3
Fixes: QTBUG-101640
Change-Id: I06c3693ee69f46e95a48de724621f0c97e7cc3a8
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Multiple tests use qt_internal_add_resource that copies the
functionality that is already implemented inside the
qt_internal_add_test function. Simplify these test by replacing
the qt_internal_add_resource call with the new BUILTIN_TESTDATA
option.
Change-Id: I18475b817d6f87264f0de53817d6c26c5ccab4e2
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
With Ninja Multi-Config generator it's necessary to provide the
'-C' option matching the build configuration when running
ctest. This patchset adds the missing option to the command line
of the '_check' targets.
Pick-to: 6.2 6.3
Change-Id: I42acfba26a09877a6d3a5be9230860f15f95a0cd
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
qt_internal_undefine_global_definition disables an internal global
definition that is defined by the qt_internal_add_global_definition
function for a specific target.
Remove the ability to set the custom "undefine" flag for the
definitions since it's hard to control it using the introduced
function.
Pick-to: 6.2 6.3
Task-number: QTBUG-100334
Change-Id: Ic1637d97aa51bbdd06c5b191c57a941aa208d4dc
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Looks like Integrity doesn't deal well with external test data either.
Pick-to: 6.3
Change-Id: I8beb5ac4d2440d40a7efdceae973e430a2354e9d
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Restore the 'QT_NO_JAVA_STYLE_ITERATORS' and
'QT_NO_NARROWING_CONVERSIONS_IN_CONNECT' definitions for Qt
targets.
Add the function that adds global definitions for Qt targets according
to the provided scope and the target property-based switch to disable
the definition for a specific target.
Pick-to: 6.2 6.3
Task-number: QTBUG-100295
Change-Id: I28697e81f9aabc45c48d79aae1e5caea141e04e1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Qt's WinRT support was removed long time ago.
Change-Id: I60b220e970072c3450e3793862d6f68801d2b5b3
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The result variable was expanded at configure time rather than at
script execution time due to missing escaping.
A tidbit of information, the result variable can contain not only an
exit code, but a string as well.
For example on arm macOS with a crashed test it contains
'SIGTRAP'.
Curiously if the crashing executable is executed directly without
CMake, 'Trace/BPT trap: 5' is shown instead, perhaps because of
the shell.
Amends 3ef6af024b
Pick-to: 6.2 6.3
Change-Id: I50e57922abfc6eccde205c6252eebfda510bad41
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
This might give a few insights into why the process has failed, at least
if distinct return values are used for different errors.
Change-Id: I61fe0ede812c4dda3d0cf0f2c96a479d198d340d
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
When building for and running the '_check' target in Android we need
to make sure that target apk is assembled and ready for deployment.
Pick-to: 6.2
Change-Id: Ifc63aac230f0bb8375d319ec443174c5a436485e
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Tests for QNX are run in qemu over ssh and setting WORKING_DIRECTORY
will cause created test wrapper to try change specified directory
which will fail (expects local directory) when test is invoked over
ssh.
Task-number: QTBUG-87628
Pick-to: 6.2
Change-Id: If991002398811cc9cb9cbac54da5d2cb0d8c3589
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The format was renamed in 27db9e458c.
Pick-to: 6.2
Change-Id: I53975c7467d8768dc9dc9ac2d89c42eefa12e22f
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Currently in order to compile a Qt6 test standalone one needs to use
the qt-cmake-standalone-test script which will load the
Qt6BuildInternals/QtStandaloneTestTemplateProject/CMakeLists.txt project
with the current test source directory to create a complete CMake project.
This commit will allow a test to have these lines at top:
cmake_minimum_required(VERSION 3.16)
if (NOT QT_BUILD_STANDALONE_TESTS AND NOT QT_BUILDING_QT)
project(a_qt_test LANGUAGES C CXX ASM)
find_package(Qt6BuildInternals COMPONENTS STANDALONE_TEST)
endif()
and be standalone and work with any IDE capable of loading CMake projects.
Fixes: QTCREATORBUG-25389
Pick-to: 6.1
Change-Id: If3f878b7e560a8bfbb604a8f1aa30b7962742e66
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
QT_TESTCASE_BUILDDIR was implicitly defined by Qt::Test target for
library users in Qt5. By default, this definition will point to
CMAKE_CURRENT_BINARY_DIR. This logic works similarly to qmake logic.
From user's perspective it might be useful to not rely on standard
search paths, but specify own. This can be done by setting the
QT_TESTCASE_BUILDDIR property for the user's target. CMake will
substute the value of the QT_TESTCASE_BUILDDIR property into the
QT_TESTCASE_BUILDDIR definition.
The implicit QT_TESTCASE_SOURCEDIR also seems to be useful. According
to the current logic, it points to CMAKE_CURRENT_SOURCE_DIR.
Fixes: QTBUG-92079
Change-Id: I8a9065f08e859c713b3c8cc08142a9ced0677770
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
