The qt_install() calls in qt_internal_add_test() did not re-create
subdirectories that are part of the input test data.
Make sure that the subdirectories are created upon installation by
ensuring we specify a relative installation path that includes all the
path parts except for the file name.
That works the same for directories.
Amends ec1546afc4
Amends 540bd6cf20
Amends 1307736c7d
Amends 0a1256a52d
Task-number: QTBUG-117098
Change-Id: Ia80f4e7e1ec531264864bcde3ac192ce79b65746
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Revert incidental change which leaked from dev environment.
It's better to save CI resources by only having one tab
opened in browser at a given time.
Change-Id: I7f8d1af546b749b2fec4f49a6751545f52c16414
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
This is part of our testing effort where we try enabling more tests for
Web Assembly platform on CI. Not all tests work out of box, so some of
them will require followup work.
This commmit also introduces a new mechanism of automatically renaming
files when they are added many times with the same filename to single
translation unit.
Change-Id: I620536494ea83aeb9b294c4a35ef72b51e85a38b
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
VCPKG by default does static builds when building for Android.
This is at odds with the bundling-concept, so it should not be done
unconditionally.
Since we don't necessarily have the WrapOpenSSL target on-hand, let's
just do a file-exists test for the one of the paths we would include.
Pick-to: 6.6 6.5
Change-Id: I3693354308d5168d8a9c3d1659bfa51540114b7f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Amend 9d5a8162a5, after which building Qt
for Android locally required an OPENSSL_ROOT_DIR variable to be set.
Instead of making it harder for everyone to build Qt for Android (even
if no intention to work on or test network code locally), downgrade the
fatal error to a warning.
Pick-to: 6.6 6.5
Change-Id: I6f2176a40663cc9f55aa3066af78499af1f20894
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
It should be possible to build a test in any configuration
by building Qt in that configuration, and then building
the test with qt-standalone-test.
Not all Qt configurations will be able to run all tests,
(due to exec() calls and similar) but that's OK - some
tests don't have exec() calls, and we want to be able
to run tests for a given configuration to figure out
how well it works.
On CI we want to use batching and asyncify, so it makes
sense to tie usage of asyncify to batching.
Pick-to: 6.6
Change-Id: I05553d250a45c1831f43dc71a43ef02d01d70535
Reviewed-by: Mikołaj Boc <Mikolaj.Boc@qt.io>
The new argument would allow bundling the prebuilt OpenSSL libs
into the test apk so that SSL can actually run. It expects the CMake
argument OPENSSL_ROOT_DIR to be set ( we set that in Coin configs).
Task-number: QTBUG-110025
Pick-to: 6.6 6.5
Change-Id: I4c82796635ca89f5511255ae26182f41a504b026
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
- Removed the NO_UNITY_BUILD argument from commands that disable it by
default.
- Add a warning in case NO_UNITY_BUILD or NO_UNITY_BUILD_SOURCES is
being used where it is already disabled, e.g., qt_internal_add_test
- Exclude all sources of a target from unity build if NO_UNITY_BUILD
is set on the target. This sounds a bit harsh, but I have noticed that
sometimes the same source file can be included somewhere else, and
some unexpected collision may occur.
- qt_examples_build_end excludes all its examples from the unity build.
- qt_build_test now sets the CMAKE_UNITY_BUILD to OFF before configuring
the tests, and restore its value when done.
Pick-to: 6.5
Task-number: QTBUG-109394
Change-Id: Ia42e7dd5a5bfb151db241deb639325720fd91eec
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This should be handled later ideally, for now we should skip and
QML tests targets are still being produced, which leads to errors
on WASM.
Task-number: QTBUG-109786
Change-Id: I3d0d1f3115e324c7a0cb036e972226310294b216
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Tests generated with qt-cmake-standalone-test will now not be
batched by default. Defining the QT_BATCH_STANDALONE_TESTS
environment variable will make the build system batch them.
Fixes: QTBUG-111226
Change-Id: I9f01c662e22f8ffdd33e1c4d82619db0689fecc7
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
qt_internal_add_test_helper will fail if a test batch is missing and
batching is enabled. It will now copy to the parent binary directory
as without batching for simplicity, as this happens when batched tests
are skipped.
One consequence could be that when the first test specified has
the NO_BATCH argument, this will incorrectly install the helper, but
helpers still need extra work to function properly, so for now we just
make it compile.
Task-number: QTBUG-109786
Change-Id: Ib307ae79799422c2a4102885aa007ef043835e50
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Manual tests are supposed to display UI and be assessed manually, but
currently they use the auto test runner by mistake.
Use the normal wasm shell to fix this and make them work like usual
applications.
Fixes: QTBUG-111753
Change-Id: I9d3c0ad56e913b73737c5b72087e82980989d8b8
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
They now share their implementation. Manual tests can be created by
passing the MANUAL arguments to the qt_internal_add_test as well.
Pick-to: 6.5
Fixes: QTBUG-111382
Change-Id: I1c207b7e4a67526554df1ee43fe032bb20fc92fa
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Matthias Rauter <matthias.rauter@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
The previous attempt failed as target coin configuration doesn't
use superbuild. Skip the tests based on the TESTED_MODULE_COIN
environment variable
Task-number: QTBUG-109786
Change-Id: I0dbe6ff64ca4a2e81fef377865ef4e99b58c5eb2
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
- The following commands accepts NO_UNITY_BUILD, and
NO_UNITY_BUILD_SOURCES arguments to opt out of the unity build, and
to exclude some source files from unity build, respectively.
- qt_internal_add_executable
- qt_internal_add_module
- qt_internal_add_plugin
- qt_internal_add_tool
- qt_internal_extend_target
- qt_internal_add_common_qt_library_helper
- qt_internal_add_cmake_library
- qt_internal_add_simd_part
- Unity build is disabled by default in these:
- qt_internal_add_test
- qt_internal_add_test_helper
- qt_internal_add_benchmark
- qt_internal_add_3rdparty_library
- qt_update_ignore_pch_source also excludes the files from unity_build
Pick-to: 6.5
Task-number: QTBUG-109394
Change-Id: I5d0e7df633738310a015142a6c73fbb78b6c3467
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The batcher does not currently work properly with non-qtbase
submodules. Disable them temporarily so that at least the
tests in qtbase are batched. Do this from qtbase so that changes
to each and every submodule are not necessary.
Also, maintain a list of tests that were thus skipped. On any
qt cmake internal function call that refers to such a target,
identify the target as skipped and make the call a no-op.
Fixes: QTBUG-109785
Change-Id: Ib0aa5d39eee8315ffd4ac62f6d1f44fe9bbf7a2f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This replaces the qt_parse_all_arguments macro with the built-in
`cmake_parse_arguments(PARSE_ARGV`. In addition, a new function,
_qt_internal_validate_all_args_are_parsed, can be used to check whether
any _UNPARSED_ARGUMENTS have been passed to the function.
Fixes: QTBUG-99238
Change-Id: I8cee83dc92dc6acdaaf747ea6ff9084c11dc649b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This works by collecting the paths of all blacklist files
and deferring a call which ultimately reads all of the files and
glues them together to form a master blacklist file for the batch.
There might be conflicting function names inside the batch. For now
we ignore the problem, while keeping in mind that it exists.
Fixes: QTBUG-110016
Change-Id: I9c8412097418c6e93297ab89af718d7466e2e451
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Otherwise CI hangs if no existing browser can be reused (typical
case)
Fixes: QTBUG-109876
Change-Id: I63436a23c23c4b74c27c18effafde53bb6a4a34e
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
This is needed for static plugins to work correctly as currently
the only place where plugin logic is handled for non-standalone
builds is in qt_internal_add_executable - adding the library later
when encountered in one tests that get batched does not trigger
plugin handling logic.
Fixes: QTBUG-109812
Change-Id: Idcc2bc6b42b2b7ad6afb9796f554b0529de3de5f
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
When encountering a batchable target, only add its defines to the
sources that the target contains. Otherwise, the defines interfere
with other tests in the batch.
Fixes: QTBUG-109848
Change-Id: I35d1665d29bb0ce93b82059f7f3b715070539d21
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
This makes sure the browser only uses a basic password store and
doesn't try to do anything fancy with the OS pass store, since it
causes an issue on opensuse (kwallet isn't set up in CI).
Change-Id: Ib48a15f4834fa9bb26cf8cf3fff9bb4d0fd0238a
Reviewed-by: Mikołaj Boc <Mikolaj.Boc@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
By adding a CMake option() that is OFF by default.
Change-Id: Ic66cae5bb9739ae1db3e534f671cc1e9e4b09647
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This is done to avoid cmake syntax errors when adding new tests.
Change-Id: I8106b13a865b990c52e09573488881bd18e64c7f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
I keep getting this warning because docker-compose is not installed.
Change-Id: Id8e48e8f498c4a029619fffd172932bd86ed03ba
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Since a good number of tests use exec and friends, we want to use
asyncify for them. Asyncify is still possible to enable universally
with device options.
Change-Id: I0916f55328c98f2ba179cda3757eeee9d27f3e47
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Currently, emrun will pick the browser to run tests on somewhat
arbitrarily. For predictability and stability, we should use one
browser as a basis, and Chrome seems to be a good fit as wasm features
are generally in good shape on it and it has good support for launching
from command-line.
Change-Id: I1d06a5916ad24ab9df9b0826c0773c652e6d3fcd
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Make cmake changes that have ctest run the emrun test runner for
all tests that are build for wasm.
Change-Id: I8c07068d79cfd0d745dbcc3d3f025c7c48fe1069
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
_qt_is_test_executable can only be set on a test that is backed by a
real target. QMLTESTs might not backed by an executable target, if
they are missing cpp SOURCES.
Which led to errors like
CMake Error at
cmake/QtTestHelpers.cmake:510
(set_target_properties):
set_target_properties Can not find target to add properties to:
textedit
Call Stack (most recent call first):
tests/auto/qmltest/textedit/CMakeLists.txt:10 (qt_internal_add_test)
Amends 62c681a599
Change-Id: Ie66fd3e94484562061f851c0a034629959d091da
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
This is needed to later determine what kind of runner should be
selected on WASM for the executable. Tests use the test runner,
whereas other executables use qtLoader.
Change-Id: I75aa361403b72f8e82a288967b8a81b8232d68dc
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
If Qt itself is built without the deprecated APIs, but the tests are
not, tests will fail to compile and/or link.
This patch makes sure that the specified QT_DISABLE_DEPRECATED_UP_TO
and QT_WARN_DEPRECATED_UP_TO values are correctly used in the tests.
The definitions are propagated to tests, batched tests, manual tests
and benchmarks.
Fixes: QTBUG-104858
Change-Id: Idf15accaf96c47599084426ba625b985f507ca8b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
DISABLE_EXCEPTION_CATCHING is added unconditionally on WASM. Add
NO_EXCEPTIONS for all WASM tests since those are never supported.
Change-Id: I2ee10779e7ae0d285494ad650be52dee3099915a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Local file system files are difficult to reach on WASM. Build the test
data in so that it's reachable to WASM tests nevertheless.
Change-Id: I65022db7645248d040434759fb60c3076fd65bd8
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Extend the support of pointing at the right test from batch from WASM
to all supported platforms.
Change-Id: I9b6342ed0954f0fdc2f08c76f6b3f24f44f0388a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This is useful in some cases where extra parameters have to be set on
the test target.
Change-Id: Ic727fdaf0bc9c2e1c684d1a86352d31c0f974477
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The batch test target name is a useful thing to obtain in various
target-generating scripts.
Change-Id: I9605cf860fe1485e48108eba7e93f9064209d8fb
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
An approach of test batching (joining multiple tests into a single
binary) has been taken, due to long linking times/binary size on certain
platforms, including WASM. This change adds a new feature
'batch_test_support' in Qt testlib. Based on the value of the feature,
test batching may become enabled with the -batch-tests switch.
Batching works for every target added via qt_internal_add_test. When
first such target is being processed, a new combined target for all of
the future test sources is created under the name of 'test_batch'.
CMake attempts to merge the parameters of each of the tests, and some
basic checks are run for parameter differences that are impossible to
reconcile.
On the C++ level, convenience macros instantiating the tests are
redefined when batch_tests is on. The new, changed behavior triggered
by the changes in the macros registers the tests in a central test
registry, where they are available for execution based solely on their
test name. The test name is interoperable with the names CMake is aware
of, so CTest is able to run the tests one by one in the combined binary.
Task-number: QTBUG-105273
Change-Id: I2b6071d58be16979bd967eab2d405249f5a4e658
Reviewed-by: Topi Reiniö <topi.reinio@qt.io>
It's needed for creating qmake build tests.
CMake / CTest has a limitation of not allowing to create single-config
tests when using a multi-config generator using the add_test(NAME)
signature.
Using add_test(NAME) forcefully creates per-config tests, which means
that it's not possible to just run ctest to execute tests, without
specifying a -C parameter, which we do in the CI.
qmake tests need to use the add_test(NAME) signature
to specify the WORKING_DIRECTORY option.
Because of the above limitation, a work around is to not use the
add_test(NAME) signature, but instead delegate the working directory
assignment to a generated cmake script, which
_qt_internal_create_command_script can already do.
Pick-to: 6.4
Task-number: QTBUG-96058
Change-Id: I6f439165994671724157f0edb7a71e351271e329
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
This ensures that tests can be executed on the simulator or device, by
doing the necessary steps like setting a bundle identifier, Info.plist
file, launch screen, etc.
This is done by calling _qt_internal_finalize_executable in the
implementation of all internal test adding functions.
The finalizers are limited only to iOS for now, as an incremental
step, and to ensure we don't accidentally break tests on other
platforms.
At least WebAssembly uses its own finalizers which would likely cause
duplicate calls if the _qt_internal_finalize_executable was
unconditional.
Pick-to: 6.4
Fixes: QTBUG-104754
Change-Id: I729d56385dd206b22c975fc2ce4e2c683e6e4e2c
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.
Existing copyright statements remain intact
Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Warn projects not to use it because PUBLIC_LIBRARIES don't make
sense for executable targets and it also led to some issues in the
internal functions where some of them did not expect to receive
PUBLIC_LIBRARIES.
To ensure builds don't needlessly break, treat PUBLIC_LIBRARIES values
as regular LIBRARIES. In the future we might add an error instead.
Using PUBLIC_LIBRARIES in qt_internal_add_app, etc, accidentally
worked because the option name and the values following it were
parsed as values of the "previous" option, like SOURCES or
INCLUDE_DIRECTORIES or LIBRARIES, and when those got
passed through to qt_internal_extend_target, things magically worked.
We have a lot of projects using PUBLIC_LIBRARIES, mostly due to the
way qmake pro files were written and how pro2cmake converted them.
We'll have to clean up each repo.
Change-Id: I69e09d34afdf98f0d47c08d324643fc986f8131c
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Added a small helper function that also installs the script to
prefix builds. Additionally, leveraging the cmake option
CROSSCOMPILING_EMULATOR for the testrunner fits neatly in the rest
of our cmake code.
Change-Id: I75288e97c81b250ac3997f2e7a22bc7bd82b7b69
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Add a python script that allows us to run wasm tests in CI, along with
the necessary cmake logic to install the script and execute tests
accordingly.
Change-Id: I93b95c115538c4e27b2b833405acab8162be2a8a
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
CROSSCOMPILING_EMULATOR is a target property, not a test property.
Fixes: QTBUG-87864
Pick-to: 6.2 6.3
Change-Id: Icb07e9ed71a6bcbfceb7aa2116bf56eaa0a545c6
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
