androidtestrunner now checks is the apk is build and if it is, it will skip the build phase.
Now we can build the apks in parallel (which takes most of the time) and run them sequentially.
This way running tests on Android is much faster.
Change-Id: I82f34723ac08f7728cc0daab3366e03821335eed
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
androidtestrunner is a tool needed to run qt tests on Android.
Now you can run tests as simple as you run them on Linux, macOS,
Windows.
"$ make check" it's all you need to run tests on the default android
device.
ANDROID_DEVICE_SERIAL env variable can be used to use a specific android
serial.
Use cases:
$ make -j1 check
-j1 is needed to make sure we don't run multiple tests in parallel.
$ ANDROID_DEVICE_SERIAL="emulator-5554" make check
Run the test on "emulator-5554"
$ make TESTARGS="-- -xml" check
Switch to xml output. All params after -- are passed to test
application.
$ make TESTARGS="-- -o out.xml,xml -o out.txt,txt -o -,tap -vs" check
Create two files out.xml and out.txt in the current folder and print
"tap" format to stdout and enable logging of every signal emission.
[ChangeLog][Android] Make it easy to run Qt tests on Android.
"$ make check" is all it's needed to run a test on an Android device.
Change-Id: I1a7f64b62608f7367b5a6aabf5d6c6e7e50242e6
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
Linking with a framework suffix to pick up the debug version of
libraries doesn't work as long as the dependencies picked up from prl
files are not rewritten to also include the suffix. The result is that
we end up loading both release and debug versions of the Qt libraries.
Use the target environment (target_wrapper.sh) to set the image suffix
instead, which means 'make check' will automatically work.
Change-Id: I60b0840760f68e579c270245d394e1dd609a0ebb
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
The failure mode of this behavior is worse than the surprises that the
non-explicit library dependency chain has, so it should be opt-in.
This reverts back to the behavior in Qt 5.11, but lets our tests
opt in to the feature.
Fixes: QTBUG-71724
Change-Id: Iede11f02d978b637324ddf71d29e7c99fe3ee99f
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Eike Ziller <eike.ziller@qt.io>
Instead of having tests installed into a folder named like their
target, we now use their source folder name for the installation.
An upcoming patch will rely on this behavior and simplify creation
of tests that need helper applications.
Change-Id: I17d9ff15edf502d82ab698627189532b83e72546
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
relative paths must be resolved against $$OUT_PWD, not $$_PRO_FILE_PWD_.
Task-number: QTBUG-58991
Change-Id: I9ce8e9c78e0fad026a7cc355852d23f9d6e96ee6
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Some tests are not written to handle running on a retina screen,
resulting in additional test failures when trying to fix a CI test
failure on a local retina-enabled machine.
Change-Id: I0fed33c38792b686ac83abba2bfbc45623382200
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
On Windows cd does not change the drive. So when you are on drive C:
and type "cd D:\data" it will change the directory on drive D: but not
affect your current working directory.
To also change your drive you have to provide the parameter /d on
Windows, so "cd /d D:\data" will also change the drive.
Task-number: QTBUG-57080
Change-Id: Ib629879534523982eec693cef725f20a535a1a74
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
Consistent with other Unix platforms, and internally consistent between tests,
as a lot of tests were already applying CONFIG -= app_bundle manually.
Change-Id: Icd2b7e1c08015b26137af60ff82fddbc753f0ff4
Reviewed-by: Jake Petroules <jake.petroules@qt.io>
For platforms which require packaging the testdata into the application
resources, the BLACKLIST file needs to be added. Otherwise it will not
be found during runtime.
Change-Id: I2d5c3b3040b3b268bc73254459b8b902b7fae4e2
Reviewed-by: Oliver Wolff <oliver.wolff@qt.io>
This change partially reverts 1bfc7f68 about QT_HAS_BUILTIN define
and undef in src/corelib/tools/qsimd_p.h.
This change is also squashed with "Fall back to c++11 standard
compiler flag for host builds" which is done by Peter Seiderer.
Conflicts:
mkspecs/features/default_post.prf
src/3rdparty/sqlite/0001-Fixing-the-SQLite3-build-for-WEC2013-again.patch
src/3rdparty/sqlite/sqlite3.c
src/corelib/tools/qsimd_p.h
src/gui/kernel/qevent.cpp
src/gui/kernel/qwindowsysteminterface.cpp
src/gui/kernel/qwindowsysteminterface_p.h
src/plugins/bearer/blackberry/blackberry.pro
src/plugins/platforms/cocoa/qcocoasystemsettings.mm
src/plugins/platformthemes/gtk2/gtk2.pro
src/plugins/styles/bb10style/bb10style.pro
src/sql/drivers/sqlite2/qsql_sqlite2.cpp
tools/configure/configureapp.cpp
Task-number: QTBUG-51644
Done-with: Peter Seiderer <ps.report@gmx.net>
Change-Id: I6100d6ace31b2e8d41a95f0b5d5ebf8f1fd88b44
Testcases and benchmarks are rather different entities. You won't usually want
to run benchmarks in the same environment you are wanting to run tests in,
so this feature allows to differentiate between the two.
We also add a "benchmark" make target (similar to check), which runs all
configured benchmarks.
Change-Id: I33759ce44c34e42a6a3a88f34e7b9c4372380721
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@theqtcompany.com>
the primary purpose is making env var prepend mode work for unset
variables on windows. this is achieved by using a conditional and delayed
variable expansion. however, the latter is disabled by default and can
be locally enabled only in batch files. therefore, write wrapper scripts
and substitute them for the actual commands. we do this also on unix,
both for consistency and simply because the commands look much less
confusing.
this change is slightly backwards-incompatible, as invoking
qtAddToolEnv() multiple times on the same command will now make a total
mess. also, invoking it on a command that contains 'make' macro
expansions isn't a good idea, so testcase.prf needed an adjustment. the
function is an undocumented internal, so Nobody Should Care (TM).
this also reverts 80ebedecf9, as it's obsolete now.
Change-Id: I8394b77868b495abcf27b688996ca74c40b80994
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
QFINDTESTDATA is already prepared to find it there.
Change-Id: I467392786ce6bcfbf1bd0b6079f60c9df06834b1
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@theqtcompany.com>
Reviewed-by: Oliver Wolff <oliver.wolff@theqtcompany.com>
Reviewed-by: Maurice Kalinowski <maurice.kalinowski@theqtcompany.com>
qt5 qmake is perfectly capable of complex expressions on the LHS.
Change-Id: Ibf8c82a4aa1a419895c6012610269e1cc9ca93ab
Reviewed-by: Joerg Bornemann <joerg.bornemann@theqtcompany.com>
qt_framework and {app,lib}_bundle imply darwin, so there is no point in
testing for it.
Change-Id: I9fe48c26c8e271a5575b17e92df8674d3c3a3204
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
We now have support for more complex exclusive builds configurations,
e.g., on iOS with simulator and device configurations, so we can't hard
code the logic for choosing the right exclusive build to test.
Change-Id: I358687b297b7bf1eb28eef0ef0aaf44b89860404
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@theqtcompany.com>
If the user adds CONFIG+=bundle and doesn't set QMAKE_BUNDLE_EXTENSION,
we interpret that as a request to generate a test bundle.
Change-Id: Id4fbb64d39cddd6f95ac641a910a9d5543c30daa
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@theqtcompany.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@theqtcompany.com>
prepend the 'cd' command only after prepending the variables, as
otherwise they'd apply to the cd command only.
Task-number: QTBUG-44183
Change-Id: Ibf96a16ce2c9cd9c0e80ca3cd5433e64ec19b136
Reviewed-by: Jake Petroules <jake.petroules@petroules.com>
"system" refers to the system's native shell, which is what qmake's
system() invokes, and whose convention by far most commands invoked from
a makefile will need.
"shell" refers to the shell invoked by make, which diverges from the
system shell only when qmake/mingw32-make is called from an msys shell.
its conventions need to be used for anything the shell itself does
(e.g., assembling env variables, but also command line argument
unquoting) and the commands the mkspec sets according to the shell
(e.g., QMAKE_MOVE).
Change-Id: I0000aa9417c199cf8a810619d31ded24bb0675f9
Reviewed-by: Joerg Bornemann <joerg.bornemann@digia.com>
testcase.prf cannot be loaded from pro file for various reasons,
see qtbase commit history for details.
Moved runtime testdata logic from pro file to testdata.prf, and
thus made is reusable in other test cases as well.
Change-Id: I500d08dc4951e4eda862071e4ddd3e0f6de8c3d2
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
Shared between UNIX and Win generators, and allows prfs after
default_post to rely on sane TARGET and DESTDIR values.
This allows us to clean up the DESTDIR logic in testcase.prf,
which was completely busted. Doing the two in separate commits
is unfortunately not possible as the old testcase.prf logic
was so broken it would barf if only looked at.
Change-Id: Ibf21216195c760ee46ae679c162b207b77a9d813
Reviewed-by: Jan Arve Sæther <jan-arve.saether@digia.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@digia.com>
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
the precise syntax depends on what exactly the command is used for, so
we need to resolve it at the last moment. see followup commits.
This logically reverts commits 6f4ff81380
and 731e6bece5.
Change-Id: If285c91d7521069be86d32593b5c2ae2027b3038
Reviewed-by: Joerg Bornemann <joerg.bornemann@digia.com>
The former applies both on Mac OS X and iOS, but 'macx' is specific to
Mac OS X.
ios.conf and macx.conf now share most of their settings in the common
mac.conf. We set the default QMAKE_MAC_SDK before loading mac.conf, so
that any overrides in the device config will apply afterwards. This
means configure's mkspec parsing will be able to read the QMAKE_MAC_SDK.
Change-Id: I0c7e26a6a0103e19b23ef152aa9e4ab461cee632
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@digia.com>
that way we don't have to auto-generate code for that in the configures.
note that we now load qt_build_config.prf instead of just qmodule.pri,
which means that exceptions_off is set everywhere. we forcibly re-enable
them for testcases to minimize the deviation from default 3rd party usage.
testlib selftests are not qt testcases, so the one that needs exceptions
needs to enable them explicitly.
Change-Id: I1b9360bb11f2e80c92a2b63a7c45991ad17fda1b
Reviewed-by: Joerg Bornemann <joerg.bornemann@digia.com>
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@digia.com>
Make test projects declare TEST_HELPER_INSTALLS rather than calling a
function exported by testcase.prf. load(testcase) may be unsafe, as
testcase.prf should be processed after default_post.prf.
Fixes silent disabling of various autotests.
Change-Id: I56b35ffd653a637ad5ab18d64dd1a1edadfac59f
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@nokia.com>
Reviewed-by: Kurt Korbatits <kurt.korbatits@nokia.com>
the check whether we are building a lib or an app (and thus have a target)
is done by quite some feature files (and generally wrongly, as they do not
account for the new aux target), so centralize it in default_post.prf.
Change-Id: I868edbc4185be8a6c23ecd4a2c126024d73cdeb4
Reviewed-by: Joerg Bornemann <joerg.bornemann@nokia.com>
- Changed installTestHelperApp() to use OUT_PWD path to find files.
(was looking in source tree instead of build)
Change-Id: If795c17a76aebcfbbe0d4f84c5df784a255ddb19
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
qmake uses the variable name verbatim, which makes the generated
makefile look "interesting" without this cleanup.
Change-Id: If6c4f12563a0ee6429513fbfac534f40bb7c0b97
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The "check" target always depended on debug-check on projects that
explicitly loaded the testcase.prf, even if Qt was configured with
-release parameter. This obviously caused build failure. Changed the
testcase.prf to check which configuration is preferred using
CONFIG(debug, debug|release) like is done with similar cases in other
.prf files.
Task-number: QTBUG-24332
Change-Id: Ib5140b106e99efe51932bdd5a48914786de23230
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@nokia.com>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
- All subprograms installed as part of test instead of separate installs
- Added installTestHelperApp() to testcase.prf
Change-Id: I02fbbb299f095054c9216ad0e5f91f574fb0fe3d
Reviewed-by: Kurt Korbatits <kurt.korbatits@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Support the common construct of a top-level test directory having
a test/test.pro with:
TARGET = ../tst_sometest
TESTDATA = ../data1.txt ../data2.txt
Prior to this change, the ".." in TESTDATA would cause the data to
break out of the test's installation directory.
Change-Id: I22860bf3a148f278b3f4e18b476fd151f7f0f775
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@nokia.com>
This variable was renamed from `elem' to `file' at some point in
development, but the old name was left at one place.
Change-Id: I2cda42b02a753614176b466e507e8d8d9ed80a6e
Reviewed-by: Kalle Lehtonen <kalle.ju.lehtonen@nokia.com>
Tests will install under $$[QT_INSTALL_TESTS].
TESTDATA may be used to install additional testdata required by the
test.
The default install rule may be disabled by
CONFIG+=no_testcase_installs.
Change-Id: I204de60c8e844775906ffd016ca50bffbb414142
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@nokia.com>
Marking a test with CONFIG+=insignificant_test will cause the exit code
of the test to be discarded during `make check'. This is intended to be
used for tests which are valuable to run, but are known to be unstable
and are not feasible to immediately fix.
Reviewed-by: Jason McDonald
Change-Id: I50a712c33c2ebb0af39f1ea0bf2adef7f0936425
This is the beginning of revision history for this module. If you
want to look at revision history older than this, please refer to the
Qt Git wiki for how to use Git history grafting. At the time of
writing, this wiki is located here:
http://qt.gitorious.org/qt/pages/GitIntroductionWithQt
If you have already performed the grafting and you don't see any
history beyond this commit, try running "git log" with the "--follow"
argument.
Branched from the monolithic repo, Qt master branch, at commit
896db169ea224deb96c59ce8af800d019de63f12
