Commit Graph

367 Commits

Author SHA1 Message Date
Mårten Nordheim
e75e89f9de Test QTest's signal dumper
It has been completely untested for a while and saw some breakage.
So let's add a selftest for it. ('-vs' when running tests)

Change-Id: Ibfb5ac0a2d741de7c3f519d91202d4977996045e
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2019-03-27 08:56:09 +00:00
Edward Welbourne
ab53f0f24e QtTestLib: unify handling of float and double using suitable templates
The QTest::qCompare() implementations were almost duplicates; pull the
common code out into a templated version.  Tweaked the
QTest::toString() specialization for float and double (a macro) and
fixed a bous modifier in double's format.

The doubleComparisons and floatComparisons tests in the tst_float.cpp
selftest shared a large block of tests in common, aside from the
difference of type.  Break this out into a templated static function
to save duplication.

This prepares the way for using the same templated code for qfloat16.

Change-Id: I2823fd006910c5ff88335d625d1fa05cb7753513
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2019-03-04 10:01:35 +00:00
Liang Qi
388fe97f2a Merge remote-tracking branch 'origin/5.13' into dev
Conflicts:
	src/corelib/tools/qstring.cpp

Change-Id: I81dbf90fc936c9bf08197baefa071117bddb1c63
2019-05-13 08:04:58 +02:00
Edward Welbourne
3abfa4dfff QtTestLib: handle float16 the same as double and float
In QCOMPARE, handle NaNs and infinities the way tests want them
handled, rather than by strict IEEE rules. In particular, if a test
expects NaN, this lets it treat that just like any other expected
value, despite NaN != NaN as float16 values. Likewise, format
infinities and NaNs specially in toString() so that they're reported
consistently.

Enable the qfloat16 tests that depend on this QCOMPARE() behavior.
Refise the testlib selftest's float test to test qfloat16 the same way
it tests float and double (and format the test the same way).

This is a follow-up to 37f617c405.

Change-Id: I433256a09b1657e6725d68d07c5f80d805bf586a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2019-04-24 13:09:15 +00:00
Qt Forward Merge Bot
035f934d7a Merge remote-tracking branch 'origin/5.13' into dev
Conflicts:
	tests/auto/widgets/itemviews/qheaderview/tst_qheaderview.cpp

Added tests/auto/testlib/selftests/expected_crashes_5.txt to work
round the output of the crashes test (which exercises UB, see
QTBUG-73903) being truncated on one test platform.

Change-Id: I9cd3f2639b4e50c3c4513e14629a40bdca8f8273
2019-02-20 09:51:12 +01:00
Edward Welbourne
f20adcde30 Dodge qFatal() so as to get coverage results despite it
The silent and blacklisted selftests of testlib end in a qFatal(), to
test its messaging is handled correctly.  However, this prevents hooks
in main() from saving coverage data when we're gathering that.  So use
a transient signal handler that longjmp()s back to a setjmp() just
before the qFatal() to let the test complete normally (but, since
qFatal() does something different on MS-Win, don't apply this to it).

Note that testlib's internal FatalSignalHandler handles all fatal
signals *except* SIGABRT, so this isn't over-riding it.  (In any case,
this restores the prior signal handler in setjmp()'s catch branch.)

Added missing expected_silent.tap test output while checking that this
change doesn't affect (the rest of) the test output.

Change-Id: I7e460581ad93e26639c066b3229438a66fd299de
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2019-02-11 15:38:29 +00:00
Liang Qi
fbfacd33be Merge remote-tracking branch 'origin/5.12' into 5.13
Conflicts:
	src/android/templates/AndroidManifest.xml
	src/network/ssl/qsslsocket_mac.cpp
	src/widgets/styles/qstylesheetstyle.cpp
	tests/auto/corelib/kernel/qtimer/BLACKLIST
	tests/auto/testlib/selftests/blacklisted/tst_blacklisted.cpp
	tests/auto/testlib/selftests/expected_blacklisted.lightxml
	tests/auto/testlib/selftests/expected_blacklisted.tap
	tests/auto/testlib/selftests/expected_blacklisted.teamcity
	tests/auto/testlib/selftests/expected_blacklisted.txt
	tests/auto/testlib/selftests/expected_blacklisted.xml
	tests/auto/testlib/selftests/expected_blacklisted.xunitxml
	tests/auto/testlib/selftests/expected_float.tap
	tests/auto/testlib/selftests/expected_float.teamcity
	tests/auto/testlib/selftests/expected_float.txt
	tests/auto/testlib/selftests/expected_float.xunitxml

Done-With: Christian Ehrlicher <ch.ehrlicher@gmx.de>
Done-With: Edward Welbourne <edward.welbourne@qt.io>
Done-With: Timur Pocheptsov <timur.pocheptsov@qt.io>
Change-Id: If93cc432a56ae3ac1b6533d0028e4dc497415a52
2019-02-08 12:31:02 +01:00
Edward Welbourne
150c6fb74b Add testlib selftests for double and for non-finite float and double
Tidied up the existing float tests in the process.
(In particular, s/SUCCESS/PASS/ since that matches real test output.)
These verify that QCOMPARE() handles floats and doubles as intended.
Extended the existing qFuzzyCompare tests to probe the boundaries of
the ranges of values of both types, in the process.

Revised the toString<double> that qCompare() uses to give enough
precision to actually show some of the differences being tested there
(12 digits, to match what qFuzzyCompare tests, so as to show different
values rather than, e.g. 1e12 for both expected and actual) and to
give consistent results for infinities and NaN (MinGW had eccentric
versions for these, leading to different output from tests, which thus
failed); did the latter also for toString<float> and fixed stray zeros
in MinGW's exponents (which made a kludge in tst_selftest.cpp
redundant, so I removed that, too).

That's further complicated handling of floating-point types, so let's
just keep an eye on how expensive that's getting by adding a benchmark
test for QTest::toString().  Unfortunately, default settings only get
runs that take modest numbers of milliseconds (some as low as 40)
while increasing this with -minumumvalue 100 or more gets the process
killed - and I'm unable to find out who's doing the killing (it's not
QProcess::kill, ::kill or the QtTest WatchDog, as far as I can tell).
So results are rather noisy; the integral tests exhibit speed-ups by
factors up to 5, and slow-downs by factors up to 100, between runs
with and without this change, which does not affec the integral tests.
The relatively modest slow-downs and speed-ups in the floating point
tests thus seem likely to be happenstance rather than signal.

Change-Id: I4a6bbbab6a43bf14a4089e96238a7c8da2c3127e
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
2019-02-06 10:11:22 +00:00
Edward Welbourne
d84912838c Add some missing expected_*.tap files for the testlib selftest
Only for tests that have existing expected_*.* files for other
formats, though.

Change-Id: I34ca1900d88454f300e04d849a608c378009489b
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2019-02-01 14:42:23 +00:00
Edward Welbourne
7dd41b3267 generate_expected_output.py: placate FutureWarning from python
Regexes have long specified that a [ as the first character inside a
[...] is just a literal [, but apparently we need to escape it now, to
avoid a "nested set" FutureWarning.

Change-Id: I76a48c9aafb0684a1d6b0d5284fe9852c9ea0e43
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Samuel Gaist <samuel.gaist@idiap.ch>
2019-01-31 12:07:20 +00:00
Albert Astals Cid
743b3d8673 Add qabstractitemmodeltester back to the build
Change-Id: I75dcfaa692465d1a96b92c81111187c4b3aeed6b
Reviewed-by: Mikhail Svetkin <mikhail.svetkin@qt.io>
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-01-31 09:44:53 +00:00
Albert Astals Cid
4db3faef6b Change the FIXME for selftests/alive
It isn't being built in the current qmake build either (maybe because it
doesn't build because uses sleep())

Change-Id: Ie97287c6e45d1fcc32063bd9eb37ff2907dfa891
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
2019-01-30 16:11:13 +00:00
Oliver Wolff
9906cc57ed testlib: Add BXPASS and BXFAIL
Prioritize blacklisting over QEXPECT_FAIL so that a test that is
blacklisted no longer fails if QEXPECT_FAIL returns true unexpectedly. To
reflect this state properly, the two values of BXPASS and BXFAIL were
added to testlib's output.

[ChangeLog][Important Behavior Changes][QtTestLib] Blacklisting of tests
will be taken into account for XPASS and XFAIL. A blacklisted test that
causes an XPASS will no longer be a fail.

Task-number: QTBUG-72928
Change-Id: Ia2232fdc714d405fa3fd9aea6c89eb2836bc5950
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2019-01-29 12:18:15 +00:00
Daniel Smith
991c498905 Fix inline ASM to use local labels for jump operations
Non-local ASM labels break for this test when compiled with clang.

Change-Id: I15bd250a991c3b03bbc88459a6358090bd157444
Reviewed-by: Jesus Fernandez <Jesus.Fernandez@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2019-01-25 08:45:54 +00:00
Edward Welbourne
3a5f86d784 Add a feature to enable CodeCoverage analysis of testlib
Based on Asmo Saarela's advice (QTPM-686), adapted on advice from
FrogLogic support and converted to a feature so that the selftest and
testlib qmake config can be co-ordinated.

Task-number: QTPM-1385
Change-Id: Icd706f086009e1e08b3f8c5cd553f792402e28c0
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
2019-01-24 15:51:13 +00:00
Edward Welbourne
f3de22b211 Add assorted tests to exercise more QTest functions
Coverage analysis reveals some unexercised code.

Task-number: QTPM-1385
Change-Id: Ib3fdbcf778e7af14ca6dfa1a11bcb4943276de69
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2019-01-24 15:51:04 +00:00
Qt Forward Merge Bot
f94ca82e0f Merge remote-tracking branch 'origin/5.12' into dev
Conflicts:
	.qmake.conf

Change-Id: Ibfcb30053f3aacb8ec2ec480e146538c9bf440ea
2019-01-21 13:03:09 +01:00
Edward Welbourne
a53500f5bf Eliminate some stray misleading paths from expected output files
The output files for the tuplediagnostics selftest of testlib had a
stray non-canoical path fragment in them; so replaced with its
canonical form.

Change-Id: Ib421380036c3fb1b91447eb8c87be4ad0dfe5c96
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2019-01-16 18:47:41 +00:00
Edward Welbourne
7c7e007794 Add a testlib selftest for the watchdog
Verify that it does cut in after the specified time has elapsed.

Task-number: QTPM-1385
Change-Id: Ib18e8d6af28339f79cca4d62b869287ce07b8cc1
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-12-06 18:29:47 +00:00
Edward Welbourne
c3e6551d4d generate_expected_output.py: refine environment used for subtests
Match the environment tst_selftests.cpp uses for subtests more
faithfully.  Extends b22e50acda.  In the process, tweak how crashers
are handling, in preparation for the watchdog test.

Change-Id: I09a046460f6f3bff0b12069fad6c1437d89572ce
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-12-06 18:29:35 +00:00
Edward Welbourne
dfcb7fce35 Test that mismatches of datatype in test data columns are rejected
One test for bad data for the column, another for a bad QFETCH.
Incidentally extend blacklist testing by blacklisting them.
Reorganise a QEMU condition that needed extended as part of this.

Task-number: QTPM-1385
Change-Id: Iac72ada19760321c5c9264ddfff7740d1fdd0700
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-12-06 18:29:22 +00:00
Liang Qi
5d5c00c676 Merge remote-tracking branch 'origin/5.12' into dev
Conflicts:
	src/gui/painting/qdrawhelper.cpp

Change-Id: I4916e07b635e1d3830e9b46ef7914f99bec3098e
2018-12-04 09:58:43 +01:00
Qt Forward Merge Bot
1b5bbacdb0 Merge remote-tracking branch 'origin/5.11' into 5.12
Change-Id: I12bcee17e349edd0dd4fd08da76361d1ffb1a727
2018-11-27 01:00:09 +01:00
Sami Nurmenniemi
71bd06d516 Make developer build tests pass for boot2qt
Some tests were fixed and others were skipped/blacklisted.

Task-number: QTBUG-63152
Change-Id: Ica7df555f8d152ee589865911130525101d4b941
Reviewed-by: Liang Qi <liang.qi@qt.io>
2018-11-26 07:06:59 +00:00
Allan Sandfeld Jensen
37f617c405 Add qfloat16 support to QCOMPARE
Change-Id: Ide06f215a888328308a06e7e48edd666f790a5f0
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
2018-11-20 21:34:13 +00:00
Qt Forward Merge Bot
0441d959ff Merge remote-tracking branch 'origin/5.12' into dev
Change-Id: I5dbdc13c6133e5b03e362c5461b4a599d781bd1e
2018-11-13 01:00:13 +01:00
Tor Arne Vestbø
27fb51fa52 macOS: Remove blacklist entries for no longer supported OS versions
Change-Id: Iae6552f1fdcf1dea07a03d3788d378af9140d1a7
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2018-11-12 07:16:30 +00:00
Christian Ehrlicher
76bb804405 QTreeWidget: mark (is|set)Item(Selected|Hidden|Expanded) as deprecated
QTreeWidget::(is|set)Item(Selected|Hidden|Expanded)() are deprecated
for a long time but not marked as such. Therefore explicitly mark them
as deprecated so they can get removed with Qt6.

Change-Id: Ie4971350de61326811e0788df0d359ed3c442869
Reviewed-by: Konstantin Shegunov <kshegunov@gmail.com>
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@qt.io>
2018-11-09 19:07:51 +00:00
Edward Welbourne
3e36fb641e Test and document QTest::toHexRepresentation()
It's not actually exercised by anything in our source tree, but is
potentially useful and has been part of the documented public API for
some time.  So mention that the caller is responsible for delete[]ing
its return and add a test that exercises it.

Task-number: QTPM-1385
Change-Id: Ifc5284b9eb1b678cf3c9708c681311e874838fc6
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-11-08 15:22:21 +00:00
Christian Ehrlicher
d0f909f8db QTreeView/TableView: explicitly mark sortByColumn(int) as deprecated
QTreeView/TableView::sortByColumn(int) was deprecated a long time ago
but never got removed. Therefore mark it with QT_DEPRECATED_SINCE(5, 13)
so we can remove it with Qt6.
Also sync the handling of the sort order changes in QTableView with the
one from QTreeView.

Change-Id: I0371d9a9c21116edaa9125835827f1a200075d36
Reviewed-by: Luca Beldi <v.ronin@yahoo.it>
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@qt.io>
2018-11-02 05:56:58 +00:00
Edward Welbourne
33d7f76f0e Tweak a selftest blacklist to exercise more of the blacklisting code
In the process, corrected an inaccurate XFAIL message (an XPASS is
normally an error, unless blacklisting ignores it so turns it into a
BPASS). Added the missing .tap file to its expected output.
Documented the similarity to the silent/ selftest.

Task-number: QTPM-1385
Change-Id: Id74a1353d54af2f3bfe2c764e33c1f051958ab21
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-11-01 15:06:43 +00:00
Edward Welbourne
b22e50acda generate_expected_output.py: match tst_selftest's test environments
The testlib selftest sets various things in the environment for
crashing tests; the generator for its expected output should set the
same things, as they affect what output is produced.

Change-Id: Iec2ed59982ea1043582573530c33619d8e8ed08e
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-11-01 15:06:08 +00:00
Edward Welbourne
30101884a6 Simplify tst_Selftests::compareLine by handling the trivial case first
If the two lines have identical texts, the comparison returns true.
So don't complicate various other conditions on the way there with
filtering out that case; deal with it first so they don't need to.

Change-Id: Iebd230704ce5f53d12d5afa64aab30f83bb9d407
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-11-01 15:05:59 +00:00
Simon Hausmann
e9c45bbddd Begin port of qtbase to CMake
Done-by: Alexandru Croitor <alexandru.croitor@qt.io>
Done-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Done-by: Kevin Funk <kevin.funk@kdab.com>
Done-by: Mikhail Svetkin <mikhail.svetkin@qt.io>
Done-by: Simon Hausmann <simon.hausmann@qt.io>
Done-by: Tobias Hunger <tobias.hunger@qt.io>
Done-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Done-by: Volker Krause <volker.krause@kdab.com>
Change-Id: Ida4f8bd190f9a4849a1af7b5b7981337a5df5310
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
Reviewed-by: Mikhail Svetkin <mikhail.svetkin@qt.io>
2018-11-01 11:48:46 +00:00
Jüri Valdmann
cea2b5510c Fix building tests with -no-gui
Change-Id: I37307080e5adc334fcfcdd2fee650d675228a746
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
2018-10-20 20:38:59 +00:00
Liang Qi
e3ed2281c0 Merge remote-tracking branch 'origin/5.11' into dev
Conflicts:
	src/plugins/platforms/cocoa/qnsview.mm
	src/plugins/platforms/cocoa/qnsview_dragging.mm
	src/plugins/platforms/ios/qiosinputcontext.mm
	src/plugins/platforms/xcb/qxcbconnection.cpp
	src/plugins/platforms/xcb/qxcbconnection_xi2.cpp
	src/plugins/platforms/xcb/qxcbwindow.cpp
	src/tools/androiddeployqt/main.cpp
		Was moved from qttools into qtbase in 5.11.
		So re-apply 32398e4d here.
	tests/auto/corelib/global/qlogging/test/test.pro
	tests/auto/corelib/global/qlogging/tst_qlogging.cpp
	tests/auto/corelib/io/qfile/tst_qfile.cpp
	tests/auto/corelib/kernel/qtimer/tst_qtimer.cpp
	tests/auto/corelib/thread/qthreadstorage/test/test.pro
	tests/auto/widgets/itemviews/qheaderview/tst_qheaderview.cpp
	tests/auto/widgets/kernel/qapplication/test/test.pro

Done-with: Gatis Paeglis <gatis.paeglis@qt.io>
Done-with: Mårten Nordheim <marten.nordheim@qt.io>
Done-with: Oliver Wolff <oliver.wolff@qt.io>
Change-Id: Id970486c5315a1718c540f00deb2633533e8fc7b
2018-07-02 11:23:45 +02:00
Liang Qi
c3059391fe Make tests compile for Android
This only enables compilation, it doesn't fix any test.

Qt on Android supports process, but not TEST_HELPER_INSTALLS. See also
acdd57cb for winrt.

android-ndk-r10e is used to compile, see
http://doc-snapshots.qt.io/qt5-5.11/androidgs.html .

corelib/io/{qdir,qresourceengine} need to be fixed later.

Done-with: Frederik Gladhorn <frederik.gladhorn@qt.io>
Done-with: Mårten Nordheim <marten.nordheim@qt.io>
Change-Id: I34b924c8ae5d46d6835b8f0a6606450920f4423b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2018-06-25 08:01:19 +00:00
Mårten Nordheim
4361c0ee84 findtestdata: test 'relative to test source'-fix
Before the fix is applied this test fails because QFINDTESTDATA will
return "/usr/" instead of the folder with the same name in the current
directory.

The 'usr' folder can't be located 'next to' the application since this
does not trigger the issue (QFINDTESTDATA looks for the folder next
to the executable early on). So we put it in a subdirectory and change
the current working directory to its parent directory.

Change-Id: I627679dcb6f2f6954264e23bfc1a71de3bff7203
Reviewed-by: Jesus Fernandez <Jesus.Fernandez@qt.io>
2018-06-22 13:05:03 +00:00
Samuel Gaist
04b180f7f2 Improve std::tuple handling in tests
Currently when doing comparison with std::tuple the fallback toString
method is called which returns a Q_NULLPTR thus not allowing proper
diagnostic of the values that triggered an error. This patch
adds support for std::tuple to improve the tests output readability.

[ChangeLog][QtTest][QCOMPARE] Now outputs contents of std::tuple on
failure.

Change-Id: I046a55e2ce44c3f7728d51e4745120d38aa5e007
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2018-06-03 20:26:38 +00:00
Edward Welbourne
fec96d9a9f Remove mention of selftest.qrc from generator script's doc
This supplements b1945604a7, which
removed the qrc file in favor of test/test.pro coding for it.

Change-Id: I15507c89ca14fa6e6b8223de671ffff7092272d0
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2018-05-08 12:45:25 +00:00
Mitch Curtis
4c92fce761 Fix order of mouse selftest in selftests.pri
Put it in alphabetical order like the rest of the list.

Change-Id: I3da3bb68d1847f53419bb79490b946c935ebb518
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2018-05-07 14:53:26 +00:00
Mitch Curtis
630b06f86c Add self-tests for QTest::keyPress() and QTest::keyRelease()
They didn't exist up until now, and future patches rely on them, so
add them.

Change-Id: I8afdb9417263b45d43355c688a813bdf99ea5fc8
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2018-05-07 14:53:18 +00:00
Edward Welbourne
3ac029f674 Make generated selftest output match for in-source builds
When generate_expected_output.py is run for an in-source build, the
raw output contains no paths to the sources for the script to whittle
down, as it does for shadow builds, to just the path from qtbase down.
So kludge together some extra regexes that can fix that up and tweak
some relevant code to provide them with the data they need.

Change-Id: I656d7126087bd9ad20b2af6835fba314d90a171d
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2018-04-17 16:22:49 +00:00
Tor Arne Vestbø
f69d32b535 testlib: Ignore line locations from TAP reporter when running selftests
This is what the other reporters also do, in various forms.

Task-number: QTBUG-67351
Change-Id: I16f2c4e0991176145ee0fbcbbfeeda071603a3c2
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-27 19:34:04 +00:00
Qt Forward Merge Bot
d58f3c4878 Merge remote-tracking branch 'origin/5.11' into dev
Change-Id: Icf3b9346117ce7149d8687e4cfa182e7586713f3
2018-03-18 01:00:11 +01:00
Friedemann Kleint
ff1630ab0e Stabilize tst_QSignalSpy::waitFunctionPointer_signalEmittedTooLate()
Replace qWait()/QCOMPARE() by QTRY_COMPARE().

Change-Id: Ic5233e2f26e6663a3fbcbb2edb09a517feef4564
Reviewed-by: Sami Nurmenniemi <sami.nurmenniemi@qt.io>
Reviewed-by: Shawn Rutledge <shawn.rutledge@qt.io>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
2018-03-17 12:10:05 +00:00
Qt Forward Merge Bot
8264e495fa Merge remote-tracking branch 'origin/5.11' into dev
Change-Id: I8b5a10d897a926078895ae41f48cdbd2474902b8
2018-03-15 01:00:11 +01:00
Tor Arne Vestbø
3b42e098ef testlib: Add Test Anything Protocol (TAP) reporter
The Test Anything Protocol (TAP), was originally Perl's simple text-based
interface between testing modules and test harnesses, but has since been
adopted by a large number of producers and consumers in many different
languages, which allows colorizing and summarizing test results.

The format is very simple:

TAP version 13
ok 1 - test description
not ok 2 - test description
  ---
  message: 'Failure message'
  severity: fail
  expected: 123
  actual: 456
  ...
ok 3 - test description # SKIP
1..3

The specification [1] is very brief, so the implementation has been
based on how typical consumers behave, especially when it comes to
the undefined diagnostics block.

[1] http://testanything.org/tap-version-13-specification.html

Change-Id: I616e802ea380165c678510e940ddc6607d39c92d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-14 14:28:36 +00:00
Tor Arne Vestbø
8864aca9e5 Make testlib selftests pass for shadow-builds
Using QFileInfo to check if the file exists based on the filename
alone ignores the fact that all the expected files are embedded
as QRC resources.

The expectedResult() function already does a similar check, so we
can use that directly instead of checking twice if the file exists.

Task-number: QTBUG-66981
Change-Id: I0beb8d3503ed49682ae7d7e2a5172922fab5420d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-14 14:28:34 +00:00
Tor Arne Vestbø
3ae03c3585 testlib: Silence all Qt logging in selftests
Otherwise random debug messages from Qt might mess up the expected vs
actual results.

The setting of QT_LOGGING_RULES in initTestcase has been removed, as
the selftest overrides that for each invocation of a subtests, via
the processEnvironment() function.

Task-number: QTQAINFRA-1631
Change-Id: I855d31274f8261f8b125df23409353f7101be0e4
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2018-03-12 12:09:51 +00:00
Tor Arne Vestbø
b1945604a7 testlib selftest: Resolve expected files at qmake time
Instead of keeping a static list in a qrc file.

Change-Id: I7783db5cad5929cc3f449889e1cbcbb87f7f2637
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-09 16:36:08 +00:00
Qt Forward Merge Bot
8c04a5e964 Merge remote-tracking branch 'origin/5.11' into dev
Change-Id: I967eca8f34584fca3a1e5696978e70f620582a2a
2018-03-02 14:40:07 +01:00
Friedemann Kleint
dcc16166ef QTestlib/selftests: Fix failures due to logging system output
Silence debug output by setting QT_LOGGING_RULES
to turn off all debug output.

Task-number: QTQAINFRA-1631
Change-Id: I5c2366b4fe4bac341dcfd92f68b6da8071c5b089
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
2018-03-02 08:16:21 +00:00
Qt Forward Merge Bot
6cf3428a43 Merge remote-tracking branch 'origin/5.11' into dev
Change-Id: I42ec9e6aafc203465cbeb88af70c7af26b7df8ed
2018-03-02 01:00:10 +01:00
Gatis Paeglis
b3e91b66b9 qtestlib: fix support for pressing multiple mouse buttons
After a37785ec76 went in, it become
apparent that multi mouse button state handling in qtestlib is
non-existent, for details see QTBUG-64030 and QTBUG-63786. What
happened behind the scenes often was not what one would expect based
on the provided QTest::mouse* input sequence - events went missing,
incorrect events were generated, each subsequent test function
started with a state set from the function that run earlier. It is
easy to see how a minor change in one test could easily affect outcome
of other tests.

With a37785ec76, Qt platform plugins
are now responsible for sending explicit mouse button type and state
information; qtestlib should take full responsibility now as well.
But using the new API from a37785ec7 alone in qtestlib is not sufficient.
We need to reset mouse state between each new test function run (we do
this at function scope as that fits with the current qtestlib API user
expectations). This patch implements the necessary reseting logic.

Updated tst_qwindow.cpp::generatedMouseMove() to use QTest::mouse* APIs.
That test requires pressing multiple buttons, it was not possible with
QTest::mouse* APIs before this patch.

Added an auto test for multiple mouse button pressing/release in
tests/auto/testlib/selftests/mouse/. And few other tests which are
currently QSKIP-ed, but should be considered when re-designing qtestlib
APIs.

Task-number: QTBUG-64030
Change-Id: I39fdcbc73a467a7463ce2aed622bf22484095635
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2018-03-01 19:41:10 +00:00
Friedemann Kleint
9760f881c5 QTestlib: Add formatting for QFlags<>
Add formatting for registered enumerations based on QMetaEnum
and unregistered enumerations as hex values.

[ChangeLog][QtTest] QtTest now prints values of QFlags that failed to
compare with QCOMPARE.

Task-number: QTBUG-65845
Change-Id: I3eae6d20d3c0d72441ca6c4037d9a8dafa4b6357
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2018-02-16 17:27:55 +00:00
Friedemann Kleint
a7600bf604 QTestlib/selftests: Refactor comparison
The "crashes" subtest has several "expected" reference files. The matching
one was previously determined by checking the line count of the output.

This however does not work when there are several reference files with
identical line count as is now the case with boot2qt.

Refactor the comparison code from the QTest/preliminary void return to
the bool f(QString* errorMessage) convention so that all files can be tried.

While doing so, streamline the code and remove numerous unneeded
allocations of strings and regular expressions per compared line.

Task-number: QTBUG-65845
Change-Id: I722159d1753f2a36f0e497e315ffd81cb58cac0b
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2018-02-14 16:07:10 +00:00
Friedemann Kleint
8692f12e89 QTestlib/selftests: Add expected_crashes_4.txt for boot2qt
It has been observed that sometimes the output is not captured by QEMU.

Task-number: QTBUG-65845
Change-Id: Iddf888fb3aa019209dbe71f424d87884471784e0
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2018-02-11 09:41:24 +00:00
Friedemann Kleint
b68dfa0348 QTestlib/selftests: Refactor the code printing output differences
Use qInfo() instead of qDebug() in case QDebug is turned off.

If the 'diff' tool is available, write the output to temporary files and
run diff on it. Otherwise, print the lines as was before, but onto one
stream to avoid indentation by the testlib handler.

Change-Id: Ib5a5dfb66ce481b493b85b915aa8c785ecb6b387
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2018-01-24 14:38:58 +00:00
Giuseppe D'Angelo
ff25a7ec05 QAbstractItemModelTester: fix the autotests
Move the modeltest autotest in the right place, and fix the
other autotests that were using it to use the version now in QtTestLib.

Change-Id: Ic6838945f616d580f357c872ce0956c341be3b16
Reviewed-by: David Faure <david.faure@kdab.com>
2018-01-09 21:12:58 +00:00
Friedemann Kleint
5af4cb5a5a Tests: Raise minimum supported MSVC version to 2015
Remove code for older versions and streamline #ifdefs.

Task-number: QTBUG-51673
Change-Id: If456567691538b1a1f452111814c5f9eba401c43
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2018-01-04 20:23:25 +00:00
Liang Qi
153e8b49ad Merge remote-tracking branch 'origin/5.10' into dev
Conflicts:
	src/network/access/qhttp2protocolhandler_p.h
	src/network/kernel/kernel.pri
	src/network/ssl/qsslkey_qt.cpp
	src/plugins/platforms/cocoa/qcocoascreen.mm
	src/plugins/platforms/windows/accessible/iaccessible2.cpp
	src/plugins/platforms/windows/accessible/iaccessible2.h
	src/plugins/platforms/windows/accessible/qwindowsaccessibility.cpp
	src/plugins/platforms/windows/accessible/qwindowsmsaaaccessible.cpp
	src/plugins/platforms/windows/accessible/qwindowsmsaaaccessible.h
	src/widgets/widgets/qmenu_p.h
	tests/auto/network/access/qnetworkreply/tst_qnetworkreply.cpp
	tests/auto/other/qaccessibility/tst_qaccessibility.cpp
	tests/auto/testlib/selftests/expected_cmptest.lightxml
	tests/auto/testlib/selftests/expected_cmptest.teamcity
	tests/auto/testlib/selftests/expected_cmptest.txt
	tests/auto/testlib/selftests/expected_cmptest.xml

Done-with: Edward Welbourne <edward.welbourne@qt.io>
Change-Id: I4217cc7d840cbae3e3dd28574741544469c4c6b9
2017-11-23 09:36:03 +01:00
Mitch Curtis
c988a8a282 Make QCOMPARE print QColor alpha values on failure
Currently, when two colors are equal except for their alpha values,
QCOMPARE produces the following failure message:

FAIL!  : tst_Test::test() Compared values are not the same
   Actual   (colorA): #ff0000
   Expected (colorB): #ff0000

By using the HexArgb format instead of the default HexRgb, we can
see the full hex string, with alpha values included:

FAIL!  : tst_Test::test() Compared values are not the same
   Actual   (colorA): #88ff0000
   Expected (colorB): #ffff0000

Task-number: QTBUG-55574
Change-Id: Id82c60a1b473ac6025a6f6ac560fce95a910d782
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2017-11-03 14:29:49 +00:00
Edward Welbourne
ffd822b968 Force QT_LOGGING_RULES in testlib's selftest
On some systems (e.g. Debian/buster) a standard Qt package may install
a qtlogging.ini file, e.g. in /etc/xdg/QtProject/, which suppresses
QDEBUG output.  A ~/.config/QtProject/qtlogging.ini could interfere
similarly.  This can break the selftest, when run with expected_*
files generated without such interference.  Likewise, if those
expected_* files are generated on a system such similar, they won't
work on CI.  Given that this caused confusion and delay with
integrating the latest set of updates to the expected_* files, it
seemed best to save others from similar bafflement.

So set a standard value for QT_LOGGING_RULES in both the selftest and
the generator script, so we get consistency.

Change-Id: I649e2f1f6ead21edf8af051aaee286e369fed064
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-10-11 13:31:43 +00:00
Edward Welbourne
98ef376fb6 generate_expected_output.py: update generated expected_* files
In selftests.qrc, imposed alphabetic order (on stem of name, then on
suffix, effectively treating . as sorting before any letter) while
removing old tests and adding new tests and data.  Updated all non-csv
files and added many missing files.  (Not clear on csv support status;
the script seems to have dropped it after 5.6, but the test still uses
it.)

Left expected_crashes* alone (no new files added, no update to old) as
I don't get results resembling those anticipated.

Omitted printdatatagswithglobaltags, printdatatags due to dangling
hspace on output lines, which upset sanity-bot.  A change to the test
cpp is needed to make it viable to skip that dangling hspace.

Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-10-11 09:59:44 +00:00
David Faure
23d0c77ce8 QTestLib: flush DeferredDelete events between every test
This makes qtestlib behave more like real Qt applications with an
event loop, where exiting the application does indeed flush deleteLaters.
And since every test method is supposed to be independent from other
test methods, we should even cleanup between tests.

For "app less" tests, no flushing happens.

Real life use cases:

* A unittest for some code (e.g. KIO job) which uses a socket, runs an
event loop until the socket communication is done, and ends. However
slotDisconnected() does, as recommended, socket->deleteLater(). So the
test finishes before the socket has a chance to actually get deleted,
and memory leak tools flag a memory leak, which doesn't actually happen
outside the unittest.

* Deleting a QWebEngineView with deleteLater is even worse than a memleak,
it leads to an assert (from a global object destructor) in QtWebEngine
if the view deletion hasn't actually been processed.

Change-Id: I18fc822fd26988a0fa4e75313c1947fcaa7d9e56
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2017-10-06 10:43:03 +00:00
Friedemann Kleint
eb93d7b517 Testlib: Format QVector2/3/4D in QCOMPARE
Change-Id: Ia0709e598d2fddd6c2f448633b484749146265f7
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2017-08-27 06:17:32 +00:00
Thiago Macieira
883dfb3d15 Merge remote-tracking branch 'origin/5.9' into dev
Conflicts:
	src/widgets/kernel/qwidget.cpp

This merge also extends the expected output of the pairdiagnostics
teamcity output (added in dev in commit
c608ffc56a) after the recent addition of
the flowId attribute to the teamcity output (commit
8f03656211 in 5.9).

Change-Id: I3868166e5efc45538544fffd14d8aba438f9173c
2017-08-08 11:48:10 +02:00
Borgar Ovsthus
8f03656211 Add flowId to messages when logging in TeamCity format
Added flowId='name' to each message when using TeamCity logging format.
This is necessary to distinguish separate processes running in parallel.

[ChangeLog][QtTest] Added flowId to messages when logging in TeamCity
format. FlowId is used to distinguish logging from multiple processes
running in parallel.

Change-Id: I7f5046c1058ff02770404caa2c9b3a5398f97f6b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2017-08-04 18:48:48 +00:00
Edward Welbourne
4cb46e27fd generate_expected_output.py: more refinements to filtering of noise
Two more ways line numbers were making it through.
Corrected a doc-string to tell nearer to the truth.

Change-Id: I946aaeb936d47fffe50d7ec15e2524992cc9e428
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-13 11:27:38 +00:00
Edward Welbourne
06f956b8f9 generate_expected_output.py: refer to tst_selftests.cpp for test list
Document that the saved output is used by tst_selftests.cpp and use a
crude parse of it to get the list of subdirs that it actually tests.

Change-Id: I73023228c9e547f965b7749dd66de7ef09c3815e
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2017-06-13 11:27:29 +00:00
Samuel Gaist
c608ffc56a Improve pair-like class handling in tests
Currently when doing comparison with pair-like classes the fallback
toString method is called which returns a Q_NULLPTR thus not allowing
proper diagnostic of the values that triggered an error. This patch
adds support for QPair and std::pair to improve the tests output
readability.

[ChangeLog][QtTest][QCOMPARE] Now outputs contents of QPair and
std::pair on failure.

Change-Id: Ib0b4aad7640590d56275b6f1306fb9fb39fa81bc
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2017-04-06 21:46:32 +00:00
Sami Nurmenniemi
a262837505 Fix tst_Selftest on QEMU
Two changes are needed to pass tst_Selftest on QEMU
1. Pass QEMU specific env variables to the subtests
2. Ignore output on stderr on some tests when running on QEMU

Change-Id: Ie1f722fd183aac5973e87d408005e06cbafcde17
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2017-03-31 07:09:46 +00:00
Ulf Hermann
44af54419e Properly use the "process" feature
Replace all QT_NO_PROCESS with QT_CONFIG(process), define it in
qconfig-bootstrapped.h, add QT_REQUIRE_CONFIG(process) to the qprocess
headers, exclude the sources from compilation when switched off, guard
header inclusions in places where compilation without QProcess seems
supported, drop some unused includes, and fix some tests that were
apparently designed to work with QT_NO_PROCESS but failed to.

Change-Id: Ieceea2504dea6fdf43b81c7c6b65c547b01b9714
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
2017-02-27 15:44:46 +00:00
Ulf Hermann
0979c5304c Drop unnecessary dependencies from some tests
The future tests don't need QtConcurrent as QFuture and friends are in
QtCore. The printdevice test doesn't use QtNetwork and the lancelot as
well as the testlib tests don't use QtXml.

Change-Id: I150ac99b36682aa23ad22ba943266eb0f0952838
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
2017-01-10 16:34:52 +00:00
Tor Arne Vestbø
aec85a53df Disable core dumps for selftests that are meant to be crashing
Task-number: QTBUG-55155
Change-Id: I26a1461f35f916f3980fcb18cdddf3502e22fc90
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
2016-12-08 23:50:44 +00:00
Liang Qi
7555a92581 Merge remote-tracking branch 'origin/5.7' into 5.8
5971b88e is not needed in new configure.

This merge also reverts "fix QMAKE_DEFAULT_*DIRS resolution with
apple SDK", 2c9d15d7, because it breaks iOS build with new
configure system.

Conflicts:
	mkspecs/features/default_pre.prf
	mkspecs/features/mac/toolchain.prf
	mkspecs/features/toolchain.prf
	src/dbus/qdbusconnection.cpp
	src/plugins/sqldrivers/mysql/qsql_mysql.cpp
	src/sql/drivers/mysql/qsql_mysql.cpp
	src/widgets/widgets/qmenubar.cpp
	src/widgets/widgets/qmenubar_p.h
	tools/configure/configureapp.cpp
	tools/configure/environment.cpp
	tools/configure/environment.h

Change-Id: I995533dd334211ebd25912db05b639d6f908aaec
2016-09-21 11:57:52 +02:00
Tony Sarajärvi
7a593c8c98 Blacklist selftest runSubTest
This one selftest is currently blocking OS X 10.11
from entering the CI. It can't be reproduced when
run manually.

Task-number: QTBUG-55155
Change-Id: I4553ef2d7813b29f5dc8577976c4482686346504
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@theqtcompany.com>
2016-09-12 12:59:48 +00:00
Liang Qi
d892e6f721 Merge remote-tracking branch 'origin/5.7' into 5.8
Conflicts:
	mkspecs/features/qt_common.prf
	src/plugins/platforms/windows/qwindowsdialoghelpers.cpp

Change-Id: I03b92b6b89ecc5a8db7c95f04ebb92ed198098a8
2016-09-05 18:47:27 +02:00
Liang Qi
657c2bfbec Merge remote-tracking branch 'origin/5.6' into 5.7
Conflicts:
	tests/auto/testlib/selftests/generate_expected_output.py

Change-Id: If856162abf9a24ae2c9946d336a7d1da03520fa7
2016-09-05 13:57:12 +02:00
Edward Welbourne
55ab276700 Major re-write of generate_expected_output.py
Restructured, separated the canonicalising of output lines out (into
an object that prepares the necessary regexes and replacements)
suppress more, changed the path-stripping to strip qtbase's parent
rather than os.getcwd() and took account of shadow builds (so both
source tree and build tree provide prefixes we want to strip from
paths).  Also cope with $PWD potentially having symlinks in it, where
os.getcwd() is canonical.

It's possible some output might name files elsewhere in the source
tree; these won't be filtered by the prior cwd prefix removal; and, in
any case, the problem with cwd is only that the ancestry of qtbase is
apt to vary; paths relative to there should be consistent between test
runs.  This change shall lead to a one-off rewrite of all expected_*
files; but it should now catch all paths.  By stripping both build
root and source root (when different) it also avoids differences for
those doing out-of-source ("shadow") builds.

In our XML formats, any hyphens in root paths (e.g. I had Qt-5.6 in my
build root's path) got represented by a character entity, confounding
the replacement; so also do replacement that catches this.  We may
discover other character entity subsitutions needed along with this.

Now filtering line numbers and timing information, including benchmark
results; these numbers all get replaced with 0 to avoid noisy diffs.
Also purging dangling hspace, to placate sanity-bot.

The module can now be imported - the code it runs is packaged as a
main() function that a __name__ == '__main__' stanza runs - and all
data is localised to where it's needed, rather than held in globals.
Tidied up and organized the existing regexes.  There are doc-strings;
there is a short usage comment.  Data is localised rather than global
and modern pythonic idioms get used where apt.

Regexes are compiled once instead of repeatedly.  An object looks
after the list of patterns to apply and its construction handles all
anticipated problems.  Failures are mediated by an exception.

The output file now gets written once, instead of twice (once before
editing, then over-write to edit), and Popen uses text mode, so that
write can do the same.  Its command is delivered as an array, avoiding
the need to invoke a shell.

Instead of relying on qmake being in our path (which might give us a
bogus QT_VERSION if the one in path doesn't match our build tree), use
the relative path to qmake - we rely on being run in a specific
directory in the build tree, after all.  Escape dots in the version
properly, so that 51730 doesn't get mistaken for 5.7.0 (for example),
and moved this check later in the sequence (matching a smaller target
makes it more likely to falsely match).

Overtly check we are in the right directory and tell the user what we
actually need, if run from the wrong place.  Simplify handling of the
unsupported use-case for MS-Windows (but note what would be needed for
it).

Change-Id: Ibdff8f8cae173f6c31492648148cc345ae29022b
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-08-29 18:02:35 +00:00
Edward Welbourne
782ebeada1 Merge remote-tracking branch 'origin/5.7' into dev
Conflicts:
	qmake/library/qmakebuiltins.cpp
	qmake/library/qmakeevaluator.cpp
	qmake/library/qmakeevaluator.h
	qmake/project.h
QMakeEvaluator:
* evaluateConditional(): one side changed return type, the other
  changed a parameter type.
* split_value_list(): one side changed a parameter adjacent to where ...
* expandVariableReferences(): ... the other killed one overload and
  changed the survivor

	src/corelib/io/qlockfile_unix.cpp
One side changed a #if condition, the other moved NETBSD's part of
what it controlled.

	src/corelib/tools/qdatetime.cpp
One side fixed a reachable Q_UNREACHABLE in toMSecsSinceEpoch(), the
other moved it from the private class to the public one, in the midst
of the "short date-time" optimization, which confused diff entirely.
One side changed a QStringLiteral to QLatin1String, the other rewrote
adjoining code.

	src/network/kernel/qauthenticator.cpp
Both rewrote a line, equivalently; kept the dev version.

	src/platformsupport/fontdatabases/mac/qfontengine_coretext.mm
	src/platformsupport/fontdatabases/mac/qfontengine_coretext_p.h
One side changed #if-ery that the other removed.

	tools/configure/configureapp.cpp
One side added a check to -target parsing; the other killed -target.

	tests/auto/testlib/selftests/expected_cmptest.lightxml
	tests/auto/testlib/selftests/expected_cmptest.teamcity
	tests/auto/testlib/selftests/expected_cmptest.txt
	tests/auto/testlib/selftests/expected_cmptest.xml
	tests/auto/testlib/selftests/expected_cmptest.xunitxml
Regenerated using generate_expected_output.py
I note that quite a few other expected_* come out changed, now.

There was no git-conflict in
	src/widgets/kernel/qformlayout.cpp
but it didn't compile; one side removed some unused methods; the other
found uses for one of them.  Put FixedColumnMatrix<>::removeRow(int)
back for its new user.

Change-Id: I8cc2a71add48c0a848e13cfc47b5a7754e8ca584
2016-07-19 20:14:40 +02:00
Edward Welbourne
82ea53ad24 Merge remote-tracking branch 'origin/5.6' into 5.7
Conflicts:
	qmake/library/qmakeevaluator.cpp

One side changed the iterator to use ranged-for, the other changed its
body; they only conflicted because the latter had to add braces around
the body, intruding on the for-line.  Trivial resolution.

Change-Id: Ib487bc3bd6e3c5225db15f94b9a8f6caaa33456b
2016-07-15 20:47:57 +02:00
Friedemann Kleint
e7d4760314 Testlib: Format QRegion in QCOMPARE
Output regions as a compact list of rectangles:
QRegion(2 rectangles, 50x200+100+200, 200x50+10+10)

Change-Id: I62e42bd30897a1d31b28e0cbd6afe7fa8ecf2664
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2016-07-05 18:49:19 +00:00
Friedemann Kleint
a9613ba3f1 Testlib: generate_expected_output.py: Run in C locale
Otherwise, float numbers are formatted using a decimal comma
in German, causing huge diffs and failures.

Change-Id: Icd85a293d0564cac6be244eb0793611920d0c89c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2016-07-05 15:00:58 +00:00
Liang Qi
56d6e000f7 Merge remote-tracking branch 'origin/5.7' into dev
Conflicts:
	mkspecs/wince80colibri-armv7-msvc2012/qmake.conf
	qmake/generators/win32/msvc_vcproj.cpp
	src/corelib/global/qnamespace.h
	src/corelib/global/qnamespace.qdoc
	src/corelib/io/qfsfileengine_win.cpp
	src/corelib/tools/tools.pri
	src/network/ssl/qsslconfiguration_p.h
	src/plugins/platforms/linuxfb/qlinuxfbscreen.cpp
	src/plugins/platforms/windows/windows.pri
	src/src.pro
	src/tools/bootstrap/bootstrap.pro
	src/tools/uic/cpp/cppwriteinitialization.cpp
	src/widgets/dialogs/qfilesystemmodel.cpp
	tests/auto/testlib/selftests/expected_cmptest.teamcity
	tests/auto/testlib/selftests/expected_cmptest.txt

Change-Id: I4d2ac78f0dcc97f008186bbbc769c6fe588ab0e5
2016-05-23 21:09:46 +02:00
Liang Qi
818014b449 Merge remote-tracking branch 'origin/5.6' into 5.7
Conflicts:
	configure
	src/3rdparty/angle/src/libANGLE/renderer/d3d/d3d11/Renderer11.cpp
	src/network/access/qnetworkaccessmanager.cpp
	src/plugins/platforms/cocoa/qcocoacolordialoghelper.mm
	src/plugins/platforms/eglfs/deviceintegration/eglfs_kms/qeglfskmsgbmcursor.cpp
	src/plugins/platforms/eglfs/deviceintegration/eglfs_kms/qeglfskmsgbmcursor.h
	src/widgets/widgets/qlineedit_p.cpp
	src/widgets/widgets/qlineedit_p.h
	src/winmain/winmain.pro
	tests/auto/corelib/io/qstorageinfo/tst_qstorageinfo.cpp
	tests/auto/dbus/qdbusconnection/tst_qdbusconnection.cpp
	tests/auto/dbus/qdbusconnection/tst_qdbusconnection.h
	tests/auto/testlib/selftests/expected_cmptest.teamcity
	tests/auto/testlib/selftests/expected_cmptest.txt
	tests/auto/widgets/itemviews/qtableview/tst_qtableview.cpp
	tools/configure/configureapp.cpp

Change-Id: Ib9997b0d0f91946e4081d36c0c6b696c5c983b2a
2016-05-19 12:55:27 +02:00
Thiago Macieira
194a56ea79 QTest: Fix printing of values of Q_ENUMs
QMetaEnum::key(n) returns the n-th key, which is not necessarily the one
of value n. For the key of value n, we want QMetaEnum::valueToKey(n).

Change-Id: Ic90fe6b1cbe84978a02fffff141bf4a06074917a
Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com>
2016-05-08 08:57:16 +00:00
Giuseppe D'Angelo
3489ab60a9 QTestLib: support explicit bool conversions in QVERIFY
[ChangeLog][QTestLib] It is now possible to use variables of
types with an explicit operator bool in the QVERIFY macro.

Change-Id: I2685df164a616f6a065d689867daa9ea1de78e08
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2016-05-04 11:29:08 +00:00
Marc Mutz
1ed8a7bff5 QTest: support nullptr in QCOMPARE
This allows to write
  QCOMPARE(ptr, nullptr);
instead of
  QVERIFY(ptr);

Task-number: QTBUG-49973
Change-Id: I6e1327d4327bcf17bd9b59de4352fdcaae98ac27
Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com>
2016-04-08 05:38:33 +00:00
Friedemann Kleint
8104729e63 QtTest: Remove Windows CE.
Remove #ifdef sections for Q_OS_WINCE and wince .pro file clauses in library
and tests.

Task-number: QTBUG-51673
Change-Id: I552b3fe8d6e0eb7c8c7b3a3d41558e5e21904dd2
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
2016-04-06 10:56:02 +00:00
Friedemann Kleint
eda675b688 QtTest/generate_expected_output.py: Add TeamCity logging format.
Adapt the script to generate output for the new TeamCity format
added by fbd6acedac.

Change-Id: I9435382ec3daf80428c324c58434aa951841bf08
Reviewed-by: Borgar Øvsthus <borgar.ovsthus@fmcti.com>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@theqtcompany.com>
2016-03-17 17:29:14 +00:00
Friedemann Kleint
74c202f913 QtTest: Add formatting for QColor.
When a QCOMPARE of values of type QColor fails, their
name will now be printed.

Task-number: QTBUG-51124
Change-Id: I76565daa338f038ea4f452e47705e638d94eaeee
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@theqtcompany.com>
2016-03-17 15:35:05 +00:00
Christian Strømme
d392826959 Make it possible to preserve the library/plugin path in tst_selftest
If QT_PRESERVE_TESTLIB_PATH is set, then LD_LIBRARY_PATH and
QT_PLUGIN_PATH won't be filtered out for the sub-tests started by
tst_selftest.

Change-Id: Ic43ba9b4d882ee36b2f7495b1c880f26aefd2629
Reviewed-by: aavit <eirik.aavitsland@theqtcompany.com>
2016-02-23 21:16:44 +00:00
Borgar Ovsthus
fbd6acedac Add TeamCity logging feature to testlib
This allows TeamCity to parse realtime test-results instead of using
post-build XML Report Processing. This does not support logging of
benchmarks.

[ChangeLog][QTest] Added a new logging mode that allow test-results to
be parsed on-the-fly when using Jetbrains TeamCity as CI-server. This
mode is enabled by using the -teamcity option on the command-line.

Change-Id: Ie730beb643043eb0f448f99abe6c0b5ac48aaf03
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2016-01-23 18:30:58 +00:00
Thiago Macieira
cfe891eeb8 Add Intel copyright to files that Intel has had non-trivial contribution
I wrote a script to help find the files, but I reviewed the
contributions manually to be sure I wasn't claiming copyright for search
& replace, adding Q_DECL_NOTHROW or adding "We mean it" headers.

Change-Id: I7a9e11d7b64a4cc78e24ffff142b506368fc8842
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
2016-01-21 22:44:21 +00:00
Jani Heikkinen
f776595cc1 Updated license headers
From Qt 5.7 -> tools & applications are lisenced under GPL v3 with some
exceptions, see
http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/

Updated license headers to use new GPL-EXCEPT header instead of LGPL21 one
(in those files which will be under GPL 3 with exceptions)

Change-Id: I42a473ddc97101492a60b9287d90979d9eb35ae1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
2016-01-21 18:55:18 +00:00
Liang Qi
beb65dcd79 Merge remote-tracking branch 'origin/5.6' into dev
Conflicts:
	src/gui/painting/painting.pri
	src/plugins/platforms/xcb/qxcbconnection.cpp
	tests/auto/corelib/thread/qthreadstorage/qthreadstorage.pro
	tests/auto/corelib/tools/qlocale/test/test.pro
	tests/auto/gui/kernel/qwindow/tst_qwindow.cpp
	tools/configure/environment.cpp

Change-Id: I9c40f458b89b2c206de2d2c24e90b5f679c93495
2015-12-18 08:37:31 +01:00
Maurice Kalinowski
acdd57cb43 Fix compilation for WinRT
TEST_HELPER_INSTALLS cannot be used on platforms with no
QProcess support.

Change-Id: I2a6a283d94ca4487fc628449c53fc37140dd291d
Reviewed-by: Oliver Wolff <oliver.wolff@theqtcompany.com>
2015-12-14 08:43:15 +00:00
Robin Burchell
df757a2e62 generate_expected_output: Open the file as binary.
Otherwise write() for my python (OS X 2.7) wants strings, which is incorrect.

Change-Id: Ibd9d050646d1039ba8370d121dd25756ceffdb7a
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@theqtcompany.com>
2015-11-06 11:40:24 +00:00
Robin Burchell
9626baaea9 Fix QCOMPARE with enum classes.
As these are strongly typed, they won't implicitly convert to int, so make sure
to cast explicitly.

Change-Id: Ic8daa31c528bbd8f399ab401d0963e13db191312
Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com>
2015-11-06 11:40:15 +00:00
Friedemann Kleint
84042523f2 Testlib: Output total test time in plain text logger.
This makes it easier to spot slow tests.

Produces:

Totals: 314 passed, 0 failed, 0 skipped, 0 blacklisted, 137ms
********* Finished testing of tst_QStringRef *********

Task-number: QTBUG-38890
Change-Id: Iaee4dd677fd8f18b82f7c9c4934eb5b1119ade43
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@theqtcompany.com>
2015-10-30 19:19:10 +00:00
Liang Qi
4456984da7 Merge remote-tracking branch 'origin/5.6' into dev
Conflicts:
	tests/auto/corelib/io/qfile/tst_qfile.cpp
	tests/auto/corelib/io/qprocess/tst_qprocess.cpp
	tests/auto/corelib/tools/qversionnumber/qversionnumber.pro

Change-Id: Ia93ce500349d96a2fbf0b4a37b73f088cc505c6e
2015-10-14 15:45:35 +02:00
Friedemann Kleint
9b6cd2764a Testlib: Output function / total time along with crash dump.
Previously, only QXmlTestLogger had timers to take elapsed times
and log them. Move those into class QTestLog for access by
the loggers and output the times in the crash dump to make it
easier to spot hangs/recursion crashes.

Produces:

QFATAL : foo() Received signal 11
         Function time: 22ms Total time: 23ms

A crash occurred in ...
Function time: 24ms Total time: 26ms

Task-number: QTBUG-47370
Change-Id: Ia530a63104087daffc9a15f68c15d93378b9407e
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
2015-10-13 16:38:43 +00:00
Friedemann Kleint
9facf1be9b Tests: Always verify whether QTemporaryDir/File creation succeeded.
Use QVERIFY2() with QTemporaryDir/File::errorString() consistently.
Attempt to catch issues like the below warning and follow-up issues.

QSYSTEM: tst_QFiledialog::clearLineEdit() QFileSystemWatcher: FindNextChangeNotification failed for "C:\Users\qt\_____aaaaaaaaaaaaaaaaaaaaaa"  (Access is denied.)

Task-number: QTBUG-47370
Change-Id: I58a6e87c502627e976efa62ad73c912f3b2d49fa
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@theqtcompany.com>
2015-09-28 15:05:43 +00:00
Liang Qi
a1ad9a74eb Merge remote-tracking branch 'origin/5.6' into dev
Conflicts:
	src/corelib/io/io.pri
	src/corelib/io/qdatastream.cpp
	src/corelib/io/qdatastream.h
	src/network/socket/qabstractsocket.cpp
	src/plugins/platforminputcontexts/ibus/qibusplatforminputcontext.cpp
	src/plugins/platforms/cocoa/qcocoaaccessibilityelement.h
	src/widgets/styles/qgtkstyle.cpp
	tests/auto/corelib/mimetypes/qmimedatabase/qmimedatabase-cache/qmimedatabase-cache.pro
	tests/auto/corelib/mimetypes/qmimedatabase/qmimedatabase-xml/qmimedatabase-xml.pro
	tests/auto/dbus/qdbusconnection/qdbusconnection.pro
	tests/auto/dbus/qdbuspendingcall/tst_qdbuspendingcall.cpp
	tests/auto/network/access/qnetworkreply/tst_qnetworkreply.cpp

Change-Id: I347549a024eb5bfa986699e0a11f96cc55c797a7
2015-09-25 14:02:04 +02:00
Friedemann Kleint
76cf88157f Tests: Remove CONFIG += parallel_test.
The keyword no longer has a meaning for the new CI.

Change-Id: Ibcea4c7a82fb7f982cf4569fdff19f82066543d1
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
2015-09-05 07:16:50 +00:00
Friedemann Kleint
c486cacba8 Testlib/Windows: Output crash information on stdout instead of stderr.
Make it easier to capture information about crashing tests in log files,
since it is not possible to do something like
foo > log.txt 2> errlog.txt
on Windows.
Adapt selftest.

[ChangeLog][QtTest][Important Behavior Changes] Crash/exception
information is now logged to standard output on Windows.

Task-number: QTBUG-47370
Change-Id: I3e6a2d25855ed0f20516418a031990b562f5f757
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
2015-09-03 14:16:30 +00:00
Frederik Gladhorn
77da617dc8 Merge remote-tracking branch 'origin/5.5' into dev
Conflicts:
	doc/global/qt-cpp-defines.qdocconf
	src/3rdparty/forkfd/forkfd.c
	src/corelib/codecs/qtextcodec.cpp
	src/corelib/kernel/qmetatype.cpp
	src/corelib/tools/qset.qdoc
	src/gui/accessible/qaccessible.cpp
	src/gui/image/qpixmapcache.cpp
	src/opengl/qgl.cpp
	src/tools/qdoc/generator.cpp
	src/widgets/kernel/qwidget.cpp
	tests/auto/widgets/widgets/qcombobox/tst_qcombobox.cpp

Change-Id: I4fbe1fa756a54c6843aa75f4ef70a1069ba7b085
2015-08-06 10:54:01 +02:00
Thiago Macieira
644ac04af0 Change how QDebug escapes QStrings in the output
[ChangeLog][Important Behavior Changes] QDebug output for QStrings
changed compared to Qt 5.5.0 to more closely match the output of
previous Qt versions. Like Qt 5.5.0, QDebug will escape non-printable
characters, the backslash and quote characters, but will no longer
escape the printable characters.

Task-number: QTBUG-47316
Change-Id: I52dd43c12685407bb9a6ffff13f62ef68cbc80c5
Reviewed-by: Kai Koehne <kai.koehne@theqtcompany.com>
2015-08-06 04:53:38 +00:00
Friedemann Kleint
5f5db1c38b QTestLib: Add macros QTRY_VERIFY2_WITH_TIMEOUT(), QTRY_VERIFY2().
Add QTRY_VERIFY2_WITH_TIMEOUT() similar to QTRY_VERIFY_WITH_TIMEOUT()
except that QTRY_VERIFY2() is used below the loop and QTRY_VERIFY2()
based on it.

Add tests to cmptest.

[ChangeLog][QtTest] Added macros QTRY_VERIFY2_WITH_TIMEOUT(), QTRY_VERIFY2()
making it possible to output a message after the timeout has expired.

Change-Id: I587e24f3aeb73542dbf3ccb936a16f2e0806555f
Reviewed-by: Mitch Curtis <mitch.curtis@theqtcompany.com>
2015-07-22 19:32:39 +00:00
Friedemann Kleint
4b8ba3c91a Fix sub test float of tests/auto/testlib/selftests/tst_selftests for MSVC 2015.
From MSVC 2015 onwards, MSVC formats floats using printf("%g") like
g++, so the fiddling of the expected output needs to restricted
accordingly.

Change-Id: I6e7f78e5e90f70886a8b2ef37c0fb9bf82b5e1a3
Reviewed-by: Kai Koehne <kai.koehne@theqtcompany.com>
2015-07-17 14:15:19 +00:00
Friedemann Kleint
d0f64542c8 Output registered enumeration names in QCOMPARE.
Produces output:

FAIL!  : tst_qquickwindow::cursor() Compared values are not the same
   Actual   (clippedItem.cursor().shape()): ForbiddenCursor
   Expected (Qt::ArrowCursor)             : ArrowCursor
   Loc: [tst_qquickwindow.cpp(1465)]

Change-Id: I8a9dfa099a6011cbe0c07da683d4be3d07e22d5d
Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@theqtcompany.com>
2015-06-12 07:48:22 +00:00
Simon Hausmann
e2f66f9215 Merge remote-tracking branch 'origin/5.5' into dev
Conflicts:
	src/corelib/global/qnamespace.qdoc
	src/corelib/io/qwindowspipereader.cpp
	src/corelib/io/qwindowspipereader_p.h
	src/corelib/statemachine/qstatemachine.cpp
	src/corelib/statemachine/qstatemachine_p.h
	src/plugins/platforms/xcb/qxcbconnection.h
	tests/auto/network/access/qnetworkreply/tst_qnetworkreply.cpp
	tests/auto/tools/qmake/tst_qmake.cpp
	tests/manual/touch/main.cpp

Change-Id: I917d694890e79ee3da7d65134b5b085e23e0dd62
2015-06-03 10:23:56 +02:00
Caroline Chao
96527f74e2 TestCase: Also check main source path when looking for test data
This is for example useful when looking for a possible BLACKLIST
file while doing a shadow build.

Add autotests for blacklist.

Change-Id: I41d3939d31d21d10187fefcb82604736d911b6ad
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@theqtcompany.com>
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
2015-05-05 07:06:52 +00:00
Lars Knoll
d3d10cf23d Improve error handling in qtestlib
Dump a stack trace on Linux if the process crashes. This will give us
much better log output in the new CI system. In addition, create a
watch dog thread, that kills the test if the test function times
out (ie. hangs)

Implementations of the stack trace dumping for Mac and Windows
are still pending.

Change-Id: I65426c5afe290a0a2019b881436a0c278f1cafaf
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
2015-04-21 16:19:09 +00:00
Thiago Macieira
f5b609cac8 Make the printing of complex byte arrays prettier
Similar to what we've done to QString, only we print each byte that is
not ASCII as \OOO (octal representation).

[ChangeLog][QtTest] QtTest now prints an escaped version of
QByteArrays that failed to compare with QCOMPARE, instead of the hex
dump.

Change-Id: I6a8c43f138c66c998280998a242b43cd579666a0
Reviewed-by: Richard J. Moore <rich@kde.org>
2015-02-15 06:35:43 +00:00
Jani Heikkinen
83a5694dc2 Update copyright headers
Qt copyrights are now in The Qt Company, so we could update the source
code headers accordingly. In the same go we should also fix the links to
point to qt.io.

Outdated header.LGPL removed (use header.LGPL21 instead)

Old header.LGPL3 renamed to header.LGPL3-COMM to match actual licensing
combination. New header.LGPL-COMM taken in the use file which were
using old header.LGPL3 (src/plugins/platforms/android/extract.cpp)

Added new header.LGPL3 containing Commercial + LGPLv3 + GPLv2 license
combination

Change-Id: I6f49b819a8a20cc4f88b794a8f6726d975e8ffbe
Reviewed-by: Matti Paaso <matti.paaso@theqtcompany.com>
2015-02-11 06:49:51 +00:00
Frederik Gladhorn
b6191b16d4 Merge remote-tracking branch 'origin/5.4' into dev
Conflicts:
	src/corelib/global/global.pri
	src/corelib/global/qcompilerdetection.h
	src/corelib/global/qglobal.h
	src/corelib/tools/qdatetime.cpp
	src/plugins/platforms/xcb/qxcbscreen.h
	src/plugins/platforms/xcb/qxcbwindow.h
	src/widgets/dialogs/qcolordialog.cpp
	src/widgets/dialogs/qcolordialog_p.h
	tools/configure/configureapp.cpp

Change-Id: Ie9d6e9df13e570da0a90a67745a0d05f46c532af
2015-01-21 11:10:14 +01:00
Rainer Keller
20480070a6 Autotest: Selftests fail in UTC timezone
Selftests for testlib fail when executed in UTC timezone because local and
UTC are the same, but expected to be different. A custom timezone is used
instead.
Debug output of qCompare does only handle local and non-local timezones, using
new Qt5 features allows to show the correct timezone in format string.

Change-Id: I753884a12370952b7b62a90d62896db4f2d3d1b4
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2015-01-19 10:21:26 +01:00
Thiago Macieira
2c01d402e1 QDebug: pretty-print QStrings and QStringRefs
[ChangeLog][QtCore][QDebug] Printing of QStrings and QStringRefs
whenever "noquote" is not active now prints the strings in a format that
can be copied back to C++ code. All characters that aren't printable in
US-ASCII are escaped (this includes printable Unicode characters outside
of US-ASCII). Pretty-printing will not respect QTextFormat padding or
field widths.

Change-Id: I169a8a0508e24693f5652f0129defe7f709e5d08
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@theqtcompany.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2015-01-11 03:40:38 +01:00
Kai Koehne
ef6279fd51 Add QtInfoMsg
Add an 'info' message type that can be used for messages that are neither
warnings (QtWarningMsg), nor for debugging only (QtDebugMsg). This is
useful mainly for applications that do not have to adhere to the
'do not print anything by default' paradigm that we have for
the Qt libraries itself.

[ChangeLog][QtCore][Logging] QtInfoMsg got added as a new QtMsgType.
  Use the new qInfo(), qCInfo() macros to log to it.

Change-Id: I810995d63de46c41a9a99a34d37c0d417fa87a05
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2015-01-09 13:14:05 +01:00
Matti Paaso
974c210835 Update license headers and add new license files
- Renamed LICENSE.LGPL to LICENSE.LGPLv21
- Added LICENSE.LGPLv3
- Removed LICENSE.GPL

Change-Id: Iec3406e3eb3f133be549092015cefe33d259a3f2
Reviewed-by: Iikka Eklund <iikka.eklund@digia.com>
2014-09-24 12:26:19 +02:00
Lars Knoll
10414444e1 Add support for blacklisting test functions
We need to have a finer grained control over the tests
we skip in our CI system. This adds a blacklisting
mechanism that allows blacklisting individual test
functions (or even test data) using a set of predefined
matching keys for the operating system and some other
relevant variables.

QTestlib will search for a file called BLACKLIST in the test
directory and parse it if found. The file contains a simple
ini style list of functions to blacklist. For details see
qtestblacklist.cpp.

Change-Id: Id3fae4b264ca99970cbf9f45bfb85fa75c1fd823
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2014-09-21 20:58:41 +02:00
Simon Hausmann
508b95899d Merge remote-tracking branch 'origin/stable' into dev
Change-Id: Ia36e93771066d8abcf8123dbe2362c5c9d9260fc
2014-05-22 07:46:17 +02:00
Bernd Weimer
fa3d264b0b QNX: Fix tst_selftest
GRAPHICS_ROOT and TZ environment variables are needed in child
processes in order to successfully run the auto test selftests.

Change-Id: I7befabd535b4c47b1e75acbe3d6158d0d9b811b3
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
2014-05-16 10:20:34 +02:00
Keith Gardner
6fc7d76e73 QSignalSpy: Added template function pointer constructor
Added the function pointer constructor to QSignalSpy to take advantage
of the new connect syntax.

Change-Id: I94218a096c677cdba73e1b2cfa8b9c09bc28145f
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2014-04-23 22:26:00 +02:00
Friedemann Kleint
9e4a215a25 Use new QLibraryInfo::build() in testlib to log build information.
This produces:

********* Start testing of tst_QtJson *********
Config: Using QtTest library 5.3.0, Qt 5.3.0 (Feb 13 2014, GCC 4.6.3, 64 bit, debug build)
PASS : tst_QtJson::initTestCase()

<?xml version="1.0" encoding="UTF-8"?>
<TestCase name="tst_QtJson">
<Environment>
<QtVersion>5.3.0</QtVersion>
<QtBuild>Qt 5.3.0 (Feb 13 2014, GCC 4.6.3, 64 bit, debug build)</QtBuild>
<QTestVersion>5.3.0</QTestVersion>
</Environment>

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="1" failures="1" tests="42" name="tst_QtJson">
<properties>
<property value="5.3.0" name="QTestVersion"/>
<property value="5.3.0" name="QtVersion"/>
<property value="Qt 5.3.0 (Feb 13 2014, GCC 4.6.3, 64 bit, debug build)" name="QtBuild"/>
</properties>

<Environment>
<QtVersion>5.3.0</QtVersion>
<QtBuild>Qt 5.3.0 (Feb 13 2014, GCC 4.6.3, 64 bit, debug build)</QtBuild>
<QTestVersion>5.3.0</QTestVersion>
</Environment>

[ChangeLog][QtTest] Tests now output build information.

Change-Id: I0ab473371575f2b807db725256805b8bffea3454
Reviewed-by: Sergio Ahumada <sahumada@blackberry.com>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@digia.com>
2014-02-27 13:20:19 +01:00
Friedemann Kleint
356838cd7d Improve error reporting in selftests of testlib.
Output the file name and fix line number to be 1..n.

Change-Id: I3ba7f84b99ce3664bc3e403b230307511b15fe1f
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@digia.com>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@digia.com>
2014-02-25 20:56:32 +01:00
Thiago Macieira
579526cfec Make the printing of complex Unicode in a QString prettier
This also has the advantage of not requiring the use of the locale
codec. Quite an advantage if you're debugging the locale codec. But it's
mostly so that we don't get question marks that hide the difference we
were trying to locate.

[ChangeLog][QtTest] QtTest now prints an escaped version of QStrings
that failed to compare with QCOMPARE. That is, instead of converting
non-printable characters to question marks, QtTest will print the
Unicode representation of the character in question.

Change-Id: I44c1ef3246b188c913dacd3ca4df02581356ea41
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
2014-02-16 06:54:18 +01:00
Thiago Macieira
8dcb72fd1e Add a CSV logging feature to the benchlib
This is only useful for logging benchmarks, since it won't print test
passes, failures, etc. It's useful for importing to spreadsheets to do
number-crunching.

[ChangeLog][QtTest]Added a CSV logging mode that is suitable for
importing benchmark results into spreadsheets. This can be enabled by
the -csv option on the command-line. The CSV logging mode will not print
test failures, debug messages, warnings, etc.

Change-Id: I245d6f86bb380645c9bc0d748cf474b3ed42cab8
Reviewed-by: Sergio Ahumada <sahumada@blackberry.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2014-02-14 00:34:25 +01:00
Friedemann Kleint
afe3902a30 Testlib: Use QString for messages in QAbstractTestLogger::addMessage()
Task-number: QTBUG-35743

[ChangeLog][QtTest][Windows] Use correct UTF-8 encoding for XML test results on platforms with different console encoding.
Change-Id: Ice9d03192098f931e5dac358928e0c4421ab715e
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@digia.com>
2014-01-21 06:28:23 +01:00
Friedemann Kleint
d5912b2a47 Testlib/generate_expected_output.py: Fix encoding.
Do not apply additional encoding when reading process output.
Fixes errors when encountering UTF-8:

Traceback (most recent call last):
  File "./generate_expected_output.py", line 117, in <module>
    generateTestData(path)
  File "./generate_expected_output.py", line 106, in generateTestData
    out.write(data.decode('utf-8'))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xdc' in position 5485: ordinal not in range(128)

Change-Id: Ib827787a59a18b4d3d0601645856517f43c01fc3
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@digia.com>
2014-01-18 09:20:24 +01:00
Friedemann Kleint
f504287e80 Testlib/generate_expected_output.py: Pass tests as command line arguments.
Evaluate command line arguments and use directories only when empty.

Change-Id: I818ec13c686018a3f607e91174e57d8f8bbf15f7
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@digia.com>
2014-01-18 09:19:38 +01:00
Frederik Gladhorn
a69525243f Update test result generator script to set duration to 0
The actual code in testlib now indents the duration tag when it is for a
function.

Change-Id: Iee62db9c81f11dc54e57f166bf9fb2b7012b7e03
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
2014-01-18 04:39:17 +01:00
Frederik Gladhorn
019588f307 Testlib: Add test duration to xml output
[ChangeLog][QtTest] Added test duration to xml output.
When running tests with xml output a new tag of the form
<duration msecs="123"/> is added to each test function and the test as a
whole.

Change-Id: Ibc4db066b6acf5fac6c578f5e5ca5ce4b5d8ea8e
Reviewed-by: Sergio Ahumada <sahumada@blackberry.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2014-01-13 07:13:17 +01:00
Frederik Gladhorn
8f80ff338e Make output when failing a test for diff in line numbers more verbose
Change-Id: I536ffddffe40f9f6bbcd75f312a5f902763e1131
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2014-01-07 19:21:45 +01:00
Frederik Gladhorn
15ff120b02 Update testlib selftest units
The units that were supposed to have variance were completely off in the
test leading to all benchmarks with slight variance being completely
flaky.

Change-Id: Ib76593813974adee462b3f03be9370d2248d770c
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@digia.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2014-01-07 19:21:34 +01:00
Frederik Gladhorn
a34f37cc71 Add a way to reproduce testlib selftest data
This makes testlib a lot more hackable since when the output is changed
in a defined way we can regenerate the expected output (eg adding the
test duration which will be a follow-up patch).

Note that the script does not work properly on Windows and not all
benchmarks work, but at least it reduces the work of adapting files to a few lines.

Change-Id: I910387cd92ff82aa629747a3a3033dae17fbd711
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
2014-01-06 12:51:16 +01:00
Frederik Gladhorn
4d5906989a Update tests to normalizes the paths by using script.
The newly added generate_expected_output.py was used to get the expected
output into a more reproducible state.

Change-Id: I1ca75c8e0c5778d25c1df531bd298007aac0ff4a
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
2014-01-06 12:50:47 +01:00
Kai Koehne
326e9c8962 Remove trailing space from QDebug stream
It's unexpected that all messages generated by the stream version
of qDebug and friends have a trailing space. It also makes switching
to categorized logging (which only supports the stream version) difficult,
since all autotests checking for debug output would have to be adapted.

Task-number: QTBUG-15256
Change-Id: I8d627a8379dc273d9689f5611184f03607b73823
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2013-12-06 13:33:10 +01:00
Kai Koehne
5da094c1aa TestLib: Ignore trailing space in QTest::ignoreMessage
When comparing expected with actual debug messages, allow the expected
ones to contain a trailing space. This is needed to not break all
autotests in a follow up patch, which will prevent ~QDebug() from
generating a trailing space by default.

Task-number: QTBUG-15256
Change-Id: I7f67393ddfbfe37fde1ca5ef45e4ad7f4b5324b4
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@digia.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2013-12-04 13:10:49 +01:00
Frederik Gladhorn
4a8273a6fc Merge remote-tracking branch 'origin/stable' into dev
For the conflicts in msvc_nmake.cpp the ifdefs are extended since we
need to support windows phone in the target branch while it is not there
in the current stable branch (as of Qt 5.2).

Conflicts:
	configure
	qmake/generators/win32/msvc_nmake.cpp
	src/3rdparty/angle/src/libEGL/Surface.cpp
	src/angle/src/common/common.pri
	src/corelib/global/qglobal.h
	src/corelib/io/qstandardpaths.cpp
	src/plugins/platforms/qnx/qqnxintegration.cpp
	src/plugins/platforms/qnx/qqnxscreeneventhandler.h
	src/plugins/platforms/xcb/qglxintegration.h
	src/widgets/kernel/win.pri
	tests/auto/corelib/thread/qreadwritelock/tst_qreadwritelock.cpp
	tests/auto/corelib/tools/qdatetime/tst_qdatetime.cpp
	tests/auto/gui/text/qtextdocument/tst_qtextdocument.cpp
	tools/configure/configureapp.cpp

Change-Id: I00b579eefebaf61d26ab9b00046d2b5bd5958812
2013-11-26 22:35:48 +01:00
Dmitry Ashkadov
4a9092eaad Add QVERIFY_EXCEPTION_THROWN macro for testing exceptions using QtTest
New macro QVERIFY_EXCEPTION_THROWN may be used to check that some code
really throws an exception of specified type.

Change-Id: I7cf499c7c37a35407862bc604c6eb862c5f329d3
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2013-11-26 07:37:06 +01:00
Friedemann Kleint
1b9be3f481 Testlib.selftest: Improve process execution, run on openSUSE.
Give processes a more complete environment (openSUSE
requires more variables to start an X11-process),
add test column indicating whether a test is
supposed to crash. If it is not, report crash reason.

Change-Id: I1e0ad59824963f69ee425f331d845741be2b3928
Reviewed-by: Tony Sarajärvi <tony.sarajarvi@digia.com>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@digia.com>
2013-11-19 22:47:43 +01:00
Friedemann Kleint
943ae8bb70 Add overload of QTest::ignoreMessage() taking a QRegularExpression.
Make it possible to match messages by a pattern.

Change-Id: I713312e86db5471755459f1ecc43e8f1ac7a95fb
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2013-11-17 19:10:22 +01:00
Frederik Gladhorn
840f6a40e6 Merge remote-tracking branch 'origin/stable' into dev
Change-Id: Ie56539b2e0be611a363b5f15ae5412a78d6945a2
2013-10-24 12:48:42 +02:00
Oswald Buddenhagen
9c5e63ceb3 support building with -no-gui
Change-Id: I1c418e2aa6c01dbc40156a3494a7f7ddc95beca3
Reviewed-by: Sergio Ahumada <sergio.ahumada@digia.com>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
2013-10-11 21:03:30 +02:00
Andrew Knight
c6af0a504f WinRT: Fix various test compilations
- Remove irrelevant test subdirs via .pro files
- Follow WinCE codepaths where applicable
- Replace unsupported Win32 APIs with WinRT equivalents

This does not aim to fix any failures in the tests themselves; it only
makes them compile.

Change-Id: Ia82bc0cc402891f8f6238d4c261ee9152b51be80
Reviewed-by: Maurice Kalinowski <maurice.kalinowski@digia.com>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
2013-10-02 12:36:05 +02:00
Andrew Knight
cd582f8410 Add QT_NO_PROCESS guards in tests where they are missing
A handful of tests lack QT_NO_PROCESS guards, making them non-compilable
on Qt builds with no QProcess support. This commit does not change
QProcess-specific tests, which should be left out of the build using the
.pro file mechanism.

Change-Id: Iac8562428abc1f59ccbb23bf5c3a919785e41f12
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
2013-09-03 08:42:24 +02:00