Use qInfo() instead of qDebug() in case QDebug is turned off.
If the 'diff' tool is available, write the output to temporary files and
run diff on it. Otherwise, print the lines as was before, but onto one
stream to avoid indentation by the testlib handler.
Change-Id: Ib5a5dfb66ce481b493b85b915aa8c785ecb6b387
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Currently, when two colors are equal except for their alpha values,
QCOMPARE produces the following failure message:
FAIL! : tst_Test::test() Compared values are not the same
Actual (colorA): #ff0000
Expected (colorB): #ff0000
By using the HexArgb format instead of the default HexRgb, we can
see the full hex string, with alpha values included:
FAIL! : tst_Test::test() Compared values are not the same
Actual (colorA): #88ff0000
Expected (colorB): #ffff0000
Task-number: QTBUG-55574
Change-Id: Id82c60a1b473ac6025a6f6ac560fce95a910d782
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
On some systems (e.g. Debian/buster) a standard Qt package may install
a qtlogging.ini file, e.g. in /etc/xdg/QtProject/, which suppresses
QDEBUG output. A ~/.config/QtProject/qtlogging.ini could interfere
similarly. This can break the selftest, when run with expected_*
files generated without such interference. Likewise, if those
expected_* files are generated on a system such similar, they won't
work on CI. Given that this caused confusion and delay with
integrating the latest set of updates to the expected_* files, it
seemed best to save others from similar bafflement.
So set a standard value for QT_LOGGING_RULES in both the selftest and
the generator script, so we get consistency.
Change-Id: I649e2f1f6ead21edf8af051aaee286e369fed064
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
In selftests.qrc, imposed alphabetic order (on stem of name, then on
suffix, effectively treating . as sorting before any letter) while
removing old tests and adding new tests and data. Updated all non-csv
files and added many missing files. (Not clear on csv support status;
the script seems to have dropped it after 5.6, but the test still uses
it.)
Left expected_crashes* alone (no new files added, no update to old) as
I don't get results resembling those anticipated.
Omitted printdatatagswithglobaltags, printdatatags due to dangling
hspace on output lines, which upset sanity-bot. A change to the test
cpp is needed to make it viable to skip that dangling hspace.
Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
This makes qtestlib behave more like real Qt applications with an
event loop, where exiting the application does indeed flush deleteLaters.
And since every test method is supposed to be independent from other
test methods, we should even cleanup between tests.
For "app less" tests, no flushing happens.
Real life use cases:
* A unittest for some code (e.g. KIO job) which uses a socket, runs an
event loop until the socket communication is done, and ends. However
slotDisconnected() does, as recommended, socket->deleteLater(). So the
test finishes before the socket has a chance to actually get deleted,
and memory leak tools flag a memory leak, which doesn't actually happen
outside the unittest.
* Deleting a QWebEngineView with deleteLater is even worse than a memleak,
it leads to an assert (from a global object destructor) in QtWebEngine
if the view deletion hasn't actually been processed.
Change-Id: I18fc822fd26988a0fa4e75313c1947fcaa7d9e56
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Conflicts:
src/widgets/kernel/qwidget.cpp
This merge also extends the expected output of the pairdiagnostics
teamcity output (added in dev in commit
c608ffc56a) after the recent addition of
the flowId attribute to the teamcity output (commit
8f03656211 in 5.9).
Change-Id: I3868166e5efc45538544fffd14d8aba438f9173c
Added flowId='name' to each message when using TeamCity logging format.
This is necessary to distinguish separate processes running in parallel.
[ChangeLog][QtTest] Added flowId to messages when logging in TeamCity
format. FlowId is used to distinguish logging from multiple processes
running in parallel.
Change-Id: I7f5046c1058ff02770404caa2c9b3a5398f97f6b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Two more ways line numbers were making it through.
Corrected a doc-string to tell nearer to the truth.
Change-Id: I946aaeb936d47fffe50d7ec15e2524992cc9e428
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Document that the saved output is used by tst_selftests.cpp and use a
crude parse of it to get the list of subdirs that it actually tests.
Change-Id: I73023228c9e547f965b7749dd66de7ef09c3815e
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Currently when doing comparison with pair-like classes the fallback
toString method is called which returns a Q_NULLPTR thus not allowing
proper diagnostic of the values that triggered an error. This patch
adds support for QPair and std::pair to improve the tests output
readability.
[ChangeLog][QtTest][QCOMPARE] Now outputs contents of QPair and
std::pair on failure.
Change-Id: Ib0b4aad7640590d56275b6f1306fb9fb39fa81bc
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Two changes are needed to pass tst_Selftest on QEMU
1. Pass QEMU specific env variables to the subtests
2. Ignore output on stderr on some tests when running on QEMU
Change-Id: Ie1f722fd183aac5973e87d408005e06cbafcde17
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Replace all QT_NO_PROCESS with QT_CONFIG(process), define it in
qconfig-bootstrapped.h, add QT_REQUIRE_CONFIG(process) to the qprocess
headers, exclude the sources from compilation when switched off, guard
header inclusions in places where compilation without QProcess seems
supported, drop some unused includes, and fix some tests that were
apparently designed to work with QT_NO_PROCESS but failed to.
Change-Id: Ieceea2504dea6fdf43b81c7c6b65c547b01b9714
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
The future tests don't need QtConcurrent as QFuture and friends are in
QtCore. The printdevice test doesn't use QtNetwork and the lancelot as
well as the testlib tests don't use QtXml.
Change-Id: I150ac99b36682aa23ad22ba943266eb0f0952838
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@qt.io>
5971b88e is not needed in new configure.
This merge also reverts "fix QMAKE_DEFAULT_*DIRS resolution with
apple SDK", 2c9d15d7, because it breaks iOS build with new
configure system.
Conflicts:
mkspecs/features/default_pre.prf
mkspecs/features/mac/toolchain.prf
mkspecs/features/toolchain.prf
src/dbus/qdbusconnection.cpp
src/plugins/sqldrivers/mysql/qsql_mysql.cpp
src/sql/drivers/mysql/qsql_mysql.cpp
src/widgets/widgets/qmenubar.cpp
src/widgets/widgets/qmenubar_p.h
tools/configure/configureapp.cpp
tools/configure/environment.cpp
tools/configure/environment.h
Change-Id: I995533dd334211ebd25912db05b639d6f908aaec
This one selftest is currently blocking OS X 10.11
from entering the CI. It can't be reproduced when
run manually.
Task-number: QTBUG-55155
Change-Id: I4553ef2d7813b29f5dc8577976c4482686346504
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@theqtcompany.com>
Restructured, separated the canonicalising of output lines out (into
an object that prepares the necessary regexes and replacements)
suppress more, changed the path-stripping to strip qtbase's parent
rather than os.getcwd() and took account of shadow builds (so both
source tree and build tree provide prefixes we want to strip from
paths). Also cope with $PWD potentially having symlinks in it, where
os.getcwd() is canonical.
It's possible some output might name files elsewhere in the source
tree; these won't be filtered by the prior cwd prefix removal; and, in
any case, the problem with cwd is only that the ancestry of qtbase is
apt to vary; paths relative to there should be consistent between test
runs. This change shall lead to a one-off rewrite of all expected_*
files; but it should now catch all paths. By stripping both build
root and source root (when different) it also avoids differences for
those doing out-of-source ("shadow") builds.
In our XML formats, any hyphens in root paths (e.g. I had Qt-5.6 in my
build root's path) got represented by a character entity, confounding
the replacement; so also do replacement that catches this. We may
discover other character entity subsitutions needed along with this.
Now filtering line numbers and timing information, including benchmark
results; these numbers all get replaced with 0 to avoid noisy diffs.
Also purging dangling hspace, to placate sanity-bot.
The module can now be imported - the code it runs is packaged as a
main() function that a __name__ == '__main__' stanza runs - and all
data is localised to where it's needed, rather than held in globals.
Tidied up and organized the existing regexes. There are doc-strings;
there is a short usage comment. Data is localised rather than global
and modern pythonic idioms get used where apt.
Regexes are compiled once instead of repeatedly. An object looks
after the list of patterns to apply and its construction handles all
anticipated problems. Failures are mediated by an exception.
The output file now gets written once, instead of twice (once before
editing, then over-write to edit), and Popen uses text mode, so that
write can do the same. Its command is delivered as an array, avoiding
the need to invoke a shell.
Instead of relying on qmake being in our path (which might give us a
bogus QT_VERSION if the one in path doesn't match our build tree), use
the relative path to qmake - we rely on being run in a specific
directory in the build tree, after all. Escape dots in the version
properly, so that 51730 doesn't get mistaken for 5.7.0 (for example),
and moved this check later in the sequence (matching a smaller target
makes it more likely to falsely match).
Overtly check we are in the right directory and tell the user what we
actually need, if run from the wrong place. Simplify handling of the
unsupported use-case for MS-Windows (but note what would be needed for
it).
Change-Id: Ibdff8f8cae173f6c31492648148cc345ae29022b
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Conflicts:
qmake/library/qmakebuiltins.cpp
qmake/library/qmakeevaluator.cpp
qmake/library/qmakeevaluator.h
qmake/project.h
QMakeEvaluator:
* evaluateConditional(): one side changed return type, the other
changed a parameter type.
* split_value_list(): one side changed a parameter adjacent to where ...
* expandVariableReferences(): ... the other killed one overload and
changed the survivor
src/corelib/io/qlockfile_unix.cpp
One side changed a #if condition, the other moved NETBSD's part of
what it controlled.
src/corelib/tools/qdatetime.cpp
One side fixed a reachable Q_UNREACHABLE in toMSecsSinceEpoch(), the
other moved it from the private class to the public one, in the midst
of the "short date-time" optimization, which confused diff entirely.
One side changed a QStringLiteral to QLatin1String, the other rewrote
adjoining code.
src/network/kernel/qauthenticator.cpp
Both rewrote a line, equivalently; kept the dev version.
src/platformsupport/fontdatabases/mac/qfontengine_coretext.mm
src/platformsupport/fontdatabases/mac/qfontengine_coretext_p.h
One side changed #if-ery that the other removed.
tools/configure/configureapp.cpp
One side added a check to -target parsing; the other killed -target.
tests/auto/testlib/selftests/expected_cmptest.lightxml
tests/auto/testlib/selftests/expected_cmptest.teamcity
tests/auto/testlib/selftests/expected_cmptest.txt
tests/auto/testlib/selftests/expected_cmptest.xml
tests/auto/testlib/selftests/expected_cmptest.xunitxml
Regenerated using generate_expected_output.py
I note that quite a few other expected_* come out changed, now.
There was no git-conflict in
src/widgets/kernel/qformlayout.cpp
but it didn't compile; one side removed some unused methods; the other
found uses for one of them. Put FixedColumnMatrix<>::removeRow(int)
back for its new user.
Change-Id: I8cc2a71add48c0a848e13cfc47b5a7754e8ca584
Conflicts:
qmake/library/qmakeevaluator.cpp
One side changed the iterator to use ranged-for, the other changed its
body; they only conflicted because the latter had to add braces around
the body, intruding on the for-line. Trivial resolution.
Change-Id: Ib487bc3bd6e3c5225db15f94b9a8f6caaa33456b
Output regions as a compact list of rectangles:
QRegion(2 rectangles, 50x200+100+200, 200x50+10+10)
Change-Id: I62e42bd30897a1d31b28e0cbd6afe7fa8ecf2664
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Otherwise, float numbers are formatted using a decimal comma
in German, causing huge diffs and failures.
Change-Id: Icd85a293d0564cac6be244eb0793611920d0c89c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
QMetaEnum::key(n) returns the n-th key, which is not necessarily the one
of value n. For the key of value n, we want QMetaEnum::valueToKey(n).
Change-Id: Ic90fe6b1cbe84978a02fffff141bf4a06074917a
Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com>
[ChangeLog][QTestLib] It is now possible to use variables of
types with an explicit operator bool in the QVERIFY macro.
Change-Id: I2685df164a616f6a065d689867daa9ea1de78e08
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Remove #ifdef sections for Q_OS_WINCE and wince .pro file clauses in library
and tests.
Task-number: QTBUG-51673
Change-Id: I552b3fe8d6e0eb7c8c7b3a3d41558e5e21904dd2
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
Adapt the script to generate output for the new TeamCity format
added by fbd6acedac.
Change-Id: I9435382ec3daf80428c324c58434aa951841bf08
Reviewed-by: Borgar Øvsthus <borgar.ovsthus@fmcti.com>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@theqtcompany.com>
When a QCOMPARE of values of type QColor fails, their
name will now be printed.
Task-number: QTBUG-51124
Change-Id: I76565daa338f038ea4f452e47705e638d94eaeee
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@theqtcompany.com>
If QT_PRESERVE_TESTLIB_PATH is set, then LD_LIBRARY_PATH and
QT_PLUGIN_PATH won't be filtered out for the sub-tests started by
tst_selftest.
Change-Id: Ic43ba9b4d882ee36b2f7495b1c880f26aefd2629
Reviewed-by: aavit <eirik.aavitsland@theqtcompany.com>
This allows TeamCity to parse realtime test-results instead of using
post-build XML Report Processing. This does not support logging of
benchmarks.
[ChangeLog][QTest] Added a new logging mode that allow test-results to
be parsed on-the-fly when using Jetbrains TeamCity as CI-server. This
mode is enabled by using the -teamcity option on the command-line.
Change-Id: Ie730beb643043eb0f448f99abe6c0b5ac48aaf03
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
I wrote a script to help find the files, but I reviewed the
contributions manually to be sure I wasn't claiming copyright for search
& replace, adding Q_DECL_NOTHROW or adding "We mean it" headers.
Change-Id: I7a9e11d7b64a4cc78e24ffff142b506368fc8842
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
From Qt 5.7 -> tools & applications are lisenced under GPL v3 with some
exceptions, see
http://blog.qt.io/blog/2016/01/13/new-agreement-with-the-kde-free-qt-foundation/
Updated license headers to use new GPL-EXCEPT header instead of LGPL21 one
(in those files which will be under GPL 3 with exceptions)
Change-Id: I42a473ddc97101492a60b9287d90979d9eb35ae1
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Lars Knoll <lars.knoll@theqtcompany.com>
TEST_HELPER_INSTALLS cannot be used on platforms with no
QProcess support.
Change-Id: I2a6a283d94ca4487fc628449c53fc37140dd291d
Reviewed-by: Oliver Wolff <oliver.wolff@theqtcompany.com>
Otherwise write() for my python (OS X 2.7) wants strings, which is incorrect.
Change-Id: Ibd9d050646d1039ba8370d121dd25756ceffdb7a
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@theqtcompany.com>
As these are strongly typed, they won't implicitly convert to int, so make sure
to cast explicitly.
Change-Id: Ic8daa31c528bbd8f399ab401d0963e13db191312
Reviewed-by: Olivier Goffart (Woboq GmbH) <ogoffart@woboq.com>
Previously, only QXmlTestLogger had timers to take elapsed times
and log them. Move those into class QTestLog for access by
the loggers and output the times in the crash dump to make it
easier to spot hangs/recursion crashes.
Produces:
QFATAL : foo() Received signal 11
Function time: 22ms Total time: 23ms
A crash occurred in ...
Function time: 24ms Total time: 26ms
Task-number: QTBUG-47370
Change-Id: Ia530a63104087daffc9a15f68c15d93378b9407e
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
Use QVERIFY2() with QTemporaryDir/File::errorString() consistently.
Attempt to catch issues like the below warning and follow-up issues.
QSYSTEM: tst_QFiledialog::clearLineEdit() QFileSystemWatcher: FindNextChangeNotification failed for "C:\Users\qt\_____aaaaaaaaaaaaaaaaaaaaaa" (Access is denied.)
Task-number: QTBUG-47370
Change-Id: I58a6e87c502627e976efa62ad73c912f3b2d49fa
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@theqtcompany.com>
The keyword no longer has a meaning for the new CI.
Change-Id: Ibcea4c7a82fb7f982cf4569fdff19f82066543d1
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
Make it easier to capture information about crashing tests in log files,
since it is not possible to do something like
foo > log.txt 2> errlog.txt
on Windows.
Adapt selftest.
[ChangeLog][QtTest][Important Behavior Changes] Crash/exception
information is now logged to standard output on Windows.
Task-number: QTBUG-47370
Change-Id: I3e6a2d25855ed0f20516418a031990b562f5f757
Reviewed-by: Simon Hausmann <simon.hausmann@theqtcompany.com>
