This is almost 100% source compatible with Qt 5. Exceptions are
* Stability of references for large or non movable types
* taking a PMF for types that are now overloaded with r-value references
in QVector
* The missing prepend optimization in QVector (that is still planned
to come for Qt 6)
Change-Id: I96d44553304dd623def9c70d6fea8fa2fb0373b0
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Testlib's signaldumper functionality would crash inside
testlib as it dereferenced the sender after it was deleted.
Change-Id: I6013b75b0a121e2768429d8a3cf0339a940314f2
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
The test is not failing anymore on QEMU targets.
This partially reverts commit
71bd06d516.
Fixes: QTBUG-71915
Change-Id: I68593edf0ec245e14879833c8aa90661a3c2e227
Reviewed-by: Liang Qi <liang.qi@qt.io>
When running Qt autotests on a developer machine with a high
resolution, failures occur due to either some widget becoming too
small, some rounding fuzz appearing when Qt High DPI scaling is active,
or some test taking screenshots failing to deal with device pixel
ratios != 1 in the obtained pixmaps.
It is not feasible to adapt all tests to pass on high resolution
monitors in both modes (Qt High DPI scaling enabled/disabled). It
should be possible to specify the High DPI setting per test.
Previously, it was not possible to set the Qt High DPI scaling
attributes since they must be applied before QApplication
instantiation.
Enable this by checking for the presence of a static void initMain()
function on the test object and invoking it before QApplication
instantiation.
Prototypically use it in tst_qtimer and to turn off High DPI scaling for
tst_QGL.
[ChangeLog][QtTestLib] It is now possible to perform static
initialization before QApplication instantiation by implementing a
initMain() function in the test class.
Change-Id: Idec0134b189710a14c41a451fa8445bc0c5b1cf3
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
This functionality is especially convenient if meta-object system is
heavily used in a test. For example, if you need to test a bunch of
signals based on their names and/or argument types.
Change-Id: I09a4ecbbd3d0859b5fd466d9dde7679804eb7614
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
This show cases the new cleaner subdirs handling code in
pro2cmake.
Change-Id: I25001942ef020e4c4191b355f55309c4fb033ecc
Reviewed-by: Qt CMake Build Bot
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
When accidentally running a test doing screen-grabbing
with High DPI scaling active, sizes of the obtained pixmaps
can differ due to the device pixel ratio. Add a check to make that
clearer.
[ChangeLog][QtTestLib] Comparison of QImage, QPixmap now checks for the
device pixel ratio.
Change-Id: Id8d5187e99c565c44a7bfb8b9cfb09737815fb15
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Only re-enable exceptions for the modules that do
CONFIG+=exceptions
in qmake
Change-Id: I9f19078adbdc1b8fa3d4102fb51a099e7e35522e
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
This will help with future regeneration.
Amends 6d1c695b0e
Change-Id: I2454dc36cfc00f7aad72488fb7927d7807978fd8
Reviewed-by: Qt CMake Build Bot
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Actually build tst_selftest
Fix the subdirs of it not to be actually built as tests (so that ctest
doesn't pick them up and tries to run them)
Change-Id: Id9d069f44241027dbd103884d36551163382b216
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Qt CMake Build Bot
This is part of the migration of qtbase from QRexExp to
QRegularExpression.
Task-number: QTBUG-72587
Change-Id: I23bc9693fb9f553fd63d10687d51322394717ed7
Reviewed-by: Christian Ehrlicher <ch.ehrlicher@gmx.de>
Reviewed-by: Konstantin Shegunov <kshegunov@gmail.com>
Conflicts:
src/corelib/tools/qlocale_data_p.h
(Regenerated by running the scripts in util/local_database/)
src/gui/opengl/qopengltextureuploader.cpp
Done-With: Edward Welbourne <edward.welbourne@qt.io>
Done-With: Allan Sandfeld Jensen <allan.jensen@qt.io>
Change-Id: I12df7f066ed0a25eb109f61c4b8d8dea63b683e2
If the deadline is far in the future, the conversions to nanoseconds
or internal arithmetic may overflow and give an invalid object, thus
the deadline may end up in the past. Added a test to the testlib
selftest for sleep.
[ChangeLog][QtCore][QDeadlineTimer] Fixed integer overflows
leading to immediate timeouts.
Task-number: QTBUG-69750
Change-Id: I9814eccdf9f9b3add9ca66ec3e27e10cd5ad54a8
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
The first replacement had missed objective-C++ code some places ourside
the src dir.
In C-files Q_DECL_NOTHROW is replaced with Q_DECL_NOEXCEPT as we still
need to turn it off when compiled in C mode, but can get rid of the old
NOTHROW moniker.
Change-Id: I6370f57066679c5120d0265a69e7e378e09d4759
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
TestMethods::invokeTest() has an outer loop on global data (albeit
with a comment that said otherwise). On its first cycle, we run the
test function's *_data() method, if it has one; there is an inner loop
on the rows this created. If the *_data() QSKIP()s, we need to skip
the whole test; otherwise, a QSKIP() in one sub-test should not lead
to skipping the remaining sub-tests.
Moved the check for *_data() QSKIP()ping to right after *_data()
returns, inside the "first global cycle" block that runs it.
Previously, this check was done before entering the loop on local data
rows, but outside that "first global cycle" block: consequently, later
global cycles would fall foul of this check (even though the *_data()
hasn't been run in this cycle, much less QSKIP()ped in it) if the last
sub-test of the previous global cycle had QSKIP()ped.
When running a single test for one specific data row, if the test's
*_data() QSKIP()ped, this misplaced check would also have lead to a
misleading "Unknown testdata" warning.
Changed testlib/selftests' tst_globaldata::skipSingle() to trigger the
bug (by having its last local row of first global row skip, which
caused the second global row to be omitted) to verify this is also
fixed; and amended one of its comments to reflect what's now to be
expected. Updated the test's expected output files.
Task-number: QTBUG-61774
Change-Id: I99596b595c6d1184038f23383844c6ff51a0cd91
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
The test was added as commented in 2012, before it was not even
mentioned in the pro file. I guess it was never run.
Change-Id: Ib787c7811ab4f7f32a99869167cde2e12dbe3156
In particular, document the mis-behavior it takes for granted in one
test. Renamed some variables so that they're visually distinct from
one another and actually mean something intelligible. Renamed the
sub-tests to say what's actually happening. Reading the test output
now at least makes it possible to see what's wrong. I'll fix that in
a later commit.
Task-number: QTBUG-61774
Change-Id: I3e5f83b8baa3c6afbca9231b5bbc89d17f3e57e2
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
It has been completely untested for a while and saw some breakage.
So let's add a selftest for it. ('-vs' when running tests)
Change-Id: Ibfb5ac0a2d741de7c3f519d91202d4977996045e
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The QTest::qCompare() implementations were almost duplicates; pull the
common code out into a templated version. Tweaked the
QTest::toString() specialization for float and double (a macro) and
fixed a bous modifier in double's format.
The doubleComparisons and floatComparisons tests in the tst_float.cpp
selftest shared a large block of tests in common, aside from the
difference of type. Break this out into a templated static function
to save duplication.
This prepares the way for using the same templated code for qfloat16.
Change-Id: I2823fd006910c5ff88335d625d1fa05cb7753513
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
In QCOMPARE, handle NaNs and infinities the way tests want them
handled, rather than by strict IEEE rules. In particular, if a test
expects NaN, this lets it treat that just like any other expected
value, despite NaN != NaN as float16 values. Likewise, format
infinities and NaNs specially in toString() so that they're reported
consistently.
Enable the qfloat16 tests that depend on this QCOMPARE() behavior.
Refise the testlib selftest's float test to test qfloat16 the same way
it tests float and double (and format the test the same way).
This is a follow-up to 37f617c405.
Change-Id: I433256a09b1657e6725d68d07c5f80d805bf586a
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Conflicts:
tests/auto/widgets/itemviews/qheaderview/tst_qheaderview.cpp
Added tests/auto/testlib/selftests/expected_crashes_5.txt to work
round the output of the crashes test (which exercises UB, see
QTBUG-73903) being truncated on one test platform.
Change-Id: I9cd3f2639b4e50c3c4513e14629a40bdca8f8273
The silent and blacklisted selftests of testlib end in a qFatal(), to
test its messaging is handled correctly. However, this prevents hooks
in main() from saving coverage data when we're gathering that. So use
a transient signal handler that longjmp()s back to a setjmp() just
before the qFatal() to let the test complete normally (but, since
qFatal() does something different on MS-Win, don't apply this to it).
Note that testlib's internal FatalSignalHandler handles all fatal
signals *except* SIGABRT, so this isn't over-riding it. (In any case,
this restores the prior signal handler in setjmp()'s catch branch.)
Added missing expected_silent.tap test output while checking that this
change doesn't affect (the rest of) the test output.
Change-Id: I7e460581ad93e26639c066b3229438a66fd299de
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Tidied up the existing float tests in the process.
(In particular, s/SUCCESS/PASS/ since that matches real test output.)
These verify that QCOMPARE() handles floats and doubles as intended.
Extended the existing qFuzzyCompare tests to probe the boundaries of
the ranges of values of both types, in the process.
Revised the toString<double> that qCompare() uses to give enough
precision to actually show some of the differences being tested there
(12 digits, to match what qFuzzyCompare tests, so as to show different
values rather than, e.g. 1e12 for both expected and actual) and to
give consistent results for infinities and NaN (MinGW had eccentric
versions for these, leading to different output from tests, which thus
failed); did the latter also for toString<float> and fixed stray zeros
in MinGW's exponents (which made a kludge in tst_selftest.cpp
redundant, so I removed that, too).
That's further complicated handling of floating-point types, so let's
just keep an eye on how expensive that's getting by adding a benchmark
test for QTest::toString(). Unfortunately, default settings only get
runs that take modest numbers of milliseconds (some as low as 40)
while increasing this with -minumumvalue 100 or more gets the process
killed - and I'm unable to find out who's doing the killing (it's not
QProcess::kill, ::kill or the QtTest WatchDog, as far as I can tell).
So results are rather noisy; the integral tests exhibit speed-ups by
factors up to 5, and slow-downs by factors up to 100, between runs
with and without this change, which does not affec the integral tests.
The relatively modest slow-downs and speed-ups in the floating point
tests thus seem likely to be happenstance rather than signal.
Change-Id: I4a6bbbab6a43bf14a4089e96238a7c8da2c3127e
Reviewed-by: Ulf Hermann <ulf.hermann@qt.io>
Only for tests that have existing expected_*.* files for other
formats, though.
Change-Id: I34ca1900d88454f300e04d849a608c378009489b
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Regexes have long specified that a [ as the first character inside a
[...] is just a literal [, but apparently we need to escape it now, to
avoid a "nested set" FutureWarning.
Change-Id: I76a48c9aafb0684a1d6b0d5284fe9852c9ea0e43
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Samuel Gaist <samuel.gaist@idiap.ch>
It isn't being built in the current qmake build either (maybe because it
doesn't build because uses sleep())
Change-Id: Ie97287c6e45d1fcc32063bd9eb37ff2907dfa891
Reviewed-by: Tobias Hunger <tobias.hunger@qt.io>
Prioritize blacklisting over QEXPECT_FAIL so that a test that is
blacklisted no longer fails if QEXPECT_FAIL returns true unexpectedly. To
reflect this state properly, the two values of BXPASS and BXFAIL were
added to testlib's output.
[ChangeLog][Important Behavior Changes][QtTestLib] Blacklisting of tests
will be taken into account for XPASS and XFAIL. A blacklisted test that
causes an XPASS will no longer be a fail.
Task-number: QTBUG-72928
Change-Id: Ia2232fdc714d405fa3fd9aea6c89eb2836bc5950
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Non-local ASM labels break for this test when compiled with clang.
Change-Id: I15bd250a991c3b03bbc88459a6358090bd157444
Reviewed-by: Jesus Fernandez <Jesus.Fernandez@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
Based on Asmo Saarela's advice (QTPM-686), adapted on advice from
FrogLogic support and converted to a feature so that the selftest and
testlib qmake config can be co-ordinated.
Task-number: QTPM-1385
Change-Id: Icd706f086009e1e08b3f8c5cd553f792402e28c0
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
The output files for the tuplediagnostics selftest of testlib had a
stray non-canoical path fragment in them; so replaced with its
canonical form.
Change-Id: Ib421380036c3fb1b91447eb8c87be4ad0dfe5c96
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
Verify that it does cut in after the specified time has elapsed.
Task-number: QTPM-1385
Change-Id: Ib18e8d6af28339f79cca4d62b869287ce07b8cc1
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
Match the environment tst_selftests.cpp uses for subtests more
faithfully. Extends b22e50acda. In the process, tweak how crashers
are handling, in preparation for the watchdog test.
Change-Id: I09a046460f6f3bff0b12069fad6c1437d89572ce
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
One test for bad data for the column, another for a bad QFETCH.
Incidentally extend blacklist testing by blacklisting them.
Reorganise a QEMU condition that needed extended as part of this.
Task-number: QTPM-1385
Change-Id: Iac72ada19760321c5c9264ddfff7740d1fdd0700
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
Some tests were fixed and others were skipped/blacklisted.
Task-number: QTBUG-63152
Change-Id: Ica7df555f8d152ee589865911130525101d4b941
Reviewed-by: Liang Qi <liang.qi@qt.io>
QTreeWidget::(is|set)Item(Selected|Hidden|Expanded)() are deprecated
for a long time but not marked as such. Therefore explicitly mark them
as deprecated so they can get removed with Qt6.
Change-Id: Ie4971350de61326811e0788df0d359ed3c442869
Reviewed-by: Konstantin Shegunov <kshegunov@gmail.com>
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@qt.io>
It's not actually exercised by anything in our source tree, but is
potentially useful and has been part of the documented public API for
some time. So mention that the caller is responsible for delete[]ing
its return and add a test that exercises it.
Task-number: QTPM-1385
Change-Id: Ifc5284b9eb1b678cf3c9708c681311e874838fc6
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
QTreeView/TableView::sortByColumn(int) was deprecated a long time ago
but never got removed. Therefore mark it with QT_DEPRECATED_SINCE(5, 13)
so we can remove it with Qt6.
Also sync the handling of the sort order changes in QTableView with the
one from QTreeView.
Change-Id: I0371d9a9c21116edaa9125835827f1a200075d36
Reviewed-by: Luca Beldi <v.ronin@yahoo.it>
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@qt.io>
In the process, corrected an inaccurate XFAIL message (an XPASS is
normally an error, unless blacklisting ignores it so turns it into a
BPASS). Added the missing .tap file to its expected output.
Documented the similarity to the silent/ selftest.
Task-number: QTPM-1385
Change-Id: Id74a1353d54af2f3bfe2c764e33c1f051958ab21
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
The testlib selftest sets various things in the environment for
crashing tests; the generator for its expected output should set the
same things, as they affect what output is produced.
Change-Id: Iec2ed59982ea1043582573530c33619d8e8ed08e
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
If the two lines have identical texts, the comparison returns true.
So don't complicate various other conditions on the way there with
filtering out that case; deal with it first so they don't need to.
Change-Id: Iebd230704ce5f53d12d5afa64aab30f83bb9d407
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
This only enables compilation, it doesn't fix any test.
Qt on Android supports process, but not TEST_HELPER_INSTALLS. See also
acdd57cb for winrt.
android-ndk-r10e is used to compile, see
http://doc-snapshots.qt.io/qt5-5.11/androidgs.html .
corelib/io/{qdir,qresourceengine} need to be fixed later.
Done-with: Frederik Gladhorn <frederik.gladhorn@qt.io>
Done-with: Mårten Nordheim <marten.nordheim@qt.io>
Change-Id: I34b924c8ae5d46d6835b8f0a6606450920f4423b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Before the fix is applied this test fails because QFINDTESTDATA will
return "/usr/" instead of the folder with the same name in the current
directory.
The 'usr' folder can't be located 'next to' the application since this
does not trigger the issue (QFINDTESTDATA looks for the folder next
to the executable early on). So we put it in a subdirectory and change
the current working directory to its parent directory.
Change-Id: I627679dcb6f2f6954264e23bfc1a71de3bff7203
Reviewed-by: Jesus Fernandez <Jesus.Fernandez@qt.io>
Currently when doing comparison with std::tuple the fallback toString
method is called which returns a Q_NULLPTR thus not allowing proper
diagnostic of the values that triggered an error. This patch
adds support for std::tuple to improve the tests output readability.
[ChangeLog][QtTest][QCOMPARE] Now outputs contents of std::tuple on
failure.
Change-Id: I046a55e2ce44c3f7728d51e4745120d38aa5e007
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This supplements b1945604a7, which
removed the qrc file in favor of test/test.pro coding for it.
Change-Id: I15507c89ca14fa6e6b8223de671ffff7092272d0
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Put it in alphabetical order like the rest of the list.
Change-Id: I3da3bb68d1847f53419bb79490b946c935ebb518
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
They didn't exist up until now, and future patches rely on them, so
add them.
Change-Id: I8afdb9417263b45d43355c688a813bdf99ea5fc8
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
When generate_expected_output.py is run for an in-source build, the
raw output contains no paths to the sources for the script to whittle
down, as it does for shadow builds, to just the path from qtbase down.
So kludge together some extra regexes that can fix that up and tweak
some relevant code to provide them with the data they need.
Change-Id: I656d7126087bd9ad20b2af6835fba314d90a171d
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
This is what the other reporters also do, in various forms.
Task-number: QTBUG-67351
Change-Id: I16f2c4e0991176145ee0fbcbbfeeda071603a3c2
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
The Test Anything Protocol (TAP), was originally Perl's simple text-based
interface between testing modules and test harnesses, but has since been
adopted by a large number of producers and consumers in many different
languages, which allows colorizing and summarizing test results.
The format is very simple:
TAP version 13
ok 1 - test description
not ok 2 - test description
---
message: 'Failure message'
severity: fail
expected: 123
actual: 456
...
ok 3 - test description # SKIP
1..3
The specification [1] is very brief, so the implementation has been
based on how typical consumers behave, especially when it comes to
the undefined diagnostics block.
[1] http://testanything.org/tap-version-13-specification.html
Change-Id: I616e802ea380165c678510e940ddc6607d39c92d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Using QFileInfo to check if the file exists based on the filename
alone ignores the fact that all the expected files are embedded
as QRC resources.
The expectedResult() function already does a similar check, so we
can use that directly instead of checking twice if the file exists.
Task-number: QTBUG-66981
Change-Id: I0beb8d3503ed49682ae7d7e2a5172922fab5420d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Otherwise random debug messages from Qt might mess up the expected vs
actual results.
The setting of QT_LOGGING_RULES in initTestcase has been removed, as
the selftest overrides that for each invocation of a subtests, via
the processEnvironment() function.
Task-number: QTQAINFRA-1631
Change-Id: I855d31274f8261f8b125df23409353f7101be0e4
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Instead of keeping a static list in a qrc file.
Change-Id: I7783db5cad5929cc3f449889e1cbcbb87f7f2637
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Silence debug output by setting QT_LOGGING_RULES
to turn off all debug output.
Task-number: QTQAINFRA-1631
Change-Id: I5c2366b4fe4bac341dcfd92f68b6da8071c5b089
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
After a37785ec76 went in, it become
apparent that multi mouse button state handling in qtestlib is
non-existent, for details see QTBUG-64030 and QTBUG-63786. What
happened behind the scenes often was not what one would expect based
on the provided QTest::mouse* input sequence - events went missing,
incorrect events were generated, each subsequent test function
started with a state set from the function that run earlier. It is
easy to see how a minor change in one test could easily affect outcome
of other tests.
With a37785ec76, Qt platform plugins
are now responsible for sending explicit mouse button type and state
information; qtestlib should take full responsibility now as well.
But using the new API from a37785ec7 alone in qtestlib is not sufficient.
We need to reset mouse state between each new test function run (we do
this at function scope as that fits with the current qtestlib API user
expectations). This patch implements the necessary reseting logic.
Updated tst_qwindow.cpp::generatedMouseMove() to use QTest::mouse* APIs.
That test requires pressing multiple buttons, it was not possible with
QTest::mouse* APIs before this patch.
Added an auto test for multiple mouse button pressing/release in
tests/auto/testlib/selftests/mouse/. And few other tests which are
currently QSKIP-ed, but should be considered when re-designing qtestlib
APIs.
Task-number: QTBUG-64030
Change-Id: I39fdcbc73a467a7463ce2aed622bf22484095635
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Add formatting for registered enumerations based on QMetaEnum
and unregistered enumerations as hex values.
[ChangeLog][QtTest] QtTest now prints values of QFlags that failed to
compare with QCOMPARE.
Task-number: QTBUG-65845
Change-Id: I3eae6d20d3c0d72441ca6c4037d9a8dafa4b6357
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The "crashes" subtest has several "expected" reference files. The matching
one was previously determined by checking the line count of the output.
This however does not work when there are several reference files with
identical line count as is now the case with boot2qt.
Refactor the comparison code from the QTest/preliminary void return to
the bool f(QString* errorMessage) convention so that all files can be tried.
While doing so, streamline the code and remove numerous unneeded
allocations of strings and regular expressions per compared line.
Task-number: QTBUG-65845
Change-Id: I722159d1753f2a36f0e497e315ffd81cb58cac0b
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
It has been observed that sometimes the output is not captured by QEMU.
Task-number: QTBUG-65845
Change-Id: Iddf888fb3aa019209dbe71f424d87884471784e0
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Use qInfo() instead of qDebug() in case QDebug is turned off.
If the 'diff' tool is available, write the output to temporary files and
run diff on it. Otherwise, print the lines as was before, but onto one
stream to avoid indentation by the testlib handler.
Change-Id: Ib5a5dfb66ce481b493b85b915aa8c785ecb6b387
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Move the modeltest autotest in the right place, and fix the
other autotests that were using it to use the version now in QtTestLib.
Change-Id: Ic6838945f616d580f357c872ce0956c341be3b16
Reviewed-by: David Faure <david.faure@kdab.com>
Currently, when two colors are equal except for their alpha values,
QCOMPARE produces the following failure message:
FAIL! : tst_Test::test() Compared values are not the same
Actual (colorA): #ff0000
Expected (colorB): #ff0000
By using the HexArgb format instead of the default HexRgb, we can
see the full hex string, with alpha values included:
FAIL! : tst_Test::test() Compared values are not the same
Actual (colorA): #88ff0000
Expected (colorB): #ffff0000
Task-number: QTBUG-55574
Change-Id: Id82c60a1b473ac6025a6f6ac560fce95a910d782
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
On some systems (e.g. Debian/buster) a standard Qt package may install
a qtlogging.ini file, e.g. in /etc/xdg/QtProject/, which suppresses
QDEBUG output. A ~/.config/QtProject/qtlogging.ini could interfere
similarly. This can break the selftest, when run with expected_*
files generated without such interference. Likewise, if those
expected_* files are generated on a system such similar, they won't
work on CI. Given that this caused confusion and delay with
integrating the latest set of updates to the expected_* files, it
seemed best to save others from similar bafflement.
So set a standard value for QT_LOGGING_RULES in both the selftest and
the generator script, so we get consistency.
Change-Id: I649e2f1f6ead21edf8af051aaee286e369fed064
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
In selftests.qrc, imposed alphabetic order (on stem of name, then on
suffix, effectively treating . as sorting before any letter) while
removing old tests and adding new tests and data. Updated all non-csv
files and added many missing files. (Not clear on csv support status;
the script seems to have dropped it after 5.6, but the test still uses
it.)
Left expected_crashes* alone (no new files added, no update to old) as
I don't get results resembling those anticipated.
Omitted printdatatagswithglobaltags, printdatatags due to dangling
hspace on output lines, which upset sanity-bot. A change to the test
cpp is needed to make it viable to skip that dangling hspace.
Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
This makes qtestlib behave more like real Qt applications with an
event loop, where exiting the application does indeed flush deleteLaters.
And since every test method is supposed to be independent from other
test methods, we should even cleanup between tests.
For "app less" tests, no flushing happens.
Real life use cases:
* A unittest for some code (e.g. KIO job) which uses a socket, runs an
event loop until the socket communication is done, and ends. However
slotDisconnected() does, as recommended, socket->deleteLater(). So the
test finishes before the socket has a chance to actually get deleted,
and memory leak tools flag a memory leak, which doesn't actually happen
outside the unittest.
* Deleting a QWebEngineView with deleteLater is even worse than a memleak,
it leads to an assert (from a global object destructor) in QtWebEngine
if the view deletion hasn't actually been processed.
Change-Id: I18fc822fd26988a0fa4e75313c1947fcaa7d9e56
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Conflicts:
src/widgets/kernel/qwidget.cpp
This merge also extends the expected output of the pairdiagnostics
teamcity output (added in dev in commit
c608ffc56a) after the recent addition of
the flowId attribute to the teamcity output (commit
8f03656211 in 5.9).
Change-Id: I3868166e5efc45538544fffd14d8aba438f9173c
Added flowId='name' to each message when using TeamCity logging format.
This is necessary to distinguish separate processes running in parallel.
[ChangeLog][QtTest] Added flowId to messages when logging in TeamCity
format. FlowId is used to distinguish logging from multiple processes
running in parallel.
Change-Id: I7f5046c1058ff02770404caa2c9b3a5398f97f6b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Two more ways line numbers were making it through.
Corrected a doc-string to tell nearer to the truth.
Change-Id: I946aaeb936d47fffe50d7ec15e2524992cc9e428
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Document that the saved output is used by tst_selftests.cpp and use a
crude parse of it to get the list of subdirs that it actually tests.
Change-Id: I73023228c9e547f965b7749dd66de7ef09c3815e
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
Currently when doing comparison with pair-like classes the fallback
toString method is called which returns a Q_NULLPTR thus not allowing
proper diagnostic of the values that triggered an error. This patch
adds support for QPair and std::pair to improve the tests output
readability.
[ChangeLog][QtTest][QCOMPARE] Now outputs contents of QPair and
std::pair on failure.
Change-Id: Ib0b4aad7640590d56275b6f1306fb9fb39fa81bc
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Two changes are needed to pass tst_Selftest on QEMU
1. Pass QEMU specific env variables to the subtests
2. Ignore output on stderr on some tests when running on QEMU
Change-Id: Ie1f722fd183aac5973e87d408005e06cbafcde17
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
