The blacklisted test does not crash, as its expected output indeed shows.
Pick-to: 6.5 6.6
Change-Id: I07522a7d065b5f39620975a3546bcd156024c41d
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
In porting the selftest machinery to Catch2 in 24e83de8d1 we
accidentally added an unconditional early return when determining
whether to check for unexpected stderr output, resulting in not
checking error output on any platform.
The return statement has now been moved into the Q_CC_MINGW
condition, but as we now seem to have similar issues on macOS
and Linux with some of the tests outputting "Received signal 6
(SIGABRT)" as we do for QEMU, we need to add a few more explicit
early return conditions to the function.
Pick-to: 6.5 6.6
Change-Id: I7a25f000843b5f1003a5db883f08285185046b46
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
QThreeWidgetItems that have been removed from their parents (or the
widget) must be deleted manually. The treeWidgetModel() test forgot
that, driving asan nuts.
Code predates the beginning of the public history, so picking to all
active branches.
Pick-to: 6.6 6.5 6.2 5.15
Change-Id: I139549b0bd8baf4abfb90f926f6290119471046f
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
a611a9f537 (in Qt 5) added support for
mixed-type comparisons through QCOMPARE. That commit added a new
overload for qCompare taking two types, T1 and T2; but it also left the
same-type qCompare(T, T) overload around, guarded by a Qt 6 version
check.
The mixed-type version is however not a generalization of the same-type
one, because it won't work if one of the arguments doesn't participate
in FTAD. Case in point: braced-init-lists. In Qt 5 this worked:
QCOMPARE(some_container, {42});
but in Qt 6 it does not work any more. The mixed-type overload fails
deduction (can't deduce T2); in Qt 5 the same-type overload deduced
T=SomeContainer, and {42} was used to select a constructor for
SomeContainer.
--
There's a partial, straightforward workaround for this: default T2 to
T1 in the mized-type overload. In that case T2 has a "fallback" if it
cannot be deduced. This is partial because of course doesn't address
the case in which T1 cannot be deduced, but I don't think that is
common at all.
QList is special here, because it has qCompare overloads that makes it
comparable with arrays, initializer lists and more. I don't like that
very much -- we should probably have a qCompare(input_range,
input_range) overload, but that's an exercise for C++20.
Change-Id: I344ba33167829984978cd8d649a1904349a9edab
Pick-to: 6.5 6.6
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Milian Wolff <milian.wolff@kdab.com>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Add the boilerplate standalone test prelude to each test, so that they
can be opened with an IDE without the qt-cmake-standalone-test script,
but directly with qt-cmake or cmake.
Boilerplate was added using the following scripts:
https://git.qt.io/alcroito/cmake_refactor
Manual adjustments were made where the code was inserted in the wrong
location.
Task-number: QTBUG-93020
Change-Id: I77299f990692b4fe4721a9bc35071608d0d23982
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Amir Masoud Abdol <amir.abdol@qt.io>
Using chrono means one can write 10s instead of 10'000.
[ChangeLog][QTest] Added qSleep(std::chrono::milliseconds) overload.
Change-Id: Iac1b12a3fc3f692b557e2d459e6f3bc565f20e93
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
As requested in code review. Big improvement, code-wise.
tst_Sleep::wait() was failing on the CI, so be more accurate by using
QElapsedTimer::durationElapsed(), which returns nanoseconds.
Pick-to: 6.6
Change-Id: I5bed6d6bd768adfdecab2475e6cbe245c20aabd7
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
[ChangeLog][QtCore][QDebug] Added pretty formatting of C++ <chrono>
durations.
[ChangeLog][QtTest] Added pretty formatting of C++ <chrono> durations
for QCOMPARE expressions.
Change-Id: I3b169860d8bd41e9be6bfffd1757cc087ba957fa
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
The test is accessing an out of bounds index into an array on purpose.
Change-Id: I23cce4093fdfabd177eb34639f26fcd6752c8b34
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
It's unlikely we will ever use pro2cmake at this project stage,
so it doesn't make any sense to keep the 'special case' markers
in the CMake scripts. Remove them and replace with TODO where
needed.
Change-Id: I84290c20679dabbfdec3c5937ce0428fecb3e5a7
Reviewed-by: Amir Masoud Abdol <amir.abdol@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Change-Id: I3087bff60e77c0eaa5818a5405c97628ce7f6b79
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The parameter names were previously "lhs" and "rhs", which is incredibly
abstract for a testing framework. One of the parameters will tend to be
a baseline value to compare against while the other is the result of
some action we want to test. Thus I suggest they be renamed "computed"
and "baseline".
This way we can, hopefully, retain the semantic that the 'left'/first
argument is the computed ('actual' in QCOMPARE) value while the
'right'/second argument is the baseline ('expected' in QCOMPARE.)
Change-Id: I3e0fdce2a3f1faca06fdf7184ef6e0eb9724d990
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
In the process actually handle all time-spec cases in various places
that only handled UTC or LocalTime, or at least note that they don't
where that's not practical. Also tidy up header ordering and ensure
QDateTime's header is included wherever it's used, while adding the
include for QTimeZone where needed.
Task-number: QTBUG-108199
Change-Id: Ic1a5cdf0aaf737bf1396aa8ac58ce2004cef7e19
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This is a semantic patch using ClangTidyTransformator as in
qtbase/df9d882d41b741fef7c5beeddb0abe9d904443d8, but extended to
handle typedefs and accesses through pointers, too:
const std::string o = "object";
auto hasTypeIgnoringPointer = [](auto type) { return anyOf(hasType(type), hasType(pointsTo(type))); };
auto derivedFromAnyOfClasses = [&](ArrayRef<StringRef> classes) {
auto exprOfDeclaredType = [&](auto decl) {
return expr(hasTypeIgnoringPointer(hasUnqualifiedDesugaredType(recordType(hasDeclaration(decl))))).bind(o);
};
return exprOfDeclaredType(cxxRecordDecl(isSameOrDerivedFrom(hasAnyName(classes))));
};
auto renameMethod = [&] (ArrayRef<StringRef> classes,
StringRef from, StringRef to) {
return makeRule(cxxMemberCallExpr(on(derivedFromAnyOfClasses(classes)),
callee(cxxMethodDecl(hasName(from), parameterCountIs(0)))),
changeTo(cat(access(o, cat(to)), "()")),
cat("use '", to, "' instead of '", from, "'"));
};
renameMethod(<classes>, "count", "size");
renameMethod(<classes>, "length", "size");
except that the on() matcher has been replaced by one that doesn't
ignoreParens().
a.k.a qt-port-to-std-compatible-api V5 with config Scope: 'Container'.
Added two NOLINTNEXTLINEs in tst_qbitarray and tst_qcontiguouscache,
to avoid porting calls that explicitly test count().
Change-Id: Icfb8808c2ff4a30187e9935a51cad26987451c22
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
The timestamp will no longer be incremented by 500ms after a mouse
release if the delay has been explicitly specified.
The default delay is 1 ms since f5010c49a3
but the running timestamp was unconditionally post-incremented by 500ms
after every mouse release, to prevent double-clicks, which were always
deemed as unintended (because we have a mouseDClick function for that).
Now, we do that 500ms increment only if the user has not provided a
delay value in the function argument at all. We have often found it
useful in our own tests to generate double-clicks "the hard way", by
sending indivdual events, so as to be able to check state in some target
object at each step, as shown in the new snippet.
[ChangeLog][QtTest] QTest::mouseRelease() and mouseClick() can now be
used to test double-clicks, by specifying a realistic timestamp delay.
Fixes: QTBUG-102441
Change-Id: I8e8d242061f79efb4c6e02638645e03661a9cd92
Reviewed-by: Richard Moe Gustavsen <richard.gustavsen@qt.io>
Unix systems have got crash loggers in the past 15-20 years, notably
macOS and Linux (abrtd, systemd-coredumpd, etc.). By setting the core
dump limit to zero, those tools should be mostly inhibited from running
and thus not interfere with the parent process' timeouts. Even for
systems without core dump loggers, disabling the writing of a core dump
to the filesystem should also help.
Pick-to: 6.4
Change-Id: I12a088d1ae424825abd3fffd171d112d0671effe
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
QWindow::requestActivate() is not supported.
We have one tst_selftests binary, and will test it with both xcb and
wayland qpa plugin. A runtime check and skip will have different
restult files, which is not implemented in testlib yet.
Task-number: QTBUG-107578
Pick-to: 6.4 6.2
Change-Id: Idc8cb24c6f42a9f0f4dc9493e3fd1a5803ba7ce0
Reviewed-by: Liang Qi <liang.qi@qt.io>
The QTRY_* macros and QTestEventLoop exit early if the test has
resolved; however, in the cleanup phase of a test, even if the test
has failed, these loops should continue as normal.
[ChangeLog][QtTest] During the cleanup() phase of a test, the QTRY_*
macros and QTestEventLoop now ignore the test resolution, in contrast
to when they are used from the test itself, which (since 6.3.0) exits
the loops early if the test has failed.
Pick-to: 6.4 6.3
Fixes: QTBUG-104441
Change-Id: I2673161967cbbc57815155af698a9338ab98a686
Reviewed-by: Jason McDonald <macadder1@gmail.com>
The regex used a greedy .+ to match an actual dot and a space,
preceding an open parenthesis, with the result that if the message
contained any parentheses that .+ swallowed everything up to the last
of them. Correct the regex so that recently-added tests' error
messages show up correctly.
Change-Id: I6e52c9b2a6e7959335fcddbb4266f65b589eba68
Reviewed-by: Jason McDonald <macadder1@gmail.com>
The recently added test for QTestEventLoop produced a message which
the TAP test-logger mis-parses. Since that message shall soon go away
(when I fix the bug the new test exists to verify), modify one of
cmptest's QVERIFY2() messages to trip up the same bug, so we can spot
any regressions on this in future (after the imminent fix to this TAP
issue).
Change-Id: Ibbe9931c01c75df529c9571e2bbdbd34010dd9ec
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Use the standard mechanism instead of a hand-rolled way of identifying
function/data-tag combinations. Adds missing data relevant to
identifying test cases, some of which previously had apparently the
same name, with no hint at why the test-case was seemingly run
repeatedly.
Change-Id: I6225c6d1990069c94a1f1c8dbb179993b96076e7
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
As discovered in the expanded testing of QTRY_COMPARE() using the same
class, the timer needs a context object and a slot to call.
This amends commit 35ad157d88
Pick-to: 6.4 6.3
Task-number: QTBUG-104441
Change-Id: I41fc23de84ce8c7d6608db0005276a2071974494
Reviewed-by: Jason McDonald <macadder1@gmail.com>
In the process, simplify the latter while adding some actual
time-variation for the QTRY_* loop to navigate round - based on the
extendedcompare test's ClassWithDeferredSetter. Testing remains
primitive, but is at least a bit more thorough.
Pick-to: 6.4
Change-Id: I40be8fb485f3f18f0a4f4bc62ad36cccac691979
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Jason McDonald <macadder1@gmail.com>
These contain output only ever sent to stderr, which tst_selftests no
longer looks at; it merely verifies stderr is empty for all tests
except those expected to produce stderr output, and ignores the stderr
output for these last.
Change-Id: I50fee445d84c2d125e6db5303fc389e87746455e
Reviewed-by: Jason McDonald <macadder1@gmail.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Fairly minimal for now, just enough to verify a bug and serve as the
sign of success when it's fixed. Tests fail in ways they shouldn't,
for now; see expected_eventloop.* for details, notably "Earlier test
failed to clean up" messages.
Pick-to: 6.4 6.3
Task-number: QTBUG-104441
Change-Id: I59be4aa5f21fed23b19a0593a8c2f6c9956507df
Reviewed-by: Jason McDonald <macadder1@gmail.com>
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.
Existing copyright statements remain intact
Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
The skipcleanup and failcleanup tests were actually testing skip and
fail in cleanupTestCase(), not in cleanup(). Add almost-duplicate
tests and clean up so that we now have {fail,skip}cleanup(,testcase}
tests to cover all four cases. Generated expected output. The new
tests (with old names) get their fail or skip - during cleanup() -
reported against the test instead of the cleanupTestCase function.
(Results for {init,cleanup}TestCase() are always reported, even when
these slots are not defined, as no-op passes.)
Pick-to: 6.4
Change-Id: I0988d1696b50c0e2f30c45ddc25e1bd0bfd2151a
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
For built-in types, this is a compile-time assert - we should not have
any types in Qt for which we have neither debug streaming nor a
QTest::toString specialization implemented. A build of most of Qt
submodules passes with this change, after minor modifications to some
tests. We cannot declare QSizeHint::Policy as a metatype after the
QMetaType has already been instantiated for it, and the QDebug stream
operator for QElaspedTimer needs to be correctly declared within the
namespace.
Add a self-test function for a custom type, and update reference files
of the self-test.
Task-number: QTBUG-104867
Pick-to: 6.4
Change-Id: I2936db5933f4589fce45f47cf2f3224ed614d8c9
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Before 0681a2dd5a, QCOMPARE'ing types
for which no QTest::toString specialization exists did not output
Actual and Expected lines on failure, as that would only print <null>
for both values (which then look like the same value, confusingly).
Commit 0681a2dd5a changed that behavior,
and started printing the confusing <null> values.
Take care of the logic in the formatFailMessage function: if both values
are nullptr, then print only the variable names, but not the confusing
<null> text representation of the values.
Remove dead and duplicated code related to the formatting logic, add a
self-test function, and update the expected_cmptest files.
Fixes: QTBUG-104867
Pick-to: 6.4
Change-Id: I4be98e79f91196b14690a2cc0a68ffd50b431a45
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Previously trying to execute a test function with an unknown data tag
would print an error message but exit with 0.
This patch stores a test failure, and continues trying to execute the
rest of the command line arguments, if any. In the end the process exits
with the usual exit code (number of failed tests) which is now !=0.
Pick-to: 6.4 6.3 6.2
Fixes: QTBUG-24240
Change-Id: Id4d422035f173e01e77ca88028dfd94dc0f9085c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The siginfo_t parameter allows us to show what process sent a signal or
the crashing address. Additionally, it allows us to determine if the
crashing signal was indeed sent due to a crash.
The selftest tst_crashes produces now:
$ QTEST_DISABLE_STACK_DUMP=1 ./crashes
********* Start testing of tst_Crashes *********
Config: Using QtTest library 6.4.0, Qt 6.4.0 (x86_64-little_endian-lp64 shared (dynamic) debug build; by GCC 11.2.1 20220420 [revision 691af15031e00227ba6d5935c1d737026cda4129]), opensuse-tumbleweed 20220428
PASS : tst_Crashes::initTestCase()
Received signal 11 (SIGSEGV), code 1, for address 0x0000000000000004
Function time: 0ms, total time: 0ms
[1] 201995 segmentation fault (core dumped) QTEST_DISABLE_STACK_DUMP=1 ./crashes
The last line comes from the shell. The code isn't decoded, but on Linux
it's a SEGV_MAPERR. macOS prints exactly the same thing.
I've updated one of the expected_crashes_*.txt output that doesn't seem
possible (the "Received a fatal error" message does not appear in Qt
anywhere).
Pick-to: 6.4
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16ebc8391234f0e2
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
It's easier to remember what "SIGSEGV" means instead of "11".
GNU libc has offered sigabbrev_np() (non-portable) since 2.32; for older
libcs, we'll be happy with a hardcoded list.
Selftest updated to match... though it didn't seem to be necessary.
Pick-to: 6.4
Change-Id: I5ff8e16fcdcb4ffd9ab6fffd16ebc66ecf6e9465
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Two patches applied to upstream release.
Pick-to: 6.4 6.3 6.2
Fixes: QTBUG-103732
Change-Id: Id64b65c4567433806047a2a34fa85ab5f260e6cc
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
[ChangeLog][QTestLib] QCOMPARE now evaluates toString() on its
arguments lazily, speeding up the general case where the comparison
doesn't fail. This is true for the QCOMPARE functionality provided
by Qt. If you specialized qCompare() for your own types, then you
need to change its implementation in line with Qt's own qCompare()
specializations in order to enable this feature.
[ChangeLog][QTestLib] QCOMPARE calls with nullptr argument(s) will
now print the actual and expected values upon failure.
Previously it was not like that because of the compareHelper()
overload in qtestresult.cpp that treated the presence of
nullptr-arguments as a reason to ignore formatFailMessage() call.
New implementation does not have this check, and correctly
executes formatFailMessage() for all arguments.
Note that the qCompare() overloads that call QTestResult::compare()
internally were not affected by this patch, because they already
defer toString() invocation until the comparison fails.
Some numbers, collected against shared release developer build.
I checked how this change affects the test execution. The idea was
to pick some tests for types that do not have a specific
QTestResult::compare overload, so I picked a couple of QByteArray
tests.
The comparison is done by running a test 10 times and taking the
average execution duration, as reported in the log.
tst_qbytearrayapisymmetry:
Before: 15.6 ms
After: 14.2 ms
tst_qbytearray:
Before: 41 ms
After: 36 ms
The benefit is around 9% and 12% respectively.
Fixes: QTBUG-98874
Change-Id: I7d59ddc760168b15974e7720930f629fb34efa13
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
[ChangeLog][QTestLib] Add QTRY_COMPARE_{EQ,NE,LT,LE,GT,GE}_WITH_TIMEOUT
macros that repeatedly execute QCOMPARE_{EQ,NE,LT,LE,GT,GE} until either
the comparison returns true or the timeout expires. Also add
QTRY_COMPARE_{EQ,NE,LT,LE,GT,GE} macros that simply invoke the
*_WITH_TIMEOUT versions with the usual timeout of five seconds.
Task-number: QTBUG-98873
Change-Id: Ib0d7d1c8c997f442b46acd85da738a8f512cc875
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
[ChangeLog][QTestLib] Add QCOMPARE_{EQ,NE,LT,LE,GT,GE}()
macros. These new macros behave similarly to QVERIFY(a op b),
where 'op' is ==, !=, <, <=, >, >= respectively, but print
a formatted error message with argument values in case of failure.
The formatting is done lazily, which means that the strings will
be generated only when the comparison fails.
Also add a new test for tst_selftest and generate expected output
for it.
Fixes: QTBUG-98873
Task-number: QTBUG-98874
Change-Id: Ic8074798901d7a469b1f58d5cd28bbf49a3da1db
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Add the Q_OS_WIN32 guard for the include of QWinEventNotifier.
Change-Id: I7824b2ee236a370c83fd85a2f594a39cf36b36e6
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
