TestMethods::invokeTest() has an outer loop on global data (albeit
with a comment that said otherwise). On its first cycle, we run the
test function's *_data() method, if it has one; there is an inner loop
on the rows this created. If the *_data() QSKIP()s, we need to skip
the whole test; otherwise, a QSKIP() in one sub-test should not lead
to skipping the remaining sub-tests.
Moved the check for *_data() QSKIP()ping to right after *_data()
returns, inside the "first global cycle" block that runs it.
Previously, this check was done before entering the loop on local data
rows, but outside that "first global cycle" block: consequently, later
global cycles would fall foul of this check (even though the *_data()
hasn't been run in this cycle, much less QSKIP()ped in it) if the last
sub-test of the previous global cycle had QSKIP()ped.
When running a single test for one specific data row, if the test's
*_data() QSKIP()ped, this misplaced check would also have lead to a
misleading "Unknown testdata" warning.
Changed testlib/selftests' tst_globaldata::skipSingle() to trigger the
bug (by having its last local row of first global row skip, which
caused the second global row to be omitted) to verify this is also
fixed; and amended one of its comments to reflect what's now to be
expected. Updated the test's expected output files.
Task-number: QTBUG-61774
Change-Id: I99596b595c6d1184038f23383844c6ff51a0cd91
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
In particular, document the mis-behavior it takes for granted in one
test. Renamed some variables so that they're visually distinct from
one another and actually mean something intelligible. Renamed the
sub-tests to say what's actually happening. Reading the test output
now at least makes it possible to see what's wrong. I'll fix that in
a later commit.
Task-number: QTBUG-61774
Change-Id: I3e5f83b8baa3c6afbca9231b5bbc89d17f3e57e2
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
It's unexpected that all messages generated by the stream version
of qDebug and friends have a trailing space. It also makes switching
to categorized logging (which only supports the stream version) difficult,
since all autotests checking for debug output would have to be adapted.
Task-number: QTBUG-15256
Change-Id: I8d627a8379dc273d9689f5611184f03607b73823
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
When the SkipAll mode is used, tests only report a SKIP for the first
line of test data and subsequent lines are not reported at all. This
behaviour makes it impossible for anything post-processing test results
to accurately report test pass- and run- rates because they cannot see
how many lines of test data were skipped.
This commit removes SkipMode. QSKIPs in regular test functions and data
functions are treated the same as SkipSingle, so that every skipped line
of local or global test data is reported in the test log. QSKIPs
elsewhere are treated the same as SkipAll -- skipping in init() causes
the next test function to be skipped entirely, and skipping in
initTestCase() or initTestCase_data() causes all test functions to be
skipped.
This commit only changes qtestlib and the selftests. A further commit
will change the autotests to remove the SkipMode parameter from QSKIP
calls.
Note that the change in expected output for the globaldata selftest is
deliberate, as the QSKIP in the skipLocal test function has effectively
changed from SkipAll to SkipSingle.
Task-number: QTBUG-21851, QTBUG-21652
Change-Id: I7b1c53fe7ca9dde032810b789d967e2a402bbe5d
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Alex <alex.blasche@nokia.com>
Make the test use qDebug instead of printf, so that the reference test
output is the same regardless of whether the log goes to the console or
a file.
Also rename the test class, as it was evidently cut-n-pasted from the
subtest selftest without being renamed.
Change-Id: I8df1f0989f2e8c33c1f829ba8110fd740ff70f63
Reviewed-on: http://codereview.qt-project.org/5184
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
