qt5base-lts/tests/auto/testlib/selftests
Jason McDonald 64642a4d97 testlib: Skip test function if init() fails.
Prior to this commit, the following statement in the qtestlib
documentation was untrue: "If init() fails, the following testfunction
will not be executed, the test will proceed to the next testfunction."
If init() called QSKIP, the test function would be skipped, but if
init() reported a failure, the test function would still be executed
(even though doing so could be unsafe).

This commit makes testlib skip a test function if init() reports a
failure and enhances the selftests to cover skips and fails in both
init() and cleanup().

Task-number: QTBUG-20371
Change-Id: Id1cc8464ae0b8c257ae1b74dbe9189a501f5366b
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-17 05:24:46 +01:00
..
alive Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
assert Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
badxml Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
benchlibcallgrind Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
benchlibeventcounter Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
benchliboptions Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
benchlibtickcounter Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
benchlibwalltime Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
cmptest Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
commandlinedata Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
counting testlib: Skip test function if init() fails. 2012-02-17 05:24:46 +01:00
crashes Fixed tst_selftests on OSX 10.7 2012-02-09 03:31:08 +01:00
datatable Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
datetime Add data tag names to datetime selftest. 2012-02-09 04:07:45 +01:00
differentexec Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
exceptionthrow Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
expectfail testlib: Clear expected failures after every data row. 2012-02-08 03:13:21 +01:00
failcleanup testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
failinit Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
failinitdata Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
fetchbogus Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
findtestdata Fixed selftests unittest on windows 2012-02-02 05:42:07 +01:00
float Improve float selftest 2012-02-09 04:07:50 +01:00
globaldata Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
longstring Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
maxwarnings Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
multiexec Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
printdatatags Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
printdatatagswithglobaltags Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
qexecstringlist Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
singleskip Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
skip Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
skipcleanup testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
skipinit Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
skipinitdata Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
sleep Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
strcmp Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
subtest Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
test Changed selftests unittest to work from installation directory. 2012-01-11 04:09:26 +01:00
warnings testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
xunit Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
.gitignore Revert "Remove unused overload of QTest::qExec." 2011-12-12 10:43:44 +01:00
expected_alive.txt Add testlib selftest for QStringList comparison. 2011-12-14 02:40:17 +01:00
expected_assert.lightxml Fix XFAIL in testlib selftest 2011-09-17 03:21:44 +02:00
expected_assert.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_assert.xml Fix XFAIL in testlib selftest 2011-09-17 03:21:44 +02:00
expected_assert.xunitxml Fix XFAIL in testlib selftest 2011-09-17 03:21:44 +02:00
expected_badxml.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_badxml.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_badxml.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_badxml.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibcallgrind.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_benchlibeventcounter.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_benchlibeventcounter.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_benchlibeventcounter.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_benchlibeventcounter.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchliboptions.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_benchlibtickcounter.lightxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibtickcounter.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_benchlibtickcounter.xml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibtickcounter.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibwalltime.lightxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibwalltime.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_benchlibwalltime.xml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibwalltime.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_cmptest.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_cmptest.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_cmptest.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_cmptest.xunitxml QTestlib: Make QImage comparison more verbose. 2011-12-21 22:59:28 +01:00
expected_commandlinedata.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_commandlinedata.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_commandlinedata.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_commandlinedata.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_counting.lightxml testlib: Skip test function if init() fails. 2012-02-17 05:24:46 +01:00
expected_counting.txt testlib: Skip test function if init() fails. 2012-02-17 05:24:46 +01:00
expected_counting.xml testlib: Skip test function if init() fails. 2012-02-17 05:24:46 +01:00
expected_counting.xunitxml testlib: Skip test function if init() fails. 2012-02-17 05:24:46 +01:00
expected_crashes_1.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_crashes_2.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_crashes_3.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_datatable.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datatable.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datatable.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datatable.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_datetime.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datetime.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datetime.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datetime.xunitxml Add data tag names to datetime selftest. 2012-02-09 04:07:45 +01:00
expected_differentexec.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_exceptionthrow.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_exceptionthrow.txt Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_exceptionthrow.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_exceptionthrow.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_expectfail.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_expectfail.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_expectfail.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_expectfail.xunitxml testlib: Clear expected failures after every data row. 2012-02-08 03:13:21 +01:00
expected_failcleanup.lightxml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_failcleanup.txt testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_failcleanup.xml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_failcleanup.xunitxml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_failinit.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinit.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_failinit.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinit.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_failinitdata.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinitdata.txt Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinitdata.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinitdata.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_fatal.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_fetchbogus.lightxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_fetchbogus.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_fetchbogus.xml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_fetchbogus.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_findtestdata.lightxml testlib: add QFINDTESTDATA macro for finding testdata files 2011-12-01 09:12:57 +01:00
expected_findtestdata.txt testlib: add QFINDTESTDATA macro for finding testdata files 2011-12-01 09:12:57 +01:00
expected_findtestdata.xml testlib: add QFINDTESTDATA macro for finding testdata files 2011-12-01 09:12:57 +01:00
expected_findtestdata.xunitxml testlib: add QFINDTESTDATA macro for finding testdata files 2011-12-01 09:12:57 +01:00
expected_float.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_globaldata.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_globaldata.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_globaldata.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_globaldata.xunitxml Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_longstring.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_longstring.txt Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_longstring.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_longstring.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_maxwarnings.lightxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_maxwarnings.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_maxwarnings.xml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_maxwarnings.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_multiexec.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_printdatatags.txt Fixing naming of selftests. 2011-12-05 09:02:15 +01:00
expected_printdatatagswithglobaltags.txt Fixing naming of selftests. 2011-12-05 09:02:15 +01:00
expected_qexecstringlist.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_singleskip.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_singleskip.txt Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_singleskip.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_singleskip.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_skip.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_skip.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_skip.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_skip.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_skipcleanup.lightxml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_skipcleanup.txt testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_skipcleanup.xml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_skipcleanup.xunitxml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_skipinit.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_skipinit.txt Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_skipinit.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_skipinit.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_skipinitdata.lightxml Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_skipinitdata.txt Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_skipinitdata.xml Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_skipinitdata.xunitxml Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_sleep.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_strcmp.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_strcmp.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_strcmp.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_strcmp.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_subtest.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_subtest.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_subtest.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_subtest.xunitxml Split subtest selftest 2011-09-20 08:46:18 +02:00
expected_warnings.lightxml testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
expected_warnings.txt testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
expected_warnings.xml testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
expected_warnings.xunitxml testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
expected_xunit.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_xunit.txt Make QTest::qWarn() use __FILE__ and __LINE__. 2011-09-28 12:17:45 +02:00
expected_xunit.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_xunit.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
selftests.pri Make selftest.pri more readable/maintainable. 2012-02-09 04:07:53 +01:00
selftests.pro Changed selftests unittest to work from installation directory. 2012-01-11 04:09:26 +01:00
selftests.qrc Remove unused test results from testlib selftests. 2012-02-09 04:07:48 +01:00
tst_selftests.cpp CodeCoverage: Handle QTest based subtests. 2012-02-12 23:25:19 +01:00