qt5base-lts/tests/auto/testlib/selftests
Jason McDonald 8854b74cf4 selftests: Improve reporting of problems with loading expected output
If the expected output file was missing (e.g. not included in
selftests.qrc), tst_selftests would trigger an assert inside QList by
calling QList::at() on an empty list.  Make tst_selftests detect this
error instead and give a meaningful error message.

When loading expected output for the crashes selftest, where there are
several alternative versions of the expected output, the code reused the
"exp" variable when loading the alternative test output files.  This
caused the last file loaded to be used unintentionally if none of the
alternative files had the correct number of lines.  Use a different
variable so that exp remains empty if none of the alternatives are
valid and a failure can be reported.

Change-Id: I35b2a3d905d069d3ee8dcb1447836eb68d5c8612
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-03-08 05:31:40 +01:00
..
alive QEvent (and subclasses): make ctors explicit 2012-03-01 08:44:34 +01:00
assert Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
badxml Port badxml autotest to QMetaObjectBuilder 2012-02-21 13:57:12 +01:00
benchlibcallgrind Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
benchlibcounting testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
benchlibeventcounter Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
benchliboptions Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
benchlibtickcounter Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
benchlibwalltime Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
cmptest Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
commandlinedata Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
counting testlib: Make verbose2 selftest use QCOMPARE as well as QVERIFY. 2012-03-05 05:32:03 +01:00
crashes Fixed tst_selftests on OSX 10.7 2012-02-09 03:31:08 +01:00
datatable Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
datetime Add data tag names to datetime selftest. 2012-02-09 04:07:45 +01:00
differentexec Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
exceptionthrow Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
expectfail testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. 2012-02-20 03:27:02 +01:00
failcleanup testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
failinit Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
failinitdata Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
fetchbogus Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
findtestdata Fixed selftests unittest on windows 2012-02-02 05:42:07 +01:00
float Improve float selftest 2012-02-09 04:07:50 +01:00
globaldata Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
longstring Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
maxwarnings Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
multiexec Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
printdatatags Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
printdatatagswithglobaltags Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
qexecstringlist Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
silent testlib: Improve the silent logging mode 2012-02-28 05:56:03 +01:00
singleskip Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
skip Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
skipcleanup testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
skipinit Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
skipinitdata Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
sleep Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
strcmp Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
subtest Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
test Changed selftests unittest to work from installation directory. 2012-01-11 04:09:26 +01:00
verbose1 testlib: Add selftests for -v1 and -v2 command-line switches. 2012-03-02 01:06:22 +01:00
verbose2 testlib: Add selftests for -v1 and -v2 command-line switches. 2012-03-02 01:06:22 +01:00
warnings testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
xunit Remove "All rights reserved" line from license headers. 2012-01-30 03:54:59 +01:00
.gitignore Revert "Remove unused overload of QTest::qExec." 2011-12-12 10:43:44 +01:00
expected_alive.txt Add testlib selftest for QStringList comparison. 2011-12-14 02:40:17 +01:00
expected_assert.lightxml Fix XFAIL in testlib selftest 2011-09-17 03:21:44 +02:00
expected_assert.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_assert.xml Fix XFAIL in testlib selftest 2011-09-17 03:21:44 +02:00
expected_assert.xunitxml Fix XFAIL in testlib selftest 2011-09-17 03:21:44 +02:00
expected_badxml.lightxml testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_badxml.txt testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_badxml.xml testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_badxml.xunitxml testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_benchlibcallgrind.txt testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_benchlibcounting.lightxml testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_benchlibcounting.txt testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_benchlibcounting.xml testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_benchlibcounting.xunitxml testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_benchlibeventcounter.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_benchlibeventcounter.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_benchlibeventcounter.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_benchlibeventcounter.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchliboptions.txt testlib: Report one test result per benchmark test. 2012-02-20 08:05:58 +01:00
expected_benchlibtickcounter.lightxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibtickcounter.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_benchlibtickcounter.xml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibtickcounter.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibwalltime.lightxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibwalltime.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_benchlibwalltime.xml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_benchlibwalltime.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_cmptest.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_cmptest.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_cmptest.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_cmptest.xunitxml QTestlib: Make QImage comparison more verbose. 2011-12-21 22:59:28 +01:00
expected_commandlinedata.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_commandlinedata.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_commandlinedata.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_commandlinedata.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_counting.lightxml testlib: Make verbose2 selftest use QCOMPARE as well as QVERIFY. 2012-03-05 05:32:03 +01:00
expected_counting.txt testlib: Make verbose2 selftest use QCOMPARE as well as QVERIFY. 2012-03-05 05:32:03 +01:00
expected_counting.xml testlib: Make verbose2 selftest use QCOMPARE as well as QVERIFY. 2012-03-05 05:32:03 +01:00
expected_counting.xunitxml testlib: Skip test function if init() fails. 2012-02-17 05:24:46 +01:00
expected_crashes_1.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_crashes_2.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_crashes_3.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_datatable.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datatable.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datatable.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datatable.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_datetime.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datetime.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datetime.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_datetime.xunitxml Add data tag names to datetime selftest. 2012-02-09 04:07:45 +01:00
expected_differentexec.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_exceptionthrow.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_exceptionthrow.txt Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_exceptionthrow.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_exceptionthrow.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_expectfail.lightxml testlib: Improve verbose and XPASS output 2012-03-08 05:31:40 +01:00
expected_expectfail.txt testlib: Improve verbose and XPASS output 2012-03-08 05:31:40 +01:00
expected_expectfail.xml testlib: Improve verbose and XPASS output 2012-03-08 05:31:40 +01:00
expected_expectfail.xunitxml testlib: Improve verbose and XPASS output 2012-03-08 05:31:40 +01:00
expected_failcleanup.lightxml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_failcleanup.txt testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_failcleanup.xml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_failcleanup.xunitxml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_failinit.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinit.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_failinit.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinit.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_failinitdata.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinitdata.txt Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinitdata.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_failinitdata.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_fetchbogus.lightxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_fetchbogus.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_fetchbogus.xml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_fetchbogus.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_findtestdata.lightxml testlib: add QFINDTESTDATA macro for finding testdata files 2011-12-01 09:12:57 +01:00
expected_findtestdata.txt testlib: add QFINDTESTDATA macro for finding testdata files 2011-12-01 09:12:57 +01:00
expected_findtestdata.xml testlib: add QFINDTESTDATA macro for finding testdata files 2011-12-01 09:12:57 +01:00
expected_findtestdata.xunitxml testlib: add QFINDTESTDATA macro for finding testdata files 2011-12-01 09:12:57 +01:00
expected_float.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_globaldata.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_globaldata.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_globaldata.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_globaldata.xunitxml Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_longstring.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_longstring.txt Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_longstring.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_longstring.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_maxwarnings.lightxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_maxwarnings.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_maxwarnings.xml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_maxwarnings.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_multiexec.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_printdatatags.txt Fixing naming of selftests. 2011-12-05 09:02:15 +01:00
expected_printdatatagswithglobaltags.txt Fixing naming of selftests. 2011-12-05 09:02:15 +01:00
expected_qexecstringlist.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_silent.txt testlib: Improve the silent logging mode 2012-02-28 05:56:03 +01:00
expected_singleskip.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_singleskip.txt Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_singleskip.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_singleskip.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_skip.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_skip.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_skip.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_skip.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_skipcleanup.lightxml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_skipcleanup.txt testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_skipcleanup.xml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_skipcleanup.xunitxml testlib: Add selftests for skip and fail inside cleanupTestCase(). 2012-02-09 04:07:33 +01:00
expected_skipinit.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_skipinit.txt Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_skipinit.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_skipinit.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_skipinitdata.lightxml Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_skipinitdata.txt Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_skipinitdata.xml Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_skipinitdata.xunitxml Remove QTest::SkipMode from qtestlib API. 2011-10-21 01:20:29 +02:00
expected_sleep.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_strcmp.lightxml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_strcmp.txt Use placeholder for Qt version in expected output 2011-09-26 07:50:20 +02:00
expected_strcmp.xml Sanitize paths in selftest data. 2011-10-19 07:24:40 +02:00
expected_strcmp.xunitxml Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00
expected_subtest.lightxml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_subtest.txt testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_subtest.xml testlib: Count passes, fails and skips consistently. 2012-02-13 01:46:09 +01:00
expected_subtest.xunitxml Split subtest selftest 2011-09-20 08:46:18 +02:00
expected_verbose1.lightxml testlib: Make verbose2 selftest use QCOMPARE as well as QVERIFY. 2012-03-05 05:32:03 +01:00
expected_verbose1.txt testlib: Make verbose2 selftest use QCOMPARE as well as QVERIFY. 2012-03-05 05:32:03 +01:00
expected_verbose1.xml testlib: Make verbose2 selftest use QCOMPARE as well as QVERIFY. 2012-03-05 05:32:03 +01:00
expected_verbose1.xunitxml testlib: Add selftests for -v1 and -v2 command-line switches. 2012-03-02 01:06:22 +01:00
expected_verbose2.lightxml testlib: Improve verbose and XPASS output 2012-03-08 05:31:40 +01:00
expected_verbose2.txt testlib: Improve verbose and XPASS output 2012-03-08 05:31:40 +01:00
expected_verbose2.xml testlib: Improve verbose and XPASS output 2012-03-08 05:31:40 +01:00
expected_verbose2.xunitxml testlib: Improve verbose and XPASS output 2012-03-08 05:31:40 +01:00
expected_warnings.lightxml testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
expected_warnings.txt testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
expected_warnings.xml testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
expected_warnings.xunitxml testlib: Clear ignored messages after every data row 2012-02-08 03:13:23 +01:00
expected_xunit.lightxml testlib: make XPASS output slightly less confusing. 2012-02-20 03:27:02 +01:00
expected_xunit.txt testlib: make XPASS output slightly less confusing. 2012-02-20 03:27:02 +01:00
expected_xunit.xml testlib: make XPASS output slightly less confusing. 2012-02-20 03:27:02 +01:00
expected_xunit.xunitxml testlib: make XPASS output slightly less confusing. 2012-02-20 03:27:02 +01:00
selftests.pri testlib: Add selftests for -v1 and -v2 command-line switches. 2012-03-02 01:06:22 +01:00
selftests.pro Changed selftests unittest to work from installation directory. 2012-01-11 04:09:26 +01:00
selftests.qrc testlib: Add selftests for -v1 and -v2 command-line switches. 2012-03-02 01:06:22 +01:00
tst_selftests.cpp selftests: Improve reporting of problems with loading expected output 2012-03-08 05:31:40 +01:00