AuroraMiddleware/qt5base-lts
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
da18ada6bf
qt5base-lts/tests/auto/testlib/selftests/expected_counting.lightxml

79 lines
3.6 KiB
Plaintext
Raw Normal View History

Add testlib selftest for counting of test results. The test simply generates each possible pair of Pass, Fail and Skip results. At present the test simply serves to demonstrate the current shortcomings of testlib's plain text logging, namely: * If a test function passes for all data rows, that is counted as one pass, but each skipped or failed row counts as one skip or fail. * Only skipped and failed rows are reported individually in the test output. Passed rows are not reported, so it is impossible to see how many rows were executed. * A skip followed by a pass will be reported as an overall pass for the test function, but the same rows in reverse order will not report any overall result for the test function. Future commits will attempt to correct these problems. Task-number: QTBUG-22124 Change-Id: If8c7ea15fc43ba9a1bccd0e881c1efc18e705b25 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-31 04:07:58 +00:00
<Environment>
<QtVersion>@INSERT_QT_VERSION_HERE@</QtVersion>
<QTestVersion>@INSERT_QT_VERSION_HERE@</QTestVersion>
</Environment>
<TestFunction name="initTestCase">
<Incident type="pass" file="" line="0" />
</TestFunction>
<TestFunction name="testPassPass">
<Incident type="pass" file="" line="0" />
</TestFunction>
<TestFunction name="testPassSkip">
<Message type="skip" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="102">
<DataTag><![CDATA[row 2]]></DataTag>
<Description><![CDATA[Skipping]]></Description>
</Message>
</TestFunction>
<TestFunction name="testPassFail">
<Incident type="fail" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="99">
<DataTag><![CDATA[row 2]]></DataTag>
<Description><![CDATA['false' returned FALSE. ()]]></Description>
</Incident>
</TestFunction>
<TestFunction name="testSkipPass">
<Message type="skip" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="102">
<DataTag><![CDATA[row 1]]></DataTag>
<Description><![CDATA[Skipping]]></Description>
</Message>
<Incident type="pass" file="" line="0" />
</TestFunction>
<TestFunction name="testSkipSkip">
<Message type="skip" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="102">
<DataTag><![CDATA[row 1]]></DataTag>
<Description><![CDATA[Skipping]]></Description>
</Message>
<Message type="skip" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="102">
<DataTag><![CDATA[row 2]]></DataTag>
<Description><![CDATA[Skipping]]></Description>
</Message>
</TestFunction>
<TestFunction name="testSkipFail">
<Message type="skip" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="102">
<DataTag><![CDATA[row 1]]></DataTag>
<Description><![CDATA[Skipping]]></Description>
</Message>
<Incident type="fail" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="99">
<DataTag><![CDATA[row 2]]></DataTag>
<Description><![CDATA['false' returned FALSE. ()]]></Description>
</Incident>
</TestFunction>
<TestFunction name="testFailPass">
<Incident type="fail" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="99">
<DataTag><![CDATA[row 1]]></DataTag>
<Description><![CDATA['false' returned FALSE. ()]]></Description>
</Incident>
</TestFunction>
<TestFunction name="testFailSkip">
<Incident type="fail" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="99">
<DataTag><![CDATA[row 1]]></DataTag>
<Description><![CDATA['false' returned FALSE. ()]]></Description>
</Incident>
<Message type="skip" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="102">
<DataTag><![CDATA[row 2]]></DataTag>
<Description><![CDATA[Skipping]]></Description>
</Message>
</TestFunction>
<TestFunction name="testFailFail">
<Incident type="fail" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="99">
<DataTag><![CDATA[row 1]]></DataTag>
<Description><![CDATA['false' returned FALSE. ()]]></Description>
</Incident>
<Incident type="fail" file="/home/user/dev/qt5/qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp" line="99">
<DataTag><![CDATA[row 2]]></DataTag>
<Description><![CDATA['false' returned FALSE. ()]]></Description>
</Incident>
</TestFunction>
<TestFunction name="cleanupTestCase">
<Incident type="pass" file="" line="0" />
</TestFunction>
Reference in New Issue Copy Permalink