AuroraMiddleware/qt5base-lts
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
a7484002a3
qt5base-lts/tests/auto/testlib/selftests/expected_expectfail.lightxml

160 lines
6.3 KiB
Plaintext
Raw Normal View History

Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<Environment>
<QtVersion>@INSERT_QT_VERSION_HERE@</QtVersion>
Use new QLibraryInfo::build() in testlib to log build information. This produces: ********* Start testing of tst_QtJson ********* Config: Using QtTest library 5.3.0, Qt 5.3.0 (Feb 13 2014, GCC 4.6.3, 64 bit, debug build) PASS : tst_QtJson::initTestCase() <?xml version="1.0" encoding="UTF-8"?> <TestCase name="tst_QtJson"> <Environment> <QtVersion>5.3.0</QtVersion> <QtBuild>Qt 5.3.0 (Feb 13 2014, GCC 4.6.3, 64 bit, debug build)</QtBuild> <QTestVersion>5.3.0</QTestVersion> </Environment> <?xml version="1.0" encoding="UTF-8" ?> <testsuite errors="1" failures="1" tests="42" name="tst_QtJson"> <properties> <property value="5.3.0" name="QTestVersion"/> <property value="5.3.0" name="QtVersion"/> <property value="Qt 5.3.0 (Feb 13 2014, GCC 4.6.3, 64 bit, debug build)" name="QtBuild"/> </properties> <Environment> <QtVersion>5.3.0</QtVersion> <QtBuild>Qt 5.3.0 (Feb 13 2014, GCC 4.6.3, 64 bit, debug build)</QtBuild> <QTestVersion>5.3.0</QTestVersion> </Environment> [ChangeLog][QtTest] Tests now output build information. Change-Id: I0ab473371575f2b807db725256805b8bffea3454 Reviewed-by: Sergio Ahumada <sahumada@blackberry.com> Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@digia.com>
2014-02-26 16:12:45 +00:00
<QtBuild/>
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<QTestVersion>@INSERT_QT_VERSION_HERE@</QTestVersion>
</Environment>
<TestFunction name="initTestCase">
Make lightxml and xml log formatting consistent For XML logs, an Incident tag with no sub-tags has a space before the closing slash. For Light XML logs, this space was missing unless the -flush command line option was supplied. Change-Id: I6de2e140f9309f333b5b97ed7f8116e4e727a149 Reviewed-on: http://codereview.qt.nokia.com/3871 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-08-30 09:54:32 +00:00
<Incident type="pass" file="" line="0" />
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
</TestFunction>
Improve readability of expectfail selftest. Rename some of the test functions to improve consistency, and change the order so that all the xfail tests are together (some new xpass tests will be added later). Change-Id: Ice1ac5bf61bc26109bb0d08f72eb74e1a0424101 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:33:42 +00:00
<TestFunction name="xfailAndContinue">
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<Message type="qdebug" file="" line="0">
<Description><![CDATA[begin]]></Description>
</Message>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<Description><![CDATA[This should xfail]]></Description>
Fix errors in lightxml test logger The newer test logger did not produce the same light XML output as the logger that it replaced. In particular, it did not output the <DataTag> tag and it incorrectly nested a <Message> tag inside the <Incident> tag when a fatal error occured in a test. Unfortunately, it appears that the expected lightxml output for the selftests was produced by running tests using the newer logger, while the selftests did not use the older lightxml logger. Thus the errors were not detected by the selftests. This commit adds the older lightxml logger to the selftests, updates the expected test data accordingly, and modifies the newer lightxml logger to behave correctly. This last item is achieved by making the lightxml streamer copy most of the code from the xml streamer -- lightxml output is supposed to be same as xml, except for the omission of the root and <TestCase> tags. Change-Id: Ie6e1f69dd6000df2b9d0c5c8e2109fe7bbff3956 Reviewed-on: http://codereview.qt.nokia.com/3902 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-08-31 02:52:27 +00:00
</Incident>
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<Message type="qdebug" file="" line="0">
<Description><![CDATA[after]]></Description>
</Message>
Fix errors in lightxml test logger The newer test logger did not produce the same light XML output as the logger that it replaced. In particular, it did not output the <DataTag> tag and it incorrectly nested a <Message> tag inside the <Incident> tag when a fatal error occured in a test. Unfortunately, it appears that the expected lightxml output for the selftests was produced by running tests using the newer logger, while the selftests did not use the older lightxml logger. Thus the errors were not detected by the selftests. This commit adds the older lightxml logger to the selftests, updates the expected test data accordingly, and modifies the newer lightxml logger to behave correctly. This last item is achieved by making the lightxml streamer copy most of the code from the xml streamer -- lightxml output is supposed to be same as xml, except for the omission of the root and <TestCase> tags. Change-Id: Ie6e1f69dd6000df2b9d0c5c8e2109fe7bbff3956 Reviewed-on: http://codereview.qt.nokia.com/3902 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-08-31 02:52:27 +00:00
<Incident type="pass" file="" line="0" />
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
</TestFunction>
Improve readability of expectfail selftest. Rename some of the test functions to improve consistency, and change the order so that all the xfail tests are together (some new xpass tests will be added later). Change-Id: Ice1ac5bf61bc26109bb0d08f72eb74e1a0424101 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:33:42 +00:00
<TestFunction name="xfailAndAbort">
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<Message type="qdebug" file="" line="0">
<Description><![CDATA[begin]]></Description>
</Message>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<Description><![CDATA[This should xfail]]></Description>
</Incident>
Fix errors in lightxml test logger The newer test logger did not produce the same light XML output as the logger that it replaced. In particular, it did not output the <DataTag> tag and it incorrectly nested a <Message> tag inside the <Incident> tag when a fatal error occured in a test. Unfortunately, it appears that the expected lightxml output for the selftests was produced by running tests using the newer logger, while the selftests did not use the older lightxml logger. Thus the errors were not detected by the selftests. This commit adds the older lightxml logger to the selftests, updates the expected test data accordingly, and modifies the newer lightxml logger to behave correctly. This last item is achieved by making the lightxml streamer copy most of the code from the xml streamer -- lightxml output is supposed to be same as xml, except for the omission of the root and <TestCase> tags. Change-Id: Ie6e1f69dd6000df2b9d0c5c8e2109fe7bbff3956 Reviewed-on: http://codereview.qt.nokia.com/3902 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-08-31 02:52:27 +00:00
<Incident type="pass" file="" line="0" />
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
</TestFunction>
Improve readability of expectfail selftest. Rename some of the test functions to improve consistency, and change the order so that all the xfail tests are together (some new xpass tests will be added later). Change-Id: Ice1ac5bf61bc26109bb0d08f72eb74e1a0424101 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:33:42 +00:00
<TestFunction name="xfailTwice">
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="fail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Improve coverage of expectfail selftest. Add a test for the case where QEXPECT_FAIL is called twice in a row without an intervening verification step, which is supposed to result in a failure. Change-Id: Ie004652a76a2c35c2679945670ff5561e8f5b62e Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-12 03:16:44 +00:00
<Description><![CDATA[Already expecting a fail]]></Description>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Improve coverage of expectfail selftest. Add a test for the case where QEXPECT_FAIL is called twice in a row without an intervening verification step, which is supposed to result in a failure. Change-Id: Ie004652a76a2c35c2679945670ff5561e8f5b62e Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-12 03:16:44 +00:00
</TestFunction>
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<TestFunction name="xfailWithQString">
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<Description><![CDATA[A string]]></Description>
Fix errors in lightxml test logger The newer test logger did not produce the same light XML output as the logger that it replaced. In particular, it did not output the <DataTag> tag and it incorrectly nested a <Message> tag inside the <Incident> tag when a fatal error occured in a test. Unfortunately, it appears that the expected lightxml output for the selftests was produced by running tests using the newer logger, while the selftests did not use the older lightxml logger. Thus the errors were not detected by the selftests. This commit adds the older lightxml logger to the selftests, updates the expected test data accordingly, and modifies the newer lightxml logger to behave correctly. This last item is achieved by making the lightxml streamer copy most of the code from the xml streamer -- lightxml output is supposed to be same as xml, except for the omission of the root and <TestCase> tags. Change-Id: Ie6e1f69dd6000df2b9d0c5c8e2109fe7bbff3956 Reviewed-on: http://codereview.qt.nokia.com/3902 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-08-31 02:52:27 +00:00
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<Description><![CDATA[Bug 5 (The message)]]></Description>
</Incident>
Fix errors in lightxml test logger The newer test logger did not produce the same light XML output as the logger that it replaced. In particular, it did not output the <DataTag> tag and it incorrectly nested a <Message> tag inside the <Incident> tag when a fatal error occured in a test. Unfortunately, it appears that the expected lightxml output for the selftests was produced by running tests using the newer logger, while the selftests did not use the older lightxml logger. Thus the errors were not detected by the selftests. This commit adds the older lightxml logger to the selftests, updates the expected test data accordingly, and modifies the newer lightxml logger to behave correctly. This last item is achieved by making the lightxml streamer copy most of the code from the xml streamer -- lightxml output is supposed to be same as xml, except for the omission of the root and <TestCase> tags. Change-Id: Ie6e1f69dd6000df2b9d0c5c8e2109fe7bbff3956 Reviewed-on: http://codereview.qt.nokia.com/3902 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-08-31 02:52:27 +00:00
<Incident type="pass" file="" line="0" />
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
</TestFunction>
testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-20 00:15:57 +00:00
<TestFunction name="xfailDataDrivenWithQVerify">
testlib: Count passes, fails and skips consistently. For data-driven tests, testlib previously counted one fail or skip for each data row that failed or skipped, while it counted only one pass for a test function where all rows passed and counted no passes for a test function where some rows passed and some rows failed. A similar problem also existed for benchmark tests, which could run multiple iterations of the same test, with each fail and skip being counted but only a single pass being counted for the entire series of iterations. This commit makes testlib count one pass, fail or skip for each data row. Test functions that are not data-driven count one result for the test function, as before. Benchmark tests count one pass, fail or skip per iteration. A side-effect of this change is that the test output in plain text, xml and light xml formats now shows a result for every data row and benchmark iteration executed, allowing post-processors to correctly calculate the total number of tests executed. Previously, individual rows were not shown in the test output if they passed, making such calculations impossible. The only change to the xunitxml output format is to correct a bug where no test result was recorded for a test function if the last data row was skipped and all other rows passed -- in which case the overall result should be a pass. Note that there is also a pre-existing bug in the xunit logger, where no result is reported if all rows are skipped; that bug is unaffected by this commit. Task-number: QTBUG-21848 Task-number: QTBUG-22124 Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b Reviewed-by: Ed Baak <ed.baak@nokia.com>
2012-02-02 07:08:44 +00:00
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Pass 1]]></DataTag>
</Incident>
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Pass 2]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Improve selftest coverage of QEXPECT_FAIL feature. The existing expectfail selftest did not test QEXPECT_FAIL with a data-driven test function. This commit adds such a test. Change-Id: I39fa9aa227b58779ce5268dd37bf55468e7269c5 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-11 06:18:55 +00:00
<DataTag><![CDATA[Abort]]></DataTag>
<Description><![CDATA[This test should xfail]]></Description>
</Incident>
testlib: Count passes, fails and skips consistently. For data-driven tests, testlib previously counted one fail or skip for each data row that failed or skipped, while it counted only one pass for a test function where all rows passed and counted no passes for a test function where some rows passed and some rows failed. A similar problem also existed for benchmark tests, which could run multiple iterations of the same test, with each fail and skip being counted but only a single pass being counted for the entire series of iterations. This commit makes testlib count one pass, fail or skip for each data row. Test functions that are not data-driven count one result for the test function, as before. Benchmark tests count one pass, fail or skip per iteration. A side-effect of this change is that the test output in plain text, xml and light xml formats now shows a result for every data row and benchmark iteration executed, allowing post-processors to correctly calculate the total number of tests executed. Previously, individual rows were not shown in the test output if they passed, making such calculations impossible. The only change to the xunitxml output format is to correct a bug where no test result was recorded for a test function if the last data row was skipped and all other rows passed -- in which case the overall result should be a pass. Note that there is also a pre-existing bug in the xunit logger, where no result is reported if all rows are skipped; that bug is unaffected by this commit. Task-number: QTBUG-21848 Task-number: QTBUG-22124 Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b Reviewed-by: Ed Baak <ed.baak@nokia.com>
2012-02-02 07:08:44 +00:00
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Abort]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-20 00:15:57 +00:00
<DataTag><![CDATA[Continue]]></DataTag>
<Description><![CDATA[This test should xfail]]></Description>
</Incident>
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Continue]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-20 00:15:57 +00:00
</TestFunction>
<TestFunction name="xfailDataDrivenWithQCompare">
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Pass 1]]></DataTag>
</Incident>
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Pass 2]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-20 00:15:57 +00:00
<DataTag><![CDATA[Abort]]></DataTag>
<Description><![CDATA[This test should xfail]]></Description>
</Incident>
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Abort]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Improve selftest coverage of QEXPECT_FAIL feature. The existing expectfail selftest did not test QEXPECT_FAIL with a data-driven test function. This commit adds such a test. Change-Id: I39fa9aa227b58779ce5268dd37bf55468e7269c5 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-11 06:18:55 +00:00
<DataTag><![CDATA[Continue]]></DataTag>
<Description><![CDATA[This test should xfail]]></Description>
</Incident>
testlib: Count passes, fails and skips consistently. For data-driven tests, testlib previously counted one fail or skip for each data row that failed or skipped, while it counted only one pass for a test function where all rows passed and counted no passes for a test function where some rows passed and some rows failed. A similar problem also existed for benchmark tests, which could run multiple iterations of the same test, with each fail and skip being counted but only a single pass being counted for the entire series of iterations. This commit makes testlib count one pass, fail or skip for each data row. Test functions that are not data-driven count one result for the test function, as before. Benchmark tests count one pass, fail or skip per iteration. A side-effect of this change is that the test output in plain text, xml and light xml formats now shows a result for every data row and benchmark iteration executed, allowing post-processors to correctly calculate the total number of tests executed. Previously, individual rows were not shown in the test output if they passed, making such calculations impossible. The only change to the xunitxml output format is to correct a bug where no test result was recorded for a test function if the last data row was skipped and all other rows passed -- in which case the overall result should be a pass. Note that there is also a pre-existing bug in the xunit logger, where no result is reported if all rows are skipped; that bug is unaffected by this commit. Task-number: QTBUG-21848 Task-number: QTBUG-22124 Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b Reviewed-by: Ed Baak <ed.baak@nokia.com>
2012-02-02 07:08:44 +00:00
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Continue]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Improve selftest coverage of QEXPECT_FAIL feature. The existing expectfail selftest did not test QEXPECT_FAIL with a data-driven test function. This commit adds such a test. Change-Id: I39fa9aa227b58779ce5268dd37bf55468e7269c5 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-11 06:18:55 +00:00
</TestFunction>
Improve readability of expectfail selftest. Rename some of the test functions to improve consistency, and change the order so that all the xfail tests are together (some new xpass tests will be added later). Change-Id: Ice1ac5bf61bc26109bb0d08f72eb74e1a0424101 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:33:42 +00:00
<TestFunction name="xfailOnWrongRow">
testlib: Count passes, fails and skips consistently. For data-driven tests, testlib previously counted one fail or skip for each data row that failed or skipped, while it counted only one pass for a test function where all rows passed and counted no passes for a test function where some rows passed and some rows failed. A similar problem also existed for benchmark tests, which could run multiple iterations of the same test, with each fail and skip being counted but only a single pass being counted for the entire series of iterations. This commit makes testlib count one pass, fail or skip for each data row. Test functions that are not data-driven count one result for the test function, as before. Benchmark tests count one pass, fail or skip per iteration. A side-effect of this change is that the test output in plain text, xml and light xml formats now shows a result for every data row and benchmark iteration executed, allowing post-processors to correctly calculate the total number of tests executed. Previously, individual rows were not shown in the test output if they passed, making such calculations impossible. The only change to the xunitxml output format is to correct a bug where no test result was recorded for a test function if the last data row was skipped and all other rows passed -- in which case the overall result should be a pass. Note that there is also a pre-existing bug in the xunit logger, where no result is reported if all rows are skipped; that bug is unaffected by this commit. Task-number: QTBUG-21848 Task-number: QTBUG-22124 Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b Reviewed-by: Ed Baak <ed.baak@nokia.com>
2012-02-02 07:08:44 +00:00
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[right row]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Improve coverage of expectfail selftest Add a test for the case where a call to QEXPECT_FAIL nominates a different data row from the current row, in which case the call to QEXPECT_FAIL should be ignored. Change-Id: Iff774b89f9374fa26105acdd8769177272a95229 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-12 03:47:25 +00:00
</TestFunction>
Improve readability of expectfail selftest. Rename some of the test functions to improve consistency, and change the order so that all the xfail tests are together (some new xpass tests will be added later). Change-Id: Ice1ac5bf61bc26109bb0d08f72eb74e1a0424101 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:33:42 +00:00
<TestFunction name="xfailOnAnyRow">
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Improve coverage of expectfail selftest Add a test case for calling QEXPECT_FAIL in a data-driven test without passing a specific data tag for the expected failure, in which the expected failure should apply to the current data row. Change-Id: I70a6592de77b1718c3a749fd2216e66f8f312805 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-12 04:57:08 +00:00
<DataTag><![CDATA[first row]]></DataTag>
<Description><![CDATA[This test should xfail]]></Description>
</Incident>
testlib: Count passes, fails and skips consistently. For data-driven tests, testlib previously counted one fail or skip for each data row that failed or skipped, while it counted only one pass for a test function where all rows passed and counted no passes for a test function where some rows passed and some rows failed. A similar problem also existed for benchmark tests, which could run multiple iterations of the same test, with each fail and skip being counted but only a single pass being counted for the entire series of iterations. This commit makes testlib count one pass, fail or skip for each data row. Test functions that are not data-driven count one result for the test function, as before. Benchmark tests count one pass, fail or skip per iteration. A side-effect of this change is that the test output in plain text, xml and light xml formats now shows a result for every data row and benchmark iteration executed, allowing post-processors to correctly calculate the total number of tests executed. Previously, individual rows were not shown in the test output if they passed, making such calculations impossible. The only change to the xunitxml output format is to correct a bug where no test result was recorded for a test function if the last data row was skipped and all other rows passed -- in which case the overall result should be a pass. Note that there is also a pre-existing bug in the xunit logger, where no result is reported if all rows are skipped; that bug is unaffected by this commit. Task-number: QTBUG-21848 Task-number: QTBUG-22124 Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b Reviewed-by: Ed Baak <ed.baak@nokia.com>
2012-02-02 07:08:44 +00:00
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[first row]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xfail" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Improve coverage of expectfail selftest Add a test case for calling QEXPECT_FAIL in a data-driven test without passing a specific data tag for the expected failure, in which the expected failure should apply to the current data row. Change-Id: I70a6592de77b1718c3a749fd2216e66f8f312805 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-12 04:57:08 +00:00
<DataTag><![CDATA[second row]]></DataTag>
<Description><![CDATA[This test should xfail]]></Description>
</Incident>
testlib: Count passes, fails and skips consistently. For data-driven tests, testlib previously counted one fail or skip for each data row that failed or skipped, while it counted only one pass for a test function where all rows passed and counted no passes for a test function where some rows passed and some rows failed. A similar problem also existed for benchmark tests, which could run multiple iterations of the same test, with each fail and skip being counted but only a single pass being counted for the entire series of iterations. This commit makes testlib count one pass, fail or skip for each data row. Test functions that are not data-driven count one result for the test function, as before. Benchmark tests count one pass, fail or skip per iteration. A side-effect of this change is that the test output in plain text, xml and light xml formats now shows a result for every data row and benchmark iteration executed, allowing post-processors to correctly calculate the total number of tests executed. Previously, individual rows were not shown in the test output if they passed, making such calculations impossible. The only change to the xunitxml output format is to correct a bug where no test result was recorded for a test function if the last data row was skipped and all other rows passed -- in which case the overall result should be a pass. Note that there is also a pre-existing bug in the xunit logger, where no result is reported if all rows are skipped; that bug is unaffected by this commit. Task-number: QTBUG-21848 Task-number: QTBUG-22124 Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b Reviewed-by: Ed Baak <ed.baak@nokia.com>
2012-02-02 07:08:44 +00:00
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[second row]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Improve coverage of expectfail selftest Add a test case for calling QEXPECT_FAIL in a data-driven test without passing a specific data tag for the expected failure, in which the expected failure should apply to the current data row. Change-Id: I70a6592de77b1718c3a749fd2216e66f8f312805 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-12 04:57:08 +00:00
</TestFunction>
testlib: Clear expected failures after every data row. Previously, expected failures were only cleared at the end of each test function, i.e. after all data rows were finished. This meant that if a data-driven test function called QEXPECT_FAIL and then didn't perform any further verification steps to trigger the expected failure, the expected failure would be carried over to the next data row, probably causing the first verification step in the test function to XPASS (with a seemingly irrelevant error message) for the next data row. This commit adds the new function QTestResult::finishedCurrentTestData() to cleanup after each data row is executed. This function treats calls to QEXPECT_FAIL without subsequent verification steps as a test failure. This commit also adds a regression test to demonstrate that expected failures can no longer be carried over from one data row to another. If run against the previous version of testlib, the new test would report a pass instead of an error. Change-Id: Ida5c7f080815b0dca9531131fed582b0918334cb Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 05:42:00 +00:00
<TestFunction name="xfailWithoutVerify">
testlib: Pass on file location on failure, but don't assume we have one We try our best to pass on the file location of a failure, including for fatal errors, but the reporting or logging machinery should not assume there is one. By passing on nullptr for the file location we allow the logging backends to decide how to handle the situation, e.g. by not emitting extra fields for failure location. This effectively reverts c25687fa0b6e4be043e1f8c92c093d8b06ca06c4, in favor of relying on the backends to cope with null filename, which they already did. As qFatal uses QMessageLogger, which by default disables file/line information in release builds, we need to explicitly enable this in our self-tests, to get uniform test results. Similarly, we disable file/line info from testlib itself, as reporting Qt internal file and line information for user diagnostics is less useful. The odd one out there is qtestdata.cpp, which still ends up in test output due to using QTEST_ASSERT instead of qFatal for its diagnostics. Cleaning up that, and unifying how we report testlib issues to the user, is left for another day. Pick-to: 6.2 Change-Id: Ib9451b8eed86fe3ade4a4dcaf0037e1a3450321c Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-07-28 09:44:01 +00:00
<Incident type="fail" file="" line="0">
testlib: Clear expected failures after every data row. Previously, expected failures were only cleared at the end of each test function, i.e. after all data rows were finished. This meant that if a data-driven test function called QEXPECT_FAIL and then didn't perform any further verification steps to trigger the expected failure, the expected failure would be carried over to the next data row, probably causing the first verification step in the test function to XPASS (with a seemingly irrelevant error message) for the next data row. This commit adds the new function QTestResult::finishedCurrentTestData() to cleanup after each data row is executed. This function treats calls to QEXPECT_FAIL without subsequent verification steps as a test failure. This commit also adds a regression test to demonstrate that expected failures can no longer be carried over from one data row to another. If run against the previous version of testlib, the new test would report a pass instead of an error. Change-Id: Ida5c7f080815b0dca9531131fed582b0918334cb Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 05:42:00 +00:00
<DataTag><![CDATA[first row]]></DataTag>
<Description><![CDATA[QEXPECT_FAIL was called without any subsequent verification statements]]></Description>
</Incident>
testlib: Pass on file location on failure, but don't assume we have one We try our best to pass on the file location of a failure, including for fatal errors, but the reporting or logging machinery should not assume there is one. By passing on nullptr for the file location we allow the logging backends to decide how to handle the situation, e.g. by not emitting extra fields for failure location. This effectively reverts c25687fa0b6e4be043e1f8c92c093d8b06ca06c4, in favor of relying on the backends to cope with null filename, which they already did. As qFatal uses QMessageLogger, which by default disables file/line information in release builds, we need to explicitly enable this in our self-tests, to get uniform test results. Similarly, we disable file/line info from testlib itself, as reporting Qt internal file and line information for user diagnostics is less useful. The odd one out there is qtestdata.cpp, which still ends up in test output due to using QTEST_ASSERT instead of qFatal for its diagnostics. Cleaning up that, and unifying how we report testlib issues to the user, is left for another day. Pick-to: 6.2 Change-Id: Ib9451b8eed86fe3ade4a4dcaf0037e1a3450321c Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io> Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-07-28 09:44:01 +00:00
<Incident type="fail" file="" line="0">
testlib: Clear expected failures after every data row. Previously, expected failures were only cleared at the end of each test function, i.e. after all data rows were finished. This meant that if a data-driven test function called QEXPECT_FAIL and then didn't perform any further verification steps to trigger the expected failure, the expected failure would be carried over to the next data row, probably causing the first verification step in the test function to XPASS (with a seemingly irrelevant error message) for the next data row. This commit adds the new function QTestResult::finishedCurrentTestData() to cleanup after each data row is executed. This function treats calls to QEXPECT_FAIL without subsequent verification steps as a test failure. This commit also adds a regression test to demonstrate that expected failures can no longer be carried over from one data row to another. If run against the previous version of testlib, the new test would report a pass instead of an error. Change-Id: Ida5c7f080815b0dca9531131fed582b0918334cb Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 05:42:00 +00:00
<DataTag><![CDATA[second row]]></DataTag>
<Description><![CDATA[QEXPECT_FAIL was called without any subsequent verification statements]]></Description>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
testlib: Clear expected failures after every data row. Previously, expected failures were only cleared at the end of each test function, i.e. after all data rows were finished. This meant that if a data-driven test function called QEXPECT_FAIL and then didn't perform any further verification steps to trigger the expected failure, the expected failure would be carried over to the next data row, probably causing the first verification step in the test function to XPASS (with a seemingly irrelevant error message) for the next data row. This commit adds the new function QTestResult::finishedCurrentTestData() to cleanup after each data row is executed. This function treats calls to QEXPECT_FAIL without subsequent verification steps as a test failure. This commit also adds a regression test to demonstrate that expected failures can no longer be carried over from one data row to another. If run against the previous version of testlib, the new test would report a pass instead of an error. Change-Id: Ida5c7f080815b0dca9531131fed582b0918334cb Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 05:42:00 +00:00
</TestFunction>
Improve readability of expectfail selftest. Rename some of the test functions to improve consistency, and change the order so that all the xfail tests are together (some new xpass tests will be added later). Change-Id: Ice1ac5bf61bc26109bb0d08f72eb74e1a0424101 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:33:42 +00:00
<TestFunction name="xpass">
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xpass" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
testlib: make XPASS output slightly less confusing. Prior to this commit, when an unexpected pass occurred the test output showed the message "XPASS : tst_foo::function() 'expr' returned FALSE", where the problem was actually that the expression evaluated to true when it was expected to be false. This commit changes the output to make it clear that the expression evaluated to true unexpectedly. Task-number: QTBUG-22118 Change-Id: Id22c178073d3b75789675ca37a8ef019029b1f91 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-17 05:11:20 +00:00
<Description><![CDATA['true' returned TRUE unexpectedly. ()]]></Description>
Add selftest for xpass in a data-driven test. Change-Id: I9491d1e17ad31453ec353a11c9efb31522a16459 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:53:33 +00:00
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Add selftest for xpass in a data-driven test. Change-Id: I9491d1e17ad31453ec353a11c9efb31522a16459 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:53:33 +00:00
</TestFunction>
testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-20 00:15:57 +00:00
<TestFunction name="xpassDataDrivenWithQVerify">
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xpass" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
Add selftest for xpass in a data-driven test. Change-Id: I9491d1e17ad31453ec353a11c9efb31522a16459 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:53:33 +00:00
<DataTag><![CDATA[XPass]]></DataTag>
testlib: make XPASS output slightly less confusing. Prior to this commit, when an unexpected pass occurred the test output showed the message "XPASS : tst_foo::function() 'expr' returned FALSE", where the problem was actually that the expression evaluated to true when it was expected to be false. This commit changes the output to make it clear that the expression evaluated to true unexpectedly. Task-number: QTBUG-22118 Change-Id: Id22c178073d3b75789675ca37a8ef019029b1f91 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-17 05:11:20 +00:00
<Description><![CDATA['true' returned TRUE unexpectedly. ()]]></Description>
Improve readability of expectfail selftest. Rename some of the test functions to improve consistency, and change the order so that all the xfail tests are together (some new xpass tests will be added later). Change-Id: Ice1ac5bf61bc26109bb0d08f72eb74e1a0424101 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:33:42 +00:00
</Incident>
testlib: Count passes, fails and skips consistently. For data-driven tests, testlib previously counted one fail or skip for each data row that failed or skipped, while it counted only one pass for a test function where all rows passed and counted no passes for a test function where some rows passed and some rows failed. A similar problem also existed for benchmark tests, which could run multiple iterations of the same test, with each fail and skip being counted but only a single pass being counted for the entire series of iterations. This commit makes testlib count one pass, fail or skip for each data row. Test functions that are not data-driven count one result for the test function, as before. Benchmark tests count one pass, fail or skip per iteration. A side-effect of this change is that the test output in plain text, xml and light xml formats now shows a result for every data row and benchmark iteration executed, allowing post-processors to correctly calculate the total number of tests executed. Previously, individual rows were not shown in the test output if they passed, making such calculations impossible. The only change to the xunitxml output format is to correct a bug where no test result was recorded for a test function if the last data row was skipped and all other rows passed -- in which case the overall result should be a pass. Note that there is also a pre-existing bug in the xunit logger, where no result is reported if all rows are skipped; that bug is unaffected by this commit. Task-number: QTBUG-21848 Task-number: QTBUG-22124 Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b Reviewed-by: Ed Baak <ed.baak@nokia.com>
2012-02-02 07:08:44 +00:00
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Pass]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Improve readability of expectfail selftest. Rename some of the test functions to improve consistency, and change the order so that all the xfail tests are together (some new xpass tests will be added later). Change-Id: Ice1ac5bf61bc26109bb0d08f72eb74e1a0424101 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-01 04:33:42 +00:00
</TestFunction>
testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-20 00:15:57 +00:00
<TestFunction name="xpassDataDrivenWithQCompare">
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Incident type="xpass" file="qtbase/tests/auto/testlib/selftests/expectfail/tst_expectfail.cpp" line="0">
testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-20 00:15:57 +00:00
<DataTag><![CDATA[XPass]]></DataTag>
testlib: Improve verbose and XPASS output Previously, verbose (-v2) and XPASS test output showed all QCOMPAREs as "COMPARE()", making it impossible to see what was compared and difficult to match the output to the source of a test containing many calls to QCOMPARE. This commit changes testlib's internal compare_helper API so that string representations of the compared expressions are always passed to QTestResult::compare() when available, and can thus be shown in the verbose and XPASS output. The XPASS output has also been changed to state explicitly that the comparison succeeded unexpectedly, bringing it in line with the XPASS output resulting from a call to QVERIFY. This commit also changes all calls to compare_helper() to call the eight-argument version of the function, which simplifies much of the calling code. The now obsolete four-argument version of compare_helper() has been changed to output a warning that it is obsolete. It will be removed once other modules have had some time to catch up. The improved XPASS and verbose output is demonstrated by the expectfail and verbose2 selftests. Change-Id: I8baa46d5dd30e6c43b26f366c34dc5b64aab5f7c Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-29 05:43:42 +00:00
<Description><![CDATA[QCOMPARE(1, 1) returned TRUE unexpectedly.]]></Description>
testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-20 00:15:57 +00:00
</Incident>
<Incident type="pass" file="" line="0">
<DataTag><![CDATA[Pass]]></DataTag>
</Incident>
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
testlib: Add selftests for XPASS and XFAIL involving QCOMPARE. Change-Id: I4f897b411aaa51ff453208eda50e12665489074b Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-20 00:15:57 +00:00
</TestFunction>
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
<TestFunction name="cleanupTestCase">
Make lightxml and xml log formatting consistent For XML logs, an Incident tag with no sub-tags has a space before the closing slash. For Light XML logs, this space was missing unless the -flush command line option was supplied. Change-Id: I6de2e140f9309f333b5b97ed7f8116e4e727a149 Reviewed-on: http://codereview.qt.nokia.com/3871 Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com> Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-08-30 09:54:32 +00:00
<Incident type="pass" file="" line="0" />
generate_expected_output.py: update generated expected_* files In selftests.qrc, imposed alphabetic order (on stem of name, then on suffix, effectively treating . as sorting before any letter) while removing old tests and adding new tests and data. Updated all non-csv files and added many missing files. (Not clear on csv support status; the script seems to have dropped it after 5.6, but the test still uses it.) Left expected_crashes* alone (no new files added, no update to old) as I don't get results resembling those anticipated. Omitted printdatatagswithglobaltags, printdatatags due to dangling hspace on output lines, which upset sanity-bot. A change to the test cpp is needed to make it viable to skip that dangling hspace. Change-Id: Iab3fb626c44a91c249b2fb626c12c75ea0317098 Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-07-29 11:18:24 +00:00
<Duration msecs="0"/>
Initial import from the monolithic Qt. This is the beginning of revision history for this module. If you want to look at revision history older than this, please refer to the Qt Git wiki for how to use Git history grafting. At the time of writing, this wiki is located here: http://qt.gitorious.org/qt/pages/GitIntroductionWithQt If you have already performed the grafting and you don't see any history beyond this commit, try running "git log" with the "--follow" argument. Branched from the monolithic repo, Qt master branch, at commit 896db169ea224deb96c59ce8af800d019de63f12
2011-04-27 10:05:43 +00:00
</TestFunction>
Testlib: Add test duration to xml output [ChangeLog][QtTest] Added test duration to xml output. When running tests with xml output a new tag of the form <duration msecs="123"/> is added to each test function and the test as a whole. Change-Id: Ibc4db066b6acf5fac6c578f5e5ca5ce4b5d8ea8e Reviewed-by: Sergio Ahumada <sahumada@blackberry.com> Reviewed-by: Jason McDonald <macadder1@gmail.com>
2013-10-24 18:10:37 +00:00
<Duration msecs="0"/>
Reference in New Issue Copy Permalink