AuroraMiddleware/qt5base-lts
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
64b6e6b85c
qt5base-lts/tests/auto/testlib/selftests/expected_warnings.tap

167 lines
5.1 KiB
Plaintext
Raw Normal View History

testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
TAP version 13
# tst_Warnings
ok 1 - initTestCase()
ok 2 - testWarnings()
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
---
extensions:
messages:
- severity: warning
message: Warning
- severity: warning
message: Warning
- severity: debug
message: Debug
- severity: debug
message: Debug
- severity: info
message: Info
- severity: info
message: Info
- severity: debug
message: Baba
- severity: debug
message: Baba
- severity: debug
message: Bubublabla
- severity: warning
message: Babablabla
...
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
not ok 3 - testMissingWarnings()
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: Did not receive message: "Warning0"
# inform: Did not receive message: "Warning1"
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
# Not all expected messages were received
...
not ok 4 - testMissingWarningsRegularExpression()
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: Did not receive any message matching: "Warning\s\d"
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
# Not all expected messages were received
...
not ok 5 - testMissingWarningsWithData(first row)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: Did not receive message: "Warning0"
# inform: Did not receive message: "Warning1"
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
# Not all expected messages were received
...
not ok 6 - testMissingWarningsWithData(second row)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: Did not receive message: "Warning0"
# inform: Did not receive message: "Warning1"
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
# Not all expected messages were received
...
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
not ok 7 - testFailOnWarnings()
---
# Received a warning that resulted in a failure:
Ran out of cabbage!
at: tst_Warnings::testFailOnWarnings() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
extensions:
messages:
- severity: warning
message: Ran out of space!
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
...
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# debug: Ran out of tortillas!
# info: Ran out of oil!
# warning: nope
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
not ok 7 - testFailOnWarnings()
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
---
# Received a warning that resulted in a failure:
Ran out of biscuits!
at: tst_Warnings::testFailOnWarnings() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
...
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
not ok 7 - testFailOnWarnings()
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
---
# Received a warning that resulted in a failure:
Running low on toothpaste!
at: tst_Warnings::testFailOnWarnings() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
...
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# warning: Running low on flour!
# debug: Running low on toothpaste!
# info: Running low on toothpaste!
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
ok 8 - testFailOnWarningsCleared()
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
---
extensions:
messages:
- severity: warning
message: Ran out of muffins!
...
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
not ok 9 - testFailOnWarningsWithData(warning1)
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
---
# Received a warning that resulted in a failure:
warning1
at: tst_Warnings::testFailOnWarningsWithData() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
...
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# warning: warning2
# warning: warning3
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
not ok 10 - testFailOnWarningsWithData(warning2)
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
---
# Received a warning that resulted in a failure:
warning2
at: tst_Warnings::testFailOnWarningsWithData() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
extensions:
messages:
- severity: warning
message: warning1
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
...
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# warning: warning3
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
not ok 11 - testFailOnWarningsWithData(warning3)
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
---
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
# Received a warning that resulted in a failure:
warning3
at: tst_Warnings::testFailOnWarningsWithData() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
extensions:
messages:
- severity: warning
message: warning1
- severity: warning
message: warning2
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
...
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
not ok 12 - testFailOnWarningsFailInHelper()
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
---
# This failure message should be printed but not cause the test to abort
at: tst_Warnings::testFailOnWarningsFailInHelper() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
...
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
not ok 12 - testFailOnWarningsFailInHelper()
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
---
# Received a warning that resulted in a failure:
Ran out of cabbage!
at: tst_Warnings::testFailOnWarningsFailInHelper() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
...
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
not ok 12 - testFailOnWarningsFailInHelper()
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
---
# My cabbage! :(
at: tst_Warnings::testFailOnWarningsFailInHelper() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
...
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
not ok 13 - testFailOnWarningsThenSkip()
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
---
# Received a warning that resulted in a failure:
Ran out of cabbage!
at: tst_Warnings::testFailOnWarningsThenSkip() (qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp:0)
file: qtbase/tests/auto/testlib/selftests/warnings/tst_warnings.cpp
line: 0
...
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
ok 13 - testFailOnWarningsThenSkip() # SKIP My cabbage! :(
ok 14 - testFailOnWarningsAndIgnoreWarnings()
ok 15 - cleanupTestCase()
1..15
# tests 15
Add QTest::failOnWarning This solves the long-standing problem of not being able to easily fail a test when a certain warning is output. [ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test function, any warning that matches the given pattern will cause a test failure. The test will continue execution when a failure is added. All patterns are cleared at the end of each test function. Fixes: QTBUG-70029 Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
2021-10-08 11:58:27 +00:00
# pass 5
Make counts of various types of test result add up correctly Added tests for repeated skips and failures (from within void lambdas, to simulate skips and failures from within event handlers). These exhibit yet more ways to count more than one outcome for a test. The new QTest::failOnWarning() can also provoke more than one failure from a single test, and several existing selftests exhibited various ways for the Totals line's counts to add up to more than the number of actual tests run. Fixed counting so that only the first decisive incident is counted. Tests can still report later failure or skipping, but only the first is counted. Added a currentTestState in qtestlog.cpp, by which it keeps track of whether the test has resolved to a result, and clearCurrentTestState() by which other code can reset that at the end of each test. This brought to light various places where test-end clean-up was not being handled - due to failure or skipping in a *_data() method or init, or a skip in cleanup. Fixes: QTBUG-95661 Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2021-09-07 13:30:34 +00:00
# fail 10
Reference in New Issue Copy Permalink