AuroraMiddleware/qt5base-lts
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
bab660d124
qt5base-lts/tests/auto/testlib/selftests/expected_verbose2.tap

148 lines
4.5 KiB
Plaintext
Raw Normal View History

testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
TAP version 13
# tst_Counting
ok 1 - initTestCase()
ok 2 - testPassPass(row 1)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(true)
# inform: QCOMPARE(2 + 1, 3)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
ok 3 - testPassPass(row 2)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(true)
# inform: QCOMPARE(2 + 1, 3)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
ok 4 - testPassSkip(row 1)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(true)
# inform: QCOMPARE(2 + 1, 3)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
ok 5 - testPassSkip(row 2) # SKIP Skipping
ok 6 - testPassFail(row 1)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(true)
# inform: QCOMPARE(2 + 1, 3)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
not ok 7 - testPassFail(row 2)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(false)
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
at: tst_Counting::testPassFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
line: 0
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
...
ok 8 - testSkipPass(row 1) # SKIP Skipping
ok 9 - testSkipPass(row 2)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(true)
# inform: QCOMPARE(2 + 1, 3)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
ok 10 - testSkipSkip(row 1) # SKIP Skipping
ok 11 - testSkipSkip(row 2) # SKIP Skipping
ok 12 - testSkipFail(row 1) # SKIP Skipping
not ok 13 - testSkipFail(row 2)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(false)
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
at: tst_Counting::testSkipFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
line: 0
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
...
not ok 14 - testFailPass(row 1)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(false)
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
at: tst_Counting::testFailPass() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
line: 0
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
...
ok 15 - testFailPass(row 2)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(true)
# inform: QCOMPARE(2 + 1, 3)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
not ok 16 - testFailSkip(row 1)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(false)
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
at: tst_Counting::testFailSkip() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
line: 0
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
...
ok 17 - testFailSkip(row 2) # SKIP Skipping
not ok 18 - testFailFail(row 1)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(false)
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
line: 0
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
...
not ok 19 - testFailFail(row 2)
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
# inform: QVERIFY(false)
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
---
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
type: QVERIFY
message: Verification failed
wanted: true (false)
found: false (false)
expected: true (false)
actual: false (false)
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
line: 0
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
...
ok 20 - testFailInInit(before)
not ok 21 - testFailInInit(fail)
---
# Fail in init()
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
at: tst_Counting::testFailInInit() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
line: 0
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
...
ok 22 - testFailInInit(after)
ok 23 - testFailInCleanup(before)
not ok 24 - testFailInCleanup(fail)
---
# Fail in cleanup()
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
at: tst_Counting::testFailInCleanup() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
testlib selftest: rebaseline tap results to not include line numbers Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 11:33:55 +00:00
line: 0
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
extensions:
messages:
- severity: debug
message: This test function should execute and then QFAIL in cleanup()
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
...
ok 25 - testFailInCleanup(after)
ok 26 - testSkipInInit(before)
ok 27 - testSkipInInit(skip) # SKIP Skip in init()
ok 28 - testSkipInInit(after)
ok 29 - testSkipInCleanup(before)
ok 30 - testSkipInCleanup(skip) # SKIP Skip in cleanup()
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
---
Move TAP's messages block to after the primary report details In the process, split it from its comments block and don't bother with a YAML block if it would only have contained comments. Task-number: QTBUG-96844 Change-Id: I08c20f796252bb270ba9caa4c055cdcc0843a88b Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-29 12:32:21 +00:00
at: tst_Counting::testSkipInCleanup() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
line: 0
TAP test logger: move messages into the diagnostics block Our TAP output was delivering messages as comments before the test line, where TAP clearly expects the details of a test to follow its test line. Version 13 provides a YAML block to deliver diagnostics and encourages use of this, so accumulate our messages in a QTestCharBuffer instead of emitting them one by one. However, messages produced after a test has produced its test line belong to that test, but are too late to be included in its diagnostics block, so should be emitted immediately as before, albeit now with a type prefix. This at least separates such messages, from the end of one test, from messages produced early in the next. In the process, add a type-prefix to each, to make clear what type of message it was. Since the Yamlish supported by TAP consumers doesn't support a way to have many messages, use the extensions: top-level hash tag with a messages: sub-tag to gather our messages as a list. (This expands at least one expected output file significantly and substantially rewrites some others.) Add methods to QTestCharBuffer, and a helper function, to support this. Task-number: QTBUG-96844 Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2 Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
extensions:
messages:
- severity: debug
message: This test function should execute and then QSKIP in cleanup()
...
testlib: Add Test Anything Protocol (TAP) reporter The Test Anything Protocol (TAP), was originally Perl's simple text-based interface between testing modules and test harnesses, but has since been adopted by a large number of producers and consumers in many different languages, which allows colorizing and summarizing test results. The format is very simple: TAP version 13 ok 1 - test description not ok 2 - test description --- message: 'Failure message' severity: fail expected: 123 actual: 456 ... ok 3 - test description # SKIP 1..3 The specification [1] is very brief, so the implementation has been based on how typical consumers behave, especially when it comes to the undefined diagnostics block. [1] http://testanything.org/tap-version-13-specification.html Change-Id: I616e802ea380165c678510e940ddc6607d39c92d Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-01 16:06:23 +00:00
ok 31 - testSkipInCleanup(after)
ok 32 - cleanupTestCase()
1..32
# tests 32
# pass 16
# fail 8
Reference in New Issue Copy Permalink