2018-03-01 16:06:23 +00:00
|
|
|
TAP version 13
|
|
|
|
|
# tst_Counting
|
|
|
|
|
ok 1 - initTestCase()
|
|
|
|
|
ok 2 - testPassPass(row 1)
|
|
|
|
|
ok 3 - testPassPass(row 2)
|
|
|
|
|
ok 4 - testPassSkip(row 1)
|
|
|
|
|
ok 5 - testPassSkip(row 2) # SKIP Skipping
|
|
|
|
|
ok 6 - testPassFail(row 1)
|
|
|
|
|
not ok 7 - testPassFail(row 2)
|
|
|
|
|
---
|
|
|
|
|
type: QVERIFY
|
|
|
|
|
message: Verification failed
|
|
|
|
|
wanted: true (false)
|
|
|
|
|
found: false (false)
|
|
|
|
|
expected: true (false)
|
|
|
|
|
actual: false (false)
|
2020-07-22 11:33:55 +00:00
|
|
|
at: tst_Counting::testPassFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
|
2018-03-01 16:06:23 +00:00
|
|
|
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
|
2020-07-22 11:33:55 +00:00
|
|
|
line: 0
|
2018-03-01 16:06:23 +00:00
|
|
|
...
|
|
|
|
|
ok 8 - testSkipPass(row 1) # SKIP Skipping
|
|
|
|
|
ok 9 - testSkipPass(row 2)
|
|
|
|
|
ok 10 - testSkipSkip(row 1) # SKIP Skipping
|
|
|
|
|
ok 11 - testSkipSkip(row 2) # SKIP Skipping
|
|
|
|
|
ok 12 - testSkipFail(row 1) # SKIP Skipping
|
|
|
|
|
not ok 13 - testSkipFail(row 2)
|
|
|
|
|
---
|
|
|
|
|
type: QVERIFY
|
|
|
|
|
message: Verification failed
|
|
|
|
|
wanted: true (false)
|
|
|
|
|
found: false (false)
|
|
|
|
|
expected: true (false)
|
|
|
|
|
actual: false (false)
|
2020-07-22 11:33:55 +00:00
|
|
|
at: tst_Counting::testSkipFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
|
2018-03-01 16:06:23 +00:00
|
|
|
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
|
2020-07-22 11:33:55 +00:00
|
|
|
line: 0
|
2018-03-01 16:06:23 +00:00
|
|
|
...
|
|
|
|
|
not ok 14 - testFailPass(row 1)
|
|
|
|
|
---
|
|
|
|
|
type: QVERIFY
|
|
|
|
|
message: Verification failed
|
|
|
|
|
wanted: true (false)
|
|
|
|
|
found: false (false)
|
|
|
|
|
expected: true (false)
|
|
|
|
|
actual: false (false)
|
2020-07-22 11:33:55 +00:00
|
|
|
at: tst_Counting::testFailPass() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
|
2018-03-01 16:06:23 +00:00
|
|
|
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
|
2020-07-22 11:33:55 +00:00
|
|
|
line: 0
|
2018-03-01 16:06:23 +00:00
|
|
|
...
|
|
|
|
|
ok 15 - testFailPass(row 2)
|
|
|
|
|
not ok 16 - testFailSkip(row 1)
|
|
|
|
|
---
|
|
|
|
|
type: QVERIFY
|
|
|
|
|
message: Verification failed
|
|
|
|
|
wanted: true (false)
|
|
|
|
|
found: false (false)
|
|
|
|
|
expected: true (false)
|
|
|
|
|
actual: false (false)
|
2020-07-22 11:33:55 +00:00
|
|
|
at: tst_Counting::testFailSkip() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
|
2018-03-01 16:06:23 +00:00
|
|
|
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
|
2020-07-22 11:33:55 +00:00
|
|
|
line: 0
|
2018-03-01 16:06:23 +00:00
|
|
|
...
|
|
|
|
|
ok 17 - testFailSkip(row 2) # SKIP Skipping
|
|
|
|
|
not ok 18 - testFailFail(row 1)
|
|
|
|
|
---
|
|
|
|
|
type: QVERIFY
|
|
|
|
|
message: Verification failed
|
|
|
|
|
wanted: true (false)
|
|
|
|
|
found: false (false)
|
|
|
|
|
expected: true (false)
|
|
|
|
|
actual: false (false)
|
2020-07-22 11:33:55 +00:00
|
|
|
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
|
2018-03-01 16:06:23 +00:00
|
|
|
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
|
2020-07-22 11:33:55 +00:00
|
|
|
line: 0
|
2018-03-01 16:06:23 +00:00
|
|
|
...
|
|
|
|
|
not ok 19 - testFailFail(row 2)
|
|
|
|
|
---
|
|
|
|
|
type: QVERIFY
|
|
|
|
|
message: Verification failed
|
|
|
|
|
wanted: true (false)
|
|
|
|
|
found: false (false)
|
|
|
|
|
expected: true (false)
|
|
|
|
|
actual: false (false)
|
2020-07-22 11:33:55 +00:00
|
|
|
at: tst_Counting::testFailFail() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
|
2018-03-01 16:06:23 +00:00
|
|
|
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
|
2020-07-22 11:33:55 +00:00
|
|
|
line: 0
|
2018-03-01 16:06:23 +00:00
|
|
|
...
|
|
|
|
|
ok 20 - testFailInInit(before)
|
|
|
|
|
not ok 21 - testFailInInit(fail)
|
|
|
|
|
---
|
|
|
|
|
# Fail in init()
|
2020-07-22 11:33:55 +00:00
|
|
|
at: tst_Counting::testFailInInit() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
|
2018-03-01 16:06:23 +00:00
|
|
|
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
|
2020-07-22 11:33:55 +00:00
|
|
|
line: 0
|
2018-03-01 16:06:23 +00:00
|
|
|
...
|
|
|
|
|
ok 22 - testFailInInit(after)
|
|
|
|
|
ok 23 - testFailInCleanup(before)
|
|
|
|
|
not ok 24 - testFailInCleanup(fail)
|
|
|
|
|
---
|
TAP test logger: move messages into the diagnostics block
Our TAP output was delivering messages as comments before the test
line, where TAP clearly expects the details of a test to follow its
test line. Version 13 provides a YAML block to deliver diagnostics and
encourages use of this, so accumulate our messages in a
QTestCharBuffer instead of emitting them one by one.
However, messages produced after a test has produced its test line
belong to that test, but are too late to be included in its
diagnostics block, so should be emitted immediately as before, albeit
now with a type prefix. This at least separates such messages, from
the end of one test, from messages produced early in the next.
In the process, add a type-prefix to each, to make clear what type of
message it was. Since the Yamlish supported by TAP consumers doesn't
support a way to have many messages, use the extensions: top-level
hash tag with a messages: sub-tag to gather our messages as a list.
(This expands at least one expected output file significantly and
substantially rewrites some others.)
Add methods to QTestCharBuffer, and a helper function, to support this.
Task-number: QTBUG-96844
Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
|
|
|
extensions:
|
|
|
|
|
messages:
|
|
|
|
|
- severity: debug
|
|
|
|
|
message: This test function should execute and then QFAIL in cleanup()
|
2018-03-01 16:06:23 +00:00
|
|
|
# Fail in cleanup()
|
2020-07-22 11:33:55 +00:00
|
|
|
at: tst_Counting::testFailInCleanup() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
|
2018-03-01 16:06:23 +00:00
|
|
|
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
|
2020-07-22 11:33:55 +00:00
|
|
|
line: 0
|
2018-03-01 16:06:23 +00:00
|
|
|
...
|
|
|
|
|
ok 25 - testFailInCleanup(after)
|
|
|
|
|
ok 26 - testSkipInInit(before)
|
|
|
|
|
ok 27 - testSkipInInit(skip) # SKIP Skip in init()
|
|
|
|
|
ok 28 - testSkipInInit(after)
|
|
|
|
|
ok 29 - testSkipInCleanup(before)
|
|
|
|
|
ok 30 - testSkipInCleanup(skip) # SKIP Skip in cleanup()
|
TAP test logger: move messages into the diagnostics block
Our TAP output was delivering messages as comments before the test
line, where TAP clearly expects the details of a test to follow its
test line. Version 13 provides a YAML block to deliver diagnostics and
encourages use of this, so accumulate our messages in a
QTestCharBuffer instead of emitting them one by one.
However, messages produced after a test has produced its test line
belong to that test, but are too late to be included in its
diagnostics block, so should be emitted immediately as before, albeit
now with a type prefix. This at least separates such messages, from
the end of one test, from messages produced early in the next.
In the process, add a type-prefix to each, to make clear what type of
message it was. Since the Yamlish supported by TAP consumers doesn't
support a way to have many messages, use the extensions: top-level
hash tag with a messages: sub-tag to gather our messages as a list.
(This expands at least one expected output file significantly and
substantially rewrites some others.)
Add methods to QTestCharBuffer, and a helper function, to support this.
Task-number: QTBUG-96844
Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
|
|
|
---
|
|
|
|
|
extensions:
|
|
|
|
|
messages:
|
|
|
|
|
- severity: debug
|
|
|
|
|
message: This test function should execute and then QSKIP in cleanup()
|
2022-03-15 14:00:46 +00:00
|
|
|
at: tst_Counting::testSkipInCleanup() (qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp:0)
|
|
|
|
|
file: qtbase/tests/auto/testlib/selftests/counting/tst_counting.cpp
|
|
|
|
|
line: 0
|
TAP test logger: move messages into the diagnostics block
Our TAP output was delivering messages as comments before the test
line, where TAP clearly expects the details of a test to follow its
test line. Version 13 provides a YAML block to deliver diagnostics and
encourages use of this, so accumulate our messages in a
QTestCharBuffer instead of emitting them one by one.
However, messages produced after a test has produced its test line
belong to that test, but are too late to be included in its
diagnostics block, so should be emitted immediately as before, albeit
now with a type prefix. This at least separates such messages, from
the end of one test, from messages produced early in the next.
In the process, add a type-prefix to each, to make clear what type of
message it was. Since the Yamlish supported by TAP consumers doesn't
support a way to have many messages, use the extensions: top-level
hash tag with a messages: sub-tag to gather our messages as a list.
(This expands at least one expected output file significantly and
substantially rewrites some others.)
Add methods to QTestCharBuffer, and a helper function, to support this.
Task-number: QTBUG-96844
Change-Id: If44a33da5879ed1670ef0980042599afd516f9d2
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
2022-03-14 18:16:51 +00:00
|
|
|
...
|
2018-03-01 16:06:23 +00:00
|
|
|
ok 31 - testSkipInCleanup(after)
|
|
|
|
|
ok 32 - cleanupTestCase()
|
|
|
|
|
1..32
|
|
|
|
|
# tests 32
|
|
|
|
|
# pass 16
|
|
|
|
|
# fail 8
|
