Commit Graph

3 Commits

Author SHA1 Message Date
Edward Welbourne
8d0b5270fa TAP test logger: skip XFail results as well as pass after an XFail
The rationale for skipping pass after XFail is "to emit a single test
point for" the test; emitting several XFails violates that aim.

Task-number: QTBUG-96844
Change-Id: Ia8626dfc2dded234b3aa530fc2dc2324f1e28400
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2021-12-09 20:54:26 +01:00
Tor Arne Vestbø
f1c16139e3 testlib selftest: rebaseline tap results to not include line numbers
Change-Id: I6271ac1984a3829ba1c8dcc9d6419940cb89c62b
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
2020-07-22 23:23:01 +02:00
Tor Arne Vestbø
3b42e098ef testlib: Add Test Anything Protocol (TAP) reporter
The Test Anything Protocol (TAP), was originally Perl's simple text-based
interface between testing modules and test harnesses, but has since been
adopted by a large number of producers and consumers in many different
languages, which allows colorizing and summarizing test results.

The format is very simple:

TAP version 13
ok 1 - test description
not ok 2 - test description
  ---
  message: 'Failure message'
  severity: fail
  expected: 123
  actual: 456
  ...
ok 3 - test description # SKIP
1..3

The specification [1] is very brief, so the implementation has been
based on how typical consumers behave, especially when it comes to
the undefined diagnostics block.

[1] http://testanything.org/tap-version-13-specification.html

Change-Id: I616e802ea380165c678510e940ddc6607d39c92d
Reviewed-by: Simon Hausmann <simon.hausmann@qt.io>
2018-03-14 14:28:36 +00:00