fb2ef5fbf6
Change-Id: Ica08f7013933e9e6a7678c0ba1f5827efa6eff42 Reviewed-by: Friedemann Kleint <Friedemann.Kleint@qt.io>
72 lines
2.7 KiB
Plaintext
72 lines
2.7 KiB
Plaintext
|
||
Running the QtTestLib selftests with Catch2
|
||
===========================================
|
||
|
||
Catch2 [1] is a header only test framework that we use to allow
|
||
testing QtTestLib without relying on any part of testlib itself.
|
||
|
||
To run the test suite, execute 'make check' or './tst_selftests'
|
||
as normal. This should print:
|
||
|
||
===================================================================
|
||
All tests passed (2453 assertions in 5 test cases)
|
||
|
||
To run specific tests, first lists the available tests:
|
||
|
||
❯ ./tst_selftests -l
|
||
All available test cases:
|
||
Loggers support both old and new style arguments
|
||
Loggers can output to both file and stdout
|
||
Logging to file and stdout at the same time
|
||
All loggers can be enabled at the same time
|
||
Scenario: Test output of the loggers is as expected
|
||
5 test cases
|
||
|
||
Then pass the name of the test in quotes as the first argument:
|
||
|
||
❯ ./tst_selftests "Loggers support both old and new style arguments"
|
||
Filters: Loggers support both old and new style arguments
|
||
==================================================================
|
||
All tests passed (96 assertions in 1 test case)
|
||
|
||
You can find the tests in the sources as individual TEST_CASE
|
||
entries. Note that each of these tests run the tests once per
|
||
logger, and in the case of the test log check also all sub tests,
|
||
so the amount of actual test assertions is much higher than the
|
||
five tests listed above.
|
||
|
||
To see what the tests is actually doing, pass the -s option.
|
||
This will result in very verbose output. Each leaf test is
|
||
prefixed with a heading:
|
||
|
||
---------------------------------------------------------------
|
||
Given: The QTestLog::TAP logger
|
||
When: Passing arguments with new style
|
||
---------------------------------------------------------------
|
||
|
||
You can choose a specific subtest by passing the -c option:
|
||
|
||
❯ ./tst_selftests "Scenario: Test output of the loggers is as expected" \
|
||
-c "Given: The QTestLog::Plain logger" \
|
||
-c 'And given: The "skip" subtest'
|
||
|
||
It's possible to pass only the first -c options, to e.g. run all
|
||
tests with the Plain logger, but it's unfortunately not possible
|
||
to pass only the last -c option, to run the 'skip' subtest with
|
||
all loggers.
|
||
|
||
If a test fails it will print the expected, actual, and difference.
|
||
The test results are also left in a temporary directory for closer
|
||
inspection.
|
||
|
||
Add new tests by modifying selftest.pri and CMakeLists.txt, adding
|
||
a new subprogram.
|
||
|
||
Generating new test expectations is done using the python script
|
||
in this directory (generate_expected_output.py). In the future this
|
||
will be done with the --rebase option to ./tst_selftest, but this
|
||
is not fleshed out yet.
|
||
|
||
[1] https://github.com/catchorg/Catch2
|
||
|