72 lines
2.7 KiB
Plaintext
72 lines
2.7 KiB
Plaintext
|
|
|||
|
|
Running the QtTestLib selftests with Catch2
|
|||
|
|
===========================================
|
|||
|
|
|
|||
|
|
Catch2 [1] is a header only test framework that we use to allow
|
|||
|
|
testing QtTestLib without relying on any part of testlib itself.
|
|||
|
|
|
|||
|
|
To run the test suite, execute 'make check' or './tst_selftests'
|
|||
|
|
as normal. This should print:
|
|||
|
|
|
|||
|
|
===================================================================
|
|||
|
|
All tests passed (2453 assertions in 5 test cases)
|
|||
|
|
|
|||
|
|
To run specific tests, first lists the available tests:
|
|||
|
|
|
|||
|
|
❯ ./tst_selftests -l
|
|||
|
|
All available test cases:
|
|||
|
|
Loggers support both old and new style arguments
|
|||
|
|
Loggers can output to both file and stdout
|
|||
|
|
Logging to file and stdout at the same time
|
|||
|
|
All loggers can be enabled at the same time
|
|||
|
|
Scenario: Test output of the loggers is as expected
|
|||
|
|
5 test cases
|
|||
|
|
|
|||
|
|
Then pass the name of the test in quotes as the first argument:
|
|||
|
|
|
|||
|
|
❯ ./tst_selftests "Loggers support both old and new style arguments"
|
|||
|
|
Filters: Loggers support both old and new style arguments
|
|||
|
|
==================================================================
|
|||
|
|
All tests passed (96 assertions in 1 test case)
|
|||
|
|
|
|||
|
|
You can find the tests in the sources as individual TEST_CASE
|
|||
|
|
entries. Note that each of these tests run the tests once per
|
|||
|
|
logger, and in the case of the test log check also all sub tests,
|
|||
|
|
so the amount of actual test assertions is much higher than the
|
|||
|
|
five tests listed above.
|
|||
|
|
|
|||
|
|
To see what the tests is actually doing, pass the -s option.
|
|||
|
|
This will result in very verbose output. Each leaf test is
|
|||
|
|
prefixed with a heading:
|
|||
|
|
|
|||
|
|
---------------------------------------------------------------
|
|||
|
|
Given: The QTestLog::TAP logger
|
|||
|
|
When: Passing arguments with new style
|
|||
|
|
---------------------------------------------------------------
|
|||
|
|
|
|||
|
|
You can choose a specific subtest by passing the -c option:
|
|||
|
|
|
|||
|
|
❯ ./tst_selftests "Scenario: Test output of the loggers is as expected" \
|
|||
|
|
-c "Given: The QTestLog::Plain logger" \
|
|||
|
|
-c 'And given: The "skip" subtest'
|
|||
|
|
|
|||
|
|
It's possible to pass only the first -c options, to e.g. run all
|
|||
|
|
tests with the Plain logger, but it's unfortunately not possible
|
|||
|
|
to pass only the last -c option, to run the 'skip' subtest with
|
|||
|
|
all loggers.
|
|||
|
|
|
|||
|
|
If a test fails it will print the expected, actual, and difference.
|
|||
|
|
The test results are also left in a temporary directory for closer
|
|||
|
|
inspection.
|
|||
|
|
|
|||
|
|
Add new tests by modifying selftest.pri and CMakeLists.txt, adding
|
|||
|
|
a new subprogram.
|
|||
|
|
|
|||
|
|
Generating new test expectations is done using the python script
|
|||
|
|
in this directory (generate_expected_output.py). In the future this
|
|||
|
|
will be done with the --rebase option to ./tst_selftest, but this
|
|||
|
|
is not fleshed out yet.
|
|||
|
|
|
|||
|
|
[1] https://github.com/catchorg/Catch2
|
|||
|
|
|