ec5383efc3
It deoesn't like single quotes, which must be replaced with doubled-up double quotes. Pick-to: 6.2 6.5 6.6 Change-Id: I8e5f8047e72e4433926a9f4f2044407a4d823682 Reviewed-by: Edward Welbourne <edward.welbourne@qt.io> Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
79 lines
3.0 KiB
Plaintext
79 lines
3.0 KiB
Plaintext
|
||
Running the QtTestLib selftests with Catch2
|
||
===========================================
|
||
|
||
Catch2 [1] is a header only test framework that we use to allow
|
||
testing QtTestLib without relying on any part of testlib itself.
|
||
|
||
To run the test suite, execute 'make check' or './tst_selftests'
|
||
as normal. This should print:
|
||
|
||
===================================================================
|
||
All tests passed (2453 assertions in 5 test cases)
|
||
|
||
To run specific tests, first lists the available tests:
|
||
|
||
❯ ./tst_selftests -l
|
||
All available test cases:
|
||
Loggers support both old and new style arguments
|
||
Loggers can output to both file and stdout
|
||
Logging to file and stdout at the same time
|
||
All loggers can be enabled at the same time
|
||
Scenario: Test output of the loggers is as expected
|
||
5 test cases
|
||
|
||
Then pass the name of the test in quotes as the first argument:
|
||
|
||
❯ ./tst_selftests "Loggers support both old and new style arguments"
|
||
Filters: Loggers support both old and new style arguments
|
||
==================================================================
|
||
All tests passed (96 assertions in 1 test case)
|
||
|
||
You can find the tests in the sources as individual TEST_CASE
|
||
entries. Note that each of these tests run the tests once per
|
||
logger, and in the case of the test log check also all sub tests,
|
||
so the amount of actual test assertions is much higher than the
|
||
five tests listed above.
|
||
|
||
To see what the tests is actually doing, pass the -s option.
|
||
This will result in very verbose output. Each leaf test is
|
||
prefixed with a heading:
|
||
|
||
---------------------------------------------------------------
|
||
Given: The QTestLog::TAP logger
|
||
When: Passing arguments with new style
|
||
---------------------------------------------------------------
|
||
|
||
You can choose a specific subtest by passing the -c option:
|
||
|
||
❯ ./tst_selftests "Scenario: Test output of the loggers is as expected" \
|
||
-c "Given: The QTestLog::Plain logger" \
|
||
-c 'And given: The "skip" subtest'
|
||
|
||
Note that on Windows, when using cmd.exe, you cannot use single
|
||
quotes, so the command above becomes:
|
||
|
||
❯ ./tst_selftests "Scenario: Test output of the loggers is as expected" \
|
||
-c "Given: The QTestLog::Plain logger" \
|
||
-c "And given: The ""skip"" subtest"
|
||
|
||
It's possible to pass only the first -c options, to e.g. run all
|
||
tests with the Plain logger, but it's unfortunately not possible
|
||
to pass only the last -c option, to run the 'skip' subtest with
|
||
all loggers.
|
||
|
||
If a test fails it will print the expected, actual, and difference.
|
||
The test results are also left in a temporary directory for closer
|
||
inspection.
|
||
|
||
Add new tests by modifying selftest.pri and CMakeLists.txt, adding
|
||
a new subprogram.
|
||
|
||
Generating new test expectations is done using the python script
|
||
in this directory (generate_expected_output.py). In the future this
|
||
will be done with the --rebase option to ./tst_selftest, but this
|
||
is not fleshed out yet.
|
||
|
||
[1] https://github.com/catchorg/Catch2
|
||
|