Q_ASSERT uses __FILE__, which has undefined contents, and thus the
selftest cannot always match Q_ASSERT messages in actual output with
those in expected output. The test was calling QEXPECT_FAIL() in this
case, but doing so implies that (a) it's a failure and (b) we intend to
fix it one day. Neither of these is true, so instead we should simply
tolerate this difference in actual and expected outputs, in the same way
that we do in a couple of other similar cases.
Change-Id: I8b8609a400e9b798936875f6eb1ca18a83660efc
Reviewed-on: http://codereview.qt-project.org/5358
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Each destination and the format of output to write there is specified by
adding "-o filename,format" to the command-line. The special filename
"-" indicates that the log output is written to the standard output
stream, though standard output can be used as a destination at most
once.
The old-style testlib output options are still supported, but can only
be used to specify one logging destination, as before.
If no logging options are given on the command-line, a plain text log
will go to the console, as before.
To log to the console in plain text and to the file "test_output" in
xunit format, one would invoke a test in the following way:
tst_foo -o test_output,xunitxml -o -,txt
This commit also enhances the selftests to test with multiple loggers,
but negative tests (e.g. bad combinations of command-line options) are
left for future task QTBUG-21567.
Task-number: QTBUG-20615
Change-Id: If91e752bc7001657e15e427aba9d25ab0a29a0b0
Reviewed-on: http://codereview.qt-project.org/4125
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Modify the selftest to be able to run each subtest with a list of one or
more test loggers. The addition of tests that use this capability will
be part of a subsequent commit.
Task-number: QTBUG-20615
Change-Id: Iac3efe8220e8245aa7e5589348d2c86b8034dd28
Reviewed-on: http://codereview.qt-project.org/5292
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The multiexec test runs the same test object five times. If the -o
option is given, the output file is overwritten by each run of the test
object, meaning that tst_selftest only sees 1/5 of the test output in a
file compared with what it sees on the console. This makes it
impossible to use the expected output file to verify the test for both
console and file output. This issue is noted in QTBUG-21561.
Change-Id: I00031a2ea43a7ef78e8317473a089306ec062d8e
Reviewed-on: http://codereview.qt-project.org/5270
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The warnings selftest verifies that qtestlib correctly handles output
sent via qDebug() and friends. The test had a number of printf calls
that were used to tell the user what output they should expect to see
next, but by bypassing testlib's logging, these made the testlog differ
depending on whether the test log was directed to the console or a file.
The printf calls are no longer needed, as any regressions in qDebug will
be detected by tst_selftests, which compares the output of the warnings
test with a reference copy of the expected output.
Change-Id: I8c3a3237463141fa494c50ca02062760cb583090
Reviewed-on: http://codereview.qt-project.org/5200
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Make the test use qDebug instead of printf, so that the reference test
output is the same regardless of whether the log goes to the console or
a file.
Also rename the test class, as it was evidently cut-n-pasted from the
subtest selftest without being renamed.
Change-Id: I8df1f0989f2e8c33c1f829ba8110fd740ff70f63
Reviewed-on: http://codereview.qt-project.org/5184
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This test was attempting to verify two completely unrelated things, so
this commit splits it into two tests.
Also, printf calls are replaced by qDebug so that the test does not
bypass the testlib loggers.
Change-Id: I1a202af38ce2c69690a32d93405ba604ec6cabee
Reviewed-on: http://codereview.qt-project.org/5178
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This test is not useful for finding bugs in qtestlib's logging code,
because it bypasses the qtestlib loggers and doesn't play nice with
tst_selftest. Neither is this test very useful for finding bugs in
QTest::qWait(), as the test only proves the qWait() terminates, not that
it waits accurately, or even that it waits at all.
Change-Id: Ia5dd7cbaf3a6fbb4e94e54ed155263580e495694
Reviewed-on: http://codereview.qt-project.org/5173
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This is in preparation of removing testlib's
dependency on QtGui and QtWidgets.
Autotests that need QtWidgets api must
explicitly include it (since the types are no
longer provided by the QtGui master header).
Autotests that don't need QtGui or QtWidgets
api shouldn't link against those libraries.
Change-Id: I2808289068514fcac582808828ad4634e2631733
Reviewed-on: http://codereview.qt-project.org/5093
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
Don't store separate strings for the logger name and the associated file
suffix -- just use the same string everywhere.
Change-Id: Ie7d1af6bf906b5ac09fbd5fcc5754b68036fb370
Reviewed-on: http://codereview.qt-project.org/5060
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The selftest was only attempting to delete the output file at the end of
the entire test run. This file should actually be deleted after each
subtest to ensure that it does not cause confusing errors if the next
subtest fails to overwrite the output file.
Change-Id: Ia3827926f71a697c6108de3f2b08c969f761d978
Reviewed-on: http://codereview.qt-project.org/5051
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The removed calls related to tests that were removed when redundant
logging functionality was removed.
Change-Id: Iaebbc109eaaddb440ae78e1f4a1b4e874a8b2960
Reviewed-on: http://codereview.qt-project.org/5050
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
* Use C++ comment delimiters consistently
* Use a space after control flow keywords
* Don't nest "else if" blocks unnecessarily
* Follow indenting rules
* Added comments describing some internal functions
Change-Id: I72fd1eaf5c5f3130945fbd2b64fa59e19ad2913d
Reviewed-on: http://codereview.qt-project.org/5049
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Commit 949be8cbdd modified the assert test
case but failed to update the line number of the assert in the expected
test output files.
Change-Id: I0d4f55c0b8bc1de8487dd8c6267aa459000cda26
Reviewed-on: http://codereview.qt-project.org/5048
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This change disables failing tests to increase the
likelihood that 'make check' terminates with a zero
exit status. This is done to prevent the CI system
from blocking further commits once qtbase#refactor
is merged into qtbase#master.
Failing tests are marked as insignificant (by putting
'CONFIG += insignificant_test' in the .pro file).
Note: This is a temporary measure that needs to be cleaned up
once the refactor->master integration is complete.
Ideally all disabled tests need to be enabled and passing.
All changes will be marked by the string QTBUG-21402 in a comment
for easy location.
Task-number: QTBUG-21402
Change-Id: Ic4400671671f6d9b75b106a6aa4755b20e74378d
Reviewed-on: http://codereview.qt-project.org/4678
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Gunnar Sletta <gunnar.sletta@nokia.com>
