Commit Graph

416 Commits

Author SHA1 Message Date
Rohan McGovern
931f4501a6 Fixed tst_selftests on OSX 10.7
Our OSX 10.7 test machines send a SIGILL rather than a SIGSEGV to a
process which attempts to dereference a null pointer.

Change the "crashes" test to dereference an invalid pointer with a
value slightly greater than 0 so that we get the same crash behavior on
all (unix) platforms.

Change-Id: I700a2c7d654a9468af5e5996010a258695ed2ae5
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2012-02-09 03:31:08 +01:00
Jason McDonald
26c41ed918 testlib: Clear ignored messages after every data row
Previously, ignored messages were only cleared at the end of each test
function, i.e. after all data rows were finished. This meant that if a
data row in a data-driven test function didn't cause all of the expected
messages to be generated, the remaining messages would be carried over
to the next data row.  This would result in errors about missing
messages being associated with the last data row rather than with the
correct data row.

This commit makes testlib check for missing ignored messages after
running each data row rather than only doing so after the last data
row.

This commit also adds a regression test to demonstrate that ignored
messages can no longer be carried over from one data row to another.

Change-Id: Ibee51aa6e96866fbcbcb4acee1a8340a86a6a4ba
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-08 03:13:23 +01:00
Jason McDonald
da18ada6bf testlib: Clear expected failures after every data row.
Previously, expected failures were only cleared at the end of each test
function, i.e. after all data rows were finished.  This meant that if a
data-driven test function called QEXPECT_FAIL and then didn't perform
any further verification steps to trigger the expected failure, the
expected failure would be carried over to the next data row, probably
causing the first verification step in the test function to XPASS (with
a seemingly irrelevant error message) for the next data row.

This commit adds the new function QTestResult::finishedCurrentTestData()
to cleanup after each data row is executed.  This function treats calls
to QEXPECT_FAIL without subsequent verification steps as a test failure.

This commit also adds a regression test to demonstrate that expected
failures can no longer be carried over from one data row to another. If
run against the previous version of testlib, the new test would report a
pass instead of an error.

Change-Id: Ida5c7f080815b0dca9531131fed582b0918334cb
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-08 03:13:21 +01:00
Kurt Korbatits
3ddfa8d0ea Fixed selftests unittest on windows
- Needed to add toLower() for compare as c:\ and C:\ was causing
  a failure.

Change-Id: Idd55774d118b7249e88362688166b4f9d9712c7c
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2012-02-02 05:42:07 +01:00
Jason McDonald
ef25df9fec Add selftest for xpass in a data-driven test.
Change-Id: I9491d1e17ad31453ec353a11c9efb31522a16459
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-02 05:42:07 +01:00
Jason McDonald
4d1dcaef19 Improve readability of expectfail selftest.
Rename some of the test functions to improve consistency, and change the
order so that all the xfail tests are together (some new xpass tests
will be added later).

Change-Id: Ice1ac5bf61bc26109bb0d08f72eb74e1a0424101
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-02 05:42:07 +01:00
Jason McDonald
58c617c798 Add testlib selftest for counting of test results.
The test simply generates each possible pair of Pass, Fail and Skip
results.  At present the test simply serves to demonstrate the current
shortcomings of testlib's plain text logging, namely:

* If a test function passes for all data rows, that is counted as one
  pass, but each skipped or failed row counts as one skip or fail.

* Only skipped and failed rows are reported individually in the test
  output. Passed rows are not reported, so it is impossible to see how
  many rows were executed.

* A skip followed by a pass will be reported as an overall pass for the
  test function, but the same rows in reverse order will not report any
  overall result for the test function.

Future commits will attempt to correct these problems.

Task-number: QTBUG-22124
Change-Id: If8c7ea15fc43ba9a1bccd0e881c1efc18e705b25
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-02-02 05:42:07 +01:00
Jason McDonald
5635823e17 Remove "All rights reserved" line from license headers.
As in the past, to avoid rewriting various autotests that contain
line-number information, an extra blank line has been inserted at the
end of the license text to ensure that this commit does not change the
total number of lines in the license header.

Change-Id: I311e001373776812699d6efc045b5f742890c689
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-30 03:54:59 +01:00
Jason McDonald
629d6eda5c Update contact information in license headers.
Replace Nokia contact email address with Qt Project website.

Change-Id: I431bbbf76d7c27d8b502f87947675c116994c415
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-23 04:04:33 +01:00
Jason McDonald
0528d9dca7 Improve coverage of expectfail selftest
Add a test case for calling QEXPECT_FAIL in a data-driven test without
passing a specific data tag for the expected failure, in which the
expected failure should apply to the current data row.

Change-Id: I70a6592de77b1718c3a749fd2216e66f8f312805
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-13 04:04:17 +01:00
Jason McDonald
3dfdf34c1c Improve coverage of expectfail selftest
Add a test for the case where a call to QEXPECT_FAIL nominates a
different data row from the current row, in which case the call to
QEXPECT_FAIL should be ignored.

Change-Id: Iff774b89f9374fa26105acdd8769177272a95229
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-13 04:04:12 +01:00
Jason McDonald
fa07815892 Improve coverage of expectfail selftest.
Add a test for the case where QEXPECT_FAIL is called twice in a row
without an intervening verification step, which is supposed to result in
a failure.

Change-Id: Ie004652a76a2c35c2679945670ff5561e8f5b62e
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-13 01:41:14 +01:00
Jason McDonald
188db2b817 Make expectfail selftest test XPASS.
Change-Id: I0f7943049995c7fa4ad70a05fd2fdac4bc9f6ff6
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-12 06:41:39 +01:00
Jason McDonald
29d8d8faec Make selftest for QEXPECT_FAIL more robust.
Make the test fail if calling QEXPECT_FAIL with the mode set to Abort
does not cause the current test function to abort.  Previously, such an
error would not cause the selftest to fail and someone would have to see
the warning message in the test output to know that there is a problem.

Change-Id: I2bd58fafe6b51ea0ab86fde7c5afb781e4534ee4
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-12 06:41:35 +01:00
Jason McDonald
a59bc6a6e4 Improve selftest coverage of QEXPECT_FAIL feature.
The existing expectfail selftest did not test QEXPECT_FAIL with a
data-driven test function.  This commit adds such a test.

Change-Id: I39fa9aa227b58779ce5268dd37bf55468e7269c5
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-12 06:41:29 +01:00
Rohan McGovern
d3f6faedb5 Fixed findtestdata selftest on mac
This test relies on being able to set a custom QLibraryInfo::PrefixPath
for the duration of the test.  However, the test code neglected to
account for the following behavior on mac (from "Using qt.conf"
documentation):

  On Mac OS X, the Prefix is relative to the Contents in the application
  bundle.

Change-Id: Ie9b6d5ebfe8af7d7f332e827069b60a830d6c6f2
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2012-01-12 06:41:10 +01:00
Kurt Korbatits
de6d3d5b1d Changed selftests unittest to not fail but warn when valgrind not installed.
- Changed to detect valgrind at runtime and skip test instead of failing.
- subprograms inherit QT_QPA_PLATFORM value from parent if set.

Change-Id: I280acee389df1ee74ee6758a0dd1601226e103c7
Reviewed-by: Kurt Korbatits  <kurt.korbatits@nokia.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2012-01-11 04:09:32 +01:00
Kurt Korbatits
28bfa8f6cc Changed selftests unittest to work from installation directory.
- All subprograms installed as part of test instead of separate installs
- Added installTestHelperApp() to testcase.prf

Change-Id: I02fbbb299f095054c9216ad0e5f91f574fb0fe3d
Reviewed-by: Kurt Korbatits  <kurt.korbatits@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-11 04:09:26 +01:00
Jason McDonald
5313c28b30 Update year in Nokia copyright headers.
The previous change missed some headers from years prior to 2011, and a
few new files were merged after the previous change.

Change-Id: Ib7d1a2b7062228c2a5373da64242b2ee1f0981e1
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-10 07:12:41 +01:00
Kurt Korbatits
679c4c002d Changed selftests unittest to use specific headers instead of QtCore.
Changed selftests unittest to use specific classes of QtCore instead of
pulling in all of QtCore headers by using include <QtCore>
- Decreasing build time.

Change-Id: Ic53a70423ecafeb342cc05ae6d7dee795533d205
Reviewed-by: Kurt Korbatits  <kurt.korbatits@nokia.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2012-01-10 05:12:18 +01:00
Jason McDonald
1fdfc2abfe Update copyright year in license headers.
Change-Id: I02f2c620296fcd91d4967d58767ea33fc4e1e7dc
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-05 06:36:56 +01:00
Kurt Korbatits
77ddb00a49 Changed tests: qmake,selftest and lancelot to use QFINDTESTDATA.
Changed these tests to use QFINDTESTDATA macro to detect location of testdata.
Checking for a specific file contained in the testdata so as not to be confused by empty directories
created during configure.

Change-Id: Iac2ac6304b6b9ac79e00886025b93ec0af5a8507
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2012-01-05 03:49:45 +01:00
Bradley T. Hughes
2bbf9befd8 Add Qt::TimerType argument to QAbstractEventDispatcher::registerTimer()
... and deprecate the old registerTimer() functions. The new pure-
virtual registerTimer() breaks source-compatibility. Subclasses cannot
be instantiated anymore, since the pure virtual function signature has
changed.

QAbstractEventDispatcher::TimerInfo is no longer a QPair. It is now a
struct with timerId, interval, and timerType members. This is a source
incompatibility that should only affect subclasses of
QAbstractEventDispatcher, which will need to pass 3 arguments to the
TimerInfo constructor instead of 2. If the subclass used QPair<int,int>
instead of the TimerInfo typedef, the QPair<int,int> declarations will
need to be replaced with TimerInfo.

Call the new registerTimer() function with the type from
QObject::startTimer(). Change all subclasses of QAbstractEventDispatcher
to reimplement the new virtual function. The type argument is unused at
the momemnt, except to ensure that registeredTimers() returns the type
each timer was registered with. Implementations for the various
dispatchers will be done in separate commits.

Author: Thiago Macieira <thiago.macieira@nokia.com>
Change-Id: Ia22697e0ab0847810c5d162ef473e0e5a17a904b
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2012-01-02 10:44:00 +01:00
Friedemann Kleint
23382669b9 QTestLib-selftest: Unblock on Windows.
Do not run tests that pop up assert boxes.

Change-Id: Idac94398ff1109605eb33a042dab734eafb1a17a
Reviewed-by: Sergio Ahumada <sergio.ahumada@nokia.com>
2011-12-22 15:11:32 +01:00
Friedemann Kleint
bc8f25c7e6 QTestlib: Make QImage comparison more verbose.
Introduce a specialization for qCompare(QImage,QImage) that
checks isNull, size and format and outputs verbose messages.

Check isNull, size similarly for QPixmap.

Add an autotest:
- Add test to cmptest and make it a GUI application since
  QImage requires QGuiApplication.
- Make testlib/selftests capable of running X11-GUI applications
  by passing DISPLAY.
- Ignore stderr output for cmptest
- Add test data

Change-Id: I2b29c7822fbeedf2b22c90889739ed7ff859ce92
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2011-12-21 22:59:28 +01:00
Friedemann Kleint
92667d14a4 QTestlib-Selftest: Add initTestCase() setting the directory.
Required for launching the sub-processes. The test worked only
when launched locally, not from 'make check'.

Change-Id: I42c9202a7726c3135f94445fb336b2b8241535a4
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
2011-12-15 21:04:27 +01:00
Kurt Korbatits
0c9b050f91 Fix testlib selftests on shadow build.
Changed to use QT_INSTALL_TESTS to install subtests.

Change-Id: I4a2a7bd2d3e7d6da34dbb922bf377bee98cdedb0
Reviewed-by: Kurt Korbatits  <kurt.korbatits@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2011-12-14 07:15:46 +01:00
Jason McDonald
d939dd8791 Improve QStringList test in cmptest selftest.
The old test only verified the behaviour of QCOMPARE when comparing
lists that were different.  Add data rows for comparing empty lists and
non-empty lists that are equal.

Note that testlib currently does not report passing data rows (only
failing rows and completely passing test functions), so the new data
rows do not cause any change in the expected test output.

Change-Id: I137650ce0ca6250cee36bd9cb74b01f8abd4e89c
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-12-14 02:40:31 +01:00
Jason McDonald
4a233f9a90 Give names to data rows in cmptest selftest.
Change-Id: Ia362f0651c8d32602fa13d6639fb8386fe8d9e0b
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-12-14 02:40:26 +01:00
Jason McDonald
9bc546e492 Add testlib selftest for QStringList comparison.
The (disabled) alive selftest contained a test for the QStringList
specialization of the QTest::qCompare template.  This test is unrelated
to the rest of the alive selftest, so move it to the cmptest selftest,
where QCOMPARE is tested.

Change-Id: Ic6f0e491dd3b3ce8b4ca1d49666a099815575eaa
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-12-14 02:40:17 +01:00
Friedemann Kleint
b9ebb65c77 Revert "Remove unused overload of QTest::qExec."
The overload is used in Qt Creator
(see src/libs/extensionsystem/pluginmanager.cpp).
The use case here is an application whose internal
QObjects can be tested by passing a command line parameter.

For this use case, it is inconvenient to have to allocate
memory and create a char argv[]- array.

This reverts commit ad80d42f8e.

Change-Id: I2a2f91e2840100fd62743f6d03b33005d67b18f8
Reviewed-by: Daniel Teske <daniel.teske@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2011-12-12 10:43:44 +01:00
Friedemann Kleint
dd2b5b88be QTestlib/Selftest: Fix Windows.
- Do not run with empty environment. At least PATH
  is required at least (Qt + MSVC/MinGW runtime)
- Account for MSVC's different formatting of double numbers.

Change-Id: Ic7b1cf4a16a88c5384347b2651b011ac13c92d70
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2011-12-08 08:51:35 +01:00
Jason McDonald
ad80d42f8e Remove unused overload of QTest::qExec.
The overload of QTest::qExec() that takes a QStringList is not used
anywhere in Qt's autotests, despite having been in the qtestlib API
since Qt 4.4.

This lack of use most likely derives from the fact that none of the
QTEST_MAIN macros use the overload, and more than 99% of Qt's tests
use those macros to avoid explicitly calling QTest::qExec().

Change-Id: I264b21d7fe1a9f2d565f748cf8bbe32414a73bb0
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
2011-12-08 03:04:55 +01:00
Jason McDonald
5d74683b7a Don't mark selftest subtests as testcases.
The subtests are intended to be executed by tst_selftests and therefore
don't need to have "CONFIG += testcase", which causes 'make check" to
treat them as stand-alone tests.

Change-Id: I1b47e0186be99c591bb1e03bbec6386518f8a449
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-12-06 02:19:25 +01:00
Jason McDonald
260a2e52b2 Fixing naming of selftests.
Make the test object names correspond to the test program names, so that
it is easier to see which test produced each block of test output.

Change-Id: Ib19a955938b549f27cdcaac83bfa9150285b396d
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-12-05 09:02:15 +01:00
Rohan McGovern
2d8d258f1d Fixed tst_selftests findtestdata subtest for shadow builds.
This test uses qt.conf to enforce a predictable tests install location.
This was not working in the shadow build case, as qt.conf was only
present in the source tree; embed it as a resource at :/qt/etc/qt.conf
so it works in the shadow build case too.

Change-Id: Iec708d9686c29a1fc5ff0020dc11a93a593d20c5
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2011-12-05 06:00:04 +01:00
Rohan McGovern
8481369f3d testlib: add QFINDTESTDATA macro for finding testdata files
Automated tests often need to load some data from external files.
Currently, a wide variety of approaches for this have been used in Qt
autotests, including:

  - embed the source directory into the test binary at compile time, and
    find the testdata relative to that; this fails when the source tree
    is no longer available (e.g. when the tests are deployed to a device).

  - use a path relative to the current working directory, and trust that
    the caller always sets the current working directory such that the
    testdata can be found; this fails when the caller uses a different
    working directory than expected.

  - use a path relative to QCoreApplication::applicationDirPath();
    this fails when source tree != build tree (since testdata is not
    automatically copied into the build tree).

  - compile the files into the binary using the Qt resource system; this
    should work, but does not allow for testing of code which genuinely
    needs external files.

It seems that there is not a simple method for determining the testdata
path which can be reliably used in all circumstances, so various tests
have reinvented the testdata location method in different ways.
Therefore, this is a good candidate for an addition to the testlib API.

The current implementation of QFINDTESTDATA is able to find testdata
in all three of (build tree, install tree, source tree), in that order.

Change-Id: Ib2fed860723ccf437240da3b00db22dfe1a6b56c
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2011-12-01 09:12:57 +01:00
Jędrzej Nowacki
8fd64d22ac Make usage of internal QVariant space.
Each QVariant instance has internal storage which may be
used for well-know basic types. This patch changes the
behavior by delegating type dependent operation to QMetaType
class which knows more types than QVariant itself.

The patch significantly reduce amount of code in QVariant
implementation.

There are few side effects of this patch:
- better performance:
* for Core types when using Gui (QGuiVariant is able to
construct Core types)
* for small custom types (QVariant::Private::Data is used for all
types that has size small enough)
- comparing two QVariants can give different result for small custom
types (binary comparison instead of pointer comparison)

Change-Id: Ic17fa500d6a882110bfba896fd456c8e6c7a63a9
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
2011-11-09 10:11:34 +01:00
Jo Asplin
b68bae1132 Added -datatags option to QTestLib
(Note: This feature is ported from Qt 4.8.
 See the following commits:
  01575deafb7d26ca2431374e92c6d71de96547c7
  4866d1ba8afbab61e102942d1ea93b81fea053d6
)

Passing the -datatags option to a QTestLib program prints the
available data tags to standard output.

For completeness, the test case name is also printed
at the start of each output line. (Although the file name
is supposed to match the lower-case version of the test case
name, this is currently not true in all cases (particularly not
under tests/benchmarks/). Even if there was a script to enforce this
convention, the -datatags option provides this information in a
reliable way.)

Data tags for each test function (f() in this case) are printed in
four different ways depending on the presence of local and global
data tags:

Case 1: No tags:
    tst_MyTestCasetst_MyTestCase f

Case 2: Local tags only:
    tst_MyTestCase f local tag 1
    tst_MyTestCase f local tag 2
    ...

Case 3: Global tags only:
    tst_MyTestCase f __global__ global tag 1
    tst_MyTestCase f __global__ global tag 2
    ...

Case 4: Local and global tags:
    tst_MyTestCase f local tag 1 __global__ global tag 1
    tst_MyTestCase f local tag 2 __global__ global tag 1
    ...
    tst_MyTestCase f local tag 1 __global__ global tag 2
    tst_MyTestCase f local tag 2 __global__ global tag 2
    ...
    ...

Note that the string __global__ is assumed to be highly unlikely to occur
in a data tag (if it does, an ambiguity results).

Change-Id: Ib51aa0c3c32ad52e52ce519729292cf8f0ec5d50
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Reviewed-by: Sergio Ahumada <sergio.ahumada@nokia.com>
2011-11-04 19:23:33 +01:00
Jason McDonald
8cb52795f1 Allow extra selftests without modifying source.
Change-Id: I21708b725611eb64bdb0414cee08ca00dd62b213
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-11-03 02:42:56 +01:00
Rohan McGovern
d04bf8f407 all remaining tests: eliminated usage of qttest_p4.prf
qttest_p4.prf was added as a convenience for Qt's own autotests in Qt4.
It enables various crufty undocumented magic, of dubious value.
Stop using it, and explicitly enable the things from it which we want.

Change-Id: I02fe27b2c1800f929250fa8694ca2976c9661a12
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2011-10-25 08:42:23 +02:00
Jason McDonald
016cd01846 Remove QTest::SkipMode from qtestlib API.
When the SkipAll mode is used, tests only report a SKIP for the first
line of test data and subsequent lines are not reported at all.  This
behaviour makes it impossible for anything post-processing test results
to accurately report test pass- and run- rates because they cannot see
how many lines of test data were skipped.

This commit removes SkipMode.  QSKIPs in regular test functions and data
functions are treated the same as SkipSingle, so that every skipped line
of local or global test data is reported in the test log.  QSKIPs
elsewhere are treated the same as SkipAll -- skipping in init() causes
the next test function to be skipped entirely, and skipping in
initTestCase() or initTestCase_data() causes all test functions to be
skipped.

This commit only changes qtestlib and the selftests. A further commit
will change the autotests to remove the SkipMode parameter from QSKIP
calls.

Note that the change in expected output for the globaldata selftest is
deliberate, as the QSKIP in the skipLocal test function has effectively
changed from SkipAll to SkipSingle.

Task-number: QTBUG-21851, QTBUG-21652
Change-Id: I7b1c53fe7ca9dde032810b789d967e2a402bbe5d
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Alex <alex.blasche@nokia.com>
2011-10-21 01:20:29 +02:00
Jason McDonald
47f016f519 Sanitize paths in selftest data.
Use a standard path for filenames that appear in the selftest's expected
test data.  This will make future patches smaller.

Change-Id: I04b2e739d261f80d20b834e5b33c5b6e88d26379
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-10-19 07:24:40 +02:00
Jason McDonald
af5f17e214 Remove skipglobal testlib selftest.
This test duplicates the skipinitdata selftest and has slightly less
informative output.

Change-Id: Ifd40e3ef8030059ec8fa0089ce5b2a994624abeb
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-10-19 07:24:26 +02:00
Rohan McGovern
be9939ac89 Reduce the runtime of tst_selftests
The addition of testing for multiple loggers greatly increased the total
runtime of this test, in fact making it the slowest testcase in Qt.
Fortunately this is only due to a couple of slow subtests whose behavior
is unlikely to be affected by the loggers.  Change it to run these slow
subtests just for a couple of loggers, instead of all the combinations.

Change-Id: Ie90f6c0ca29470ed6a7c4e2e185f852602a2d162
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
2011-10-18 00:56:54 +02:00
Jason McDonald
cb57d36f45 Make QTest::qWarn() use __FILE__ and __LINE__.
This change makes QTest::qWarn() consistent with similar testlib
functions, which take __FILE__ and __LINE__.

Change-Id: I12977cb30672899ab38877b4a656f0cc0c7ea02c
Reviewed-on: http://codereview.qt-project.org/3559
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-28 12:17:45 +02:00
Jo Asplin
d13afe497a Re-enabled tests that are now passing on Linux
Task-number: QTBUG-21424

Change-Id: I093d90a2713a7e7ddace31a27a08d47e3c2da588
Reviewed-on: http://codereview.qt-project.org/5546
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Matthew Cattell <matthew.cattell@nokia.com>
2011-09-26 18:47:24 +02:00
Jason McDonald
412e6b754f Tidy and sort the list of selftests
Change-Id: I123a775e2d56fc0c64d43e98a1d2e3ab68394076
Reviewed-on: http://codereview.qt-project.org/5492
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-26 07:50:20 +02:00
Jason McDonald
9f883c7571 Use placeholder for Qt version in expected output
The selftests already used the @INSERT_QT_VERSION_HERE@ placeholder
in place of the real Qt and QTestlib versions in the expected test
output of the subtests.  For unknown reasons, the same was not true for
the expected plain text output.  In the past, this has caused Release
Managers to waste time incrementing the version numbers in these files.

Change-Id: I52f7870486fce128c04d53ff06978afa947474fd
Reviewed-on: http://codereview.qt-project.org/5375
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-26 07:50:20 +02:00
Jason McDonald
ce258242b8 Remove obsolete exclusion from selftest checking
There is no test with a datatag "float", so this code is never executed.

Change-Id: I9dd234e6575eb1c33075705edf77eb1d977061c9
Reviewed-on: http://codereview.qt-project.org/5362
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-26 03:03:09 +02:00
Jason McDonald
b211dd93a7 Don't XFAIL on the contents of assert messages.
Q_ASSERT uses __FILE__, which has undefined contents, and thus the
selftest cannot always match Q_ASSERT messages in actual output with
those in expected output.  The test was calling QEXPECT_FAIL() in this
case, but doing so implies that (a) it's a failure and (b) we intend to
fix it one day.  Neither of these is true, so instead we should simply
tolerate this difference in actual and expected outputs, in the same way
that we do in a couple of other similar cases.

Change-Id: I8b8609a400e9b798936875f6eb1ca18a83660efc
Reviewed-on: http://codereview.qt-project.org/5358
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-26 03:02:46 +02:00
Jason McDonald
51589e834e Allow tests to log to multiple destinations
Each destination and the format of output to write there is specified by
adding "-o filename,format" to the command-line.  The special filename
"-" indicates that the log output is written to the standard output
stream, though standard output can be used as a destination at most
once.

The old-style testlib output options are still supported, but can only
be used to specify one logging destination, as before.

If no logging options are given on the command-line, a plain text log
will go to the console, as before.

To log to the console in plain text and to the file "test_output" in
xunit format, one would invoke a test in the following way:

    tst_foo -o test_output,xunitxml -o -,txt

This commit also enhances the selftests to test with multiple loggers,
but negative tests (e.g. bad combinations of command-line options) are
left for future task QTBUG-21567.

Task-number: QTBUG-20615
Change-Id: If91e752bc7001657e15e427aba9d25ab0a29a0b0
Reviewed-on: http://codereview.qt-project.org/4125
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-23 00:34:23 +02:00
Jason McDonald
157fc88f90 Enable testlib self-testing with multiple loggers
Modify the selftest to be able to run each subtest with a list of one or
more test loggers.  The addition of tests that use this capability will
be part of a subsequent commit.

Task-number: QTBUG-20615
Change-Id: Iac3efe8220e8245aa7e5589348d2c86b8034dd28
Reviewed-on: http://codereview.qt-project.org/5292
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-23 00:34:19 +02:00
Jason McDonald
a097825b21 Don't run multiexec test with file logging
The multiexec test runs the same test object five times.  If the -o
option is given, the output file is overwritten by each run of the test
object, meaning that tst_selftest only sees 1/5 of the test output in a
file compared with what it sees on the console.  This makes it
impossible to use the expected output file to verify the test for both
console and file output.  This issue is noted in QTBUG-21561.

Change-Id: I00031a2ea43a7ef78e8317473a089306ec062d8e
Reviewed-on: http://codereview.qt-project.org/5270
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-21 03:59:17 +02:00
Jason McDonald
1fcc6ba3d9 Remove printf calls from warnings selftest
The warnings selftest verifies that qtestlib correctly handles output
sent via qDebug() and friends.  The test had a number of printf calls
that were used to tell the user what output they should expect to see
next, but by bypassing testlib's logging, these made the testlog differ
depending on whether the test log was directed to the console or a file.

The printf calls are no longer needed, as any regressions in qDebug will
be detected by tst_selftests, which compares the output of the warnings
test with a reference copy of the expected output.

Change-Id: I8c3a3237463141fa494c50ca02062760cb583090
Reviewed-on: http://codereview.qt-project.org/5200
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-21 03:59:09 +02:00
Jason McDonald
57368c7037 Improve globaldata testlib selftest.
Make the test use qDebug instead of printf, so that the reference test
output is the same regardless of whether the log goes to the console or
a file.

Also rename the test class, as it was evidently cut-n-pasted from the
subtest selftest without being renamed.

Change-Id: I8df1f0989f2e8c33c1f829ba8110fd740ff70f63
Reviewed-on: http://codereview.qt-project.org/5184
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-20 08:46:27 +02:00
Jason McDonald
01ece201ab Split subtest selftest
This test was attempting to verify two completely unrelated things, so
this commit splits it into two tests.

Also, printf calls are replaced by qDebug so that the test does not
bypass the testlib loggers.

Change-Id: I1a202af38ce2c69690a32d93405ba604ec6cabee
Reviewed-on: http://codereview.qt-project.org/5178
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-20 08:46:18 +02:00
Jason McDonald
d60007744b Remove waitwithoutgui testlib selftest
This test is not useful for finding bugs in qtestlib's logging code,
because it bypasses the qtestlib loggers and doesn't play nice with
tst_selftest.  Neither is this test very useful for finding bugs in
QTest::qWait(), as the test only proves the qWait() terminates, not that
it waits accurately, or even that it waits at all.

Change-Id: Ia5dd7cbaf3a6fbb4e94e54ed155263580e495694
Reviewed-on: http://codereview.qt-project.org/5173
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-20 08:46:07 +02:00
Kent Hansen
b8d656d878 Fix autotest gui/widgets dependencies
This is in preparation of removing testlib's
dependency on QtGui and QtWidgets.

Autotests that need QtWidgets api must
explicitly include it (since the types are no
longer provided by the QtGui master header).

Autotests that don't need QtGui or QtWidgets
api shouldn't link against those libraries.

Change-Id: I2808289068514fcac582808828ad4634e2631733
Reviewed-on: http://codereview.qt-project.org/5093
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
2011-09-19 10:24:25 +02:00
Jason McDonald
a021f26148 Simplify selftest
Don't store separate strings for the logger name and the associated file
suffix -- just use the same string everywhere.

Change-Id: Ie7d1af6bf906b5ac09fbd5fcc5754b68036fb370
Reviewed-on: http://codereview.qt-project.org/5060
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-17 03:22:48 +02:00
Jason McDonald
8a12d5fbf0 Improve cleanup of selftests
The selftest was only attempting to delete the output file at the end of
the entire test run.  This file should actually be deleted after each
subtest to ensure that it does not cause confusing errors if the next
subtest fails to overwrite the output file.

Change-Id: Ia3827926f71a697c6108de3f2b08c969f761d978
Reviewed-on: http://codereview.qt-project.org/5051
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-17 03:22:18 +02:00
Jason McDonald
89c7db5159 Remove obsolete QEXPECT_FAIL calls
The removed calls related to tests that were removed when redundant
logging functionality was removed.

Change-Id: Iaebbc109eaaddb440ae78e1f4a1b4e874a8b2960
Reviewed-on: http://codereview.qt-project.org/5050
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-17 03:22:10 +02:00
Jason McDonald
218f0ca3b0 Improve compliance with Qt coding guidelines
* Use C++ comment delimiters consistently
* Use a space after control flow keywords
* Don't nest "else if" blocks unnecessarily
* Follow indenting rules
* Added comments describing some internal functions

Change-Id: I72fd1eaf5c5f3130945fbd2b64fa59e19ad2913d
Reviewed-on: http://codereview.qt-project.org/5049
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-17 03:21:59 +02:00
Jason McDonald
0676e51793 Fix XFAIL in testlib selftest
Commit 949be8cbdd modified the assert test
case but failed to update the line number of the assert in the expected
test output files.

Change-Id: I0d4f55c0b8bc1de8487dd8c6267aa459000cda26
Reviewed-on: http://codereview.qt-project.org/5048
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-17 03:21:44 +02:00
Jo Asplin
831a160e00 Disabled tests for refactor->master integration
This change disables failing tests to increase the
likelihood that 'make check' terminates with a zero
exit status. This is done to prevent the CI system
from blocking further commits once qtbase#refactor
is merged into qtbase#master.

Failing tests are marked as insignificant (by putting
'CONFIG += insignificant_test' in the .pro file).

Note: This is a temporary measure that needs to be cleaned up
once the refactor->master integration is complete.
Ideally all disabled tests need to be enabled and passing.
All changes will be marked by the string QTBUG-21402 in a comment
for easy location.

Task-number: QTBUG-21402
Change-Id: Ic4400671671f6d9b75b106a6aa4755b20e74378d
Reviewed-on: http://codereview.qt-project.org/4678
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Gunnar Sletta <gunnar.sletta@nokia.com>
2011-09-13 08:25:05 +02:00
Jo Asplin
87ee382419 Moved autotests in category 'testlib' into new directory structure
Task-number: QTBUG-21353

Change-Id: I2e2ad4b7570f80b3e6500821f769e967f0a75027
Reviewed-on: http://codereview.qt-project.org/4438
Reviewed-by: Qt Sanity Bot <qt_sanity_bot@ovi.com>
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
2011-09-12 08:36:15 +02:00