Not waiting until the QPA event queue is processed after
handleMouseEvent() is wrong. Unlike the synchronous sendEvent() calls
these tests most likely utilized earlier, many of the
QWindowSystemInterace functions are asynchronous in the sense that they
just queue the event, delivery will happen when the event dispatcher on
the main thread gets there.
Change-Id: I8197d2dc4805cda684a8279ceb8d4b317f19aba7
Reviewed-by: Michael Brasser <michael.brasser@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Make the various versions of the failure message align consistently so
that it's a little easier to compare the actual and expected values. Of
course, the value won't align nicely unless the "actual" and "expected"
strings are the same length, but at least this commit makes that
consistent across all versions of the message.
Change-Id: If9ce231df3b5d279a06f6458fdb5da0aa4586068
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The statementStr parameter should always be non-null, so assert if it is
null. The description parameter can be null in some cases (particularly
when the verify is successful, and thus no error description is going to
be displayed), so tolerate this.
Change-Id: I87b416d5f3b793bc608cd4aca14a4f7fe7527488
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Previously, verbose (-v2) and XPASS test output showed all QCOMPAREs as
"COMPARE()", making it impossible to see what was compared and difficult
to match the output to the source of a test containing many calls to
QCOMPARE.
This commit changes testlib's internal compare_helper API so that string
representations of the compared expressions are always passed to
QTestResult::compare() when available, and can thus be shown in the
verbose and XPASS output. The XPASS output has also been changed to
state explicitly that the comparison succeeded unexpectedly, bringing it
in line with the XPASS output resulting from a call to QVERIFY.
This commit also changes all calls to compare_helper() to call the
eight-argument version of the function, which simplifies much of the
calling code. The now obsolete four-argument version of
compare_helper() has been changed to output a warning that it is
obsolete. It will be removed once other modules have had some time to
catch up.
The improved XPASS and verbose output is demonstrated by the expectfail
and verbose2 selftests.
Change-Id: I8baa46d5dd30e6c43b26f366c34dc5b64aab5f7c
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Popup widgets steal the keyboard grab stealthily without it being
visible via QWidget::keyboardGrabber(). To more accurately simulate a
real keyboard event, prioritize sending simulated keyboard events to
the active popup widget over QGuiApplication::focusWindow().
Task-number: QTBUG-24326
Change-Id: Id7a75c613d934e24657b521f1684ce7cce92556a
Reviewed-by: Morten Johan Sørvig <morten.sorvig@nokia.com>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
QDoc now has support for Doxygen style commands for italics, bold
and list items. This change applies that change in QDoc to the
actual documentation.
Task-number: QTBUG-24578
Change-Id: I519bf9c29b14092e3ab6067612f42bf749eeedf5
Reviewed-by: Shane Kearns <shane.kearns@accenture.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
Change the help output to show which options only work for plain text
logging.
Change-Id: I39eb7cd0793cdbe553c230334c6cd532b4929f61
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
So this code still compiles with QStringBuilder
QTest::newRow("foo") << foo + bar;
Change-Id: Ie82a21c8fd07ec2d27ad85015aa1a0c4e94700d6
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Previously the silent logging mode suppressed passes, skips and internal
testlib info messages, but did not suppress debugging output, making it
hard to see the fails in a noisy test. This commit changes silent mode
so that it suppresses all output except test failures and fatal errors,
making silent mode truly useful for seeing just the important test
output.
This commit also adds a selftest to verify the behaviour of silent mode.
Change-Id: I75420aead03682306210746a87e2a3b608b58fc6
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The code paths are actually the same, so merge them.
Change-Id: I456b8a9a825cccdd80cd6a52a0a6737cde252363
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
remove "header" and assignmets which are defaults or bogus,
reorder some assignments.
Change-Id: I67403872168c890ca3b696753ceb01c605d19be7
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Call it from qtest_qParseArgs() instead of qExec(). This will allow to
remove the code coverage calls from quicktest.
Both the installation and the saving of code coverage data are now
gathered in testlib.
Change-Id: If3baedb0cff9d894e6f232b947af7e3158703d9d
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
as QChar is actually an ushort and there is no point in taking its
address.
Merge-request: 69
Change-Id: Idcc9d621e5627514ade006aa12a789a88929d48b
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
The signature() method will soon be replaced by a function that
returns a QByteArray rather than a pointer, and calling it will be
more expensive than calling isValid(). Plus, isValid() looks nicer.
Change-Id: I6844988c84dcadf876fc86bc71c8b310a21f88de
Reviewed-by: Olivier Goffart <ogoffart@woboq.com>
Reviewed-by: Sergio Ahumada <sergio.ahumada@nokia.com>
Prior to this commit, a benchmark test could report 0..n passes and 0..m
fails or skips, where n is the number of accumulation iterations used to
collect benchmark data and m is the number of times the test function
was invoked. Depending on the type of benchmark measurer being used,
this could result in a very large volume of test output and inconsistent
pass, fail and skip counts between test runs.
This commit changes the behaviour so that each benchmark test reports
one pass, fail or skip, regardless of the number of iterations used to
collect benchmark data.
This commit also prevents benchmark data being reported in the test
output if the benchmark test failed or skipped, as any benchmark data is
of dubious value in such cases.
The latter change in behaviour requires a minor modification to the
badxml selftest, which now tests quoting of literal strings in xml test
output for both passing and failing benchmarks.
Finally, this commit also adds a new selftest specifically for verifying
correct behaviour for benchmarks that fail or skip.
Task-number: QTBUG-24313
Change-Id: I3426dc659a7511b62fd183a031c7235bc753f497
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Prior to this commit, when an unexpected pass occurred the test output
showed the message "XPASS : tst_foo::function() 'expr' returned FALSE",
where the problem was actually that the expression evaluated to true
when it was expected to be false.
This commit changes the output to make it clear that the expression
evaluated to true unexpectedly.
Task-number: QTBUG-22118
Change-Id: Id22c178073d3b75789675ca37a8ef019029b1f91
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The architecture is detected at compile time based on the predefined
macros from the compiler. Don't use QT_ARCH in .pro, .pri, or .prf
files. The PNG_NO_ASSEMBLER_CODE define from libpng.pri is not present
in the current copy of src/3rdparty/libpng, so no change in
functionality is expected.
The conditional for the SUPPORT_JIT define in pcre.pri is moved to
src/3rdparty/pcre/config.h, again so that we can use the compiler's
predefined macros to detect the architecture at compile time.
Replace QT_ARCH_ARM, QT_ARCH_MIPS, and QT_ARCH_SPARC with their
Q_PROCESSOR_* equivalents.
Replace QT_ARCH_INTEGRITY, QT_ARCH_VXWORKS, and QT_ARCH_WINDOWSCE with
their Q_OS_* equivalents.
Note that this commit also effectively disables the SPARC atomic
implementation. An inline implementation for SPARC needs to be added,
or we remove the current code and instead rely on the GCC intrinsic or
C++11 std::atomic support on SPARC.
Note also that this commit does not remove QT_ARCH from configure or
qconfig.h. This will continue to be set until all Qt 5 projects can be
moved away from using QT_ARCH.
Change-Id: I5de747cc4436d21941329974cff3016970f497b8
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
C++11 supports user-defined string literals, which makes
the C++98-accepted literal string concatenation sequence
"foo"MACRO
illegal under C++11.
The solution is to add whitespace between the string
literal and the macro. For symmetry, this patch adds it
on both sides.
Change-Id: Ie0c698f610986c4d1b12dc2083489043b696936d
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Prior to this commit, the following statement in the qtestlib
documentation was untrue: "If init() fails, the following testfunction
will not be executed, the test will proceed to the next testfunction."
If init() called QSKIP, the test function would be skipped, but if
init() reported a failure, the test function would still be executed
(even though doing so could be unsafe).
This commit makes testlib skip a test function if init() reports a
failure and enhances the selftests to cover skips and fails in both
init() and cleanup().
Task-number: QTBUG-20371
Change-Id: Id1cc8464ae0b8c257ae1b74dbe9189a501f5366b
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
A variadic version of the QSKIP macro has been left in qtestlib for
a transitional period to give upstream projects time to catch up with
the removal of the function from the API. This transitional period is
now finished.
Task-number: QTBUG-21652
Change-Id: Ib1a9158c2efbae05ffd0e69e72879f6387d4b2e8
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Testlib no longer does anything with the test location, and neither do
any of Qt's tests, so the code is no longer needed.
Change-Id: I1a464116179f3e26efadad6901f00f3c33e0ce84
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Some headers and constants are available in MinGW-w64 that are not
available in the official MinGW. STATE_SYSTEM_HASPOPUP and
STATE_SYSTEM_PROTECTED constants are defined if they are not already
defined by including oleacc.h. _CrtSetReportMode is not used and
crtdbg.h is not included when using official MinGW as crtdbg.h is
missing from official MinGW.
Change-Id: Ie7f3f3726a1663d0fdeb6ee17b86873ae3f61860
Reviewed-by: Jonathan Liu <net147@gmail.com>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
For data-driven tests, testlib previously counted one fail or skip for
each data row that failed or skipped, while it counted only one pass
for a test function where all rows passed and counted no passes for a
test function where some rows passed and some rows failed. A similar
problem also existed for benchmark tests, which could run multiple
iterations of the same test, with each fail and skip being counted but
only a single pass being counted for the entire series of iterations.
This commit makes testlib count one pass, fail or skip for each data
row. Test functions that are not data-driven count one result for the
test function, as before. Benchmark tests count one pass, fail or skip
per iteration.
A side-effect of this change is that the test output in plain text, xml
and light xml formats now shows a result for every data row and
benchmark iteration executed, allowing post-processors to correctly
calculate the total number of tests executed. Previously, individual
rows were not shown in the test output if they passed, making such
calculations impossible.
The only change to the xunitxml output format is to correct a bug where
no test result was recorded for a test function if the last data row
was skipped and all other rows passed -- in which case the overall
result should be a pass. Note that there is also a pre-existing bug
in the xunit logger, where no result is reported if all rows are
skipped; that bug is unaffected by this commit.
Task-number: QTBUG-21848
Task-number: QTBUG-22124
Change-Id: I7e17177e10d6e89e55b9684c159bd506f21d002b
Reviewed-by: Ed Baak <ed.baak@nokia.com>
Set QT_TESTCOCOON_ACTIVE environment variable when the coverage is installed
for a test and unset it when the coverage data is saved. Tests that run when
QT_TESTCOCOON_ACTIVE is set are subtests and will not be considered as
stand-alone tests for the coverage.
When a test is run as a subtest its coverage data will not be saved for
itself but for the main test it is merged with. Also its status will not be
reported since only the status of the main test is expected in the test report,
e.g. the test tests/auto/testlib/selftests.
Change-Id: Icfdf99300aae18040e1a3441a8af21f68df4c0db
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
A messagebox will be shown for an assertion in
debug mode. This introduces a need for user
interaction to proceed the execution of the
unit test. Setting the Report mode to debug,
will only print the assertion to stderr.
Change-Id: If8ae80ea96d6608cba77b9c6ca176f97d1680932
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Reviewed-by: Joerg Bornemann <joerg.bornemann@nokia.com>
Previously, ignored messages were only cleared at the end of each test
function, i.e. after all data rows were finished. This meant that if a
data row in a data-driven test function didn't cause all of the expected
messages to be generated, the remaining messages would be carried over
to the next data row. This would result in errors about missing
messages being associated with the last data row rather than with the
correct data row.
This commit makes testlib check for missing ignored messages after
running each data row rather than only doing so after the last data
row.
This commit also adds a regression test to demonstrate that ignored
messages can no longer be carried over from one data row to another.
Change-Id: Ibee51aa6e96866fbcbcb4acee1a8340a86a6a4ba
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Previously, expected failures were only cleared at the end of each test
function, i.e. after all data rows were finished. This meant that if a
data-driven test function called QEXPECT_FAIL and then didn't perform
any further verification steps to trigger the expected failure, the
expected failure would be carried over to the next data row, probably
causing the first verification step in the test function to XPASS (with
a seemingly irrelevant error message) for the next data row.
This commit adds the new function QTestResult::finishedCurrentTestData()
to cleanup after each data row is executed. This function treats calls
to QEXPECT_FAIL without subsequent verification steps as a test failure.
This commit also adds a regression test to demonstrate that expected
failures can no longer be carried over from one data row to another. If
run against the previous version of testlib, the new test would report a
pass instead of an error.
Change-Id: Ida5c7f080815b0dca9531131fed582b0918334cb
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Currently when tests are crashing, aborting or hanging, their status are
reported with a status "Unknown" because SaveCoverageTool is never called
for them. The status of the test given by the coverage tool should be the
same as the one reported in the log output when the test is run. This change
will allow more accuracy in the status reported.
Add QTestResult::setCurrentAppname and QTestResult::currentAppname.
To retrieve the name of the current application running.
Task-number: QTQAINFRA-460
Change-Id: Icc476dc2d6cb28185e5447f1e79da6a8a31cad54
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
After discussion with Liang, I'm reverting it as he requested. This change put every header into the SYNCQT.HEADER_FILES twice for in-source builds, and the qtMODULEversion.h header did not include a path component.
This reverts commit 2fbc45b58bba860abf67fb28aa1319c9f4ededaf
Change-Id: Ie84cef19193ce5e49072f1f67a41140d9d2673b8
Reviewed-by: Liang Qi <liang.qi@nokia.com>
The problem with the old updates is that it was impossible
to send details about the update.
For the Linux implementation to work properly with AT-SPI I need to know
which state changed (currently I only get a generic "state changed" event)
or which part of the text was changed for long texts (imagine a word processor sending updates).
This also gives us more options when updating with events generated from not QObject based objects.
Change-Id: If0b6c83c523092565eb48e79f3f6de5a1b905ea8
Reviewed-by: Jan-Arve Sæther <jan-arve.saether@nokia.com>
The problem in qmltest has now been fixed by adding a dummy column, so this
should be good to re-add. See qtdeclarative change id
I1bfed663fa7b58d71e0cf898db6bd8ae61dae46b.
This reverts commit 963958b4f62513cbc76fb06920de80f7f906f523.
Task-number: QTBUG-23845
Change-Id: I4b6755d3ea97a4624480cc008d280fc8e33743a6
Reviewed-by: Jonas Gastal <jgastal@profusion.mobi>
Reviewed-by: Yunqiao Yin <charles.yin@nokia.com>
These functions are no longer used, as the counters have been moved to
QTestLog.
Change-Id: I7cd13fc92a18920706fcc712804c304f9c333969
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
As in the past, to avoid rewriting various autotests that contain
line-number information, an extra blank line has been inserted at the
end of the license text to ensure that this commit does not change the
total number of lines in the license header.
Change-Id: I311e001373776812699d6efc045b5f742890c689
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The removed function was just a wrapper around QTestLog::ignoreMessage()
and was only called twice (once in qtestcase.cpp and once in
qtdeclarative). Better to just call the desired function directly and
avoid the indirection.
Change-Id: Ib900f35f5d584e8f323cef9b0e7b4f8a507c199e
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
It was put in source tree before.
Task-number: QTBUG-20439
Change-Id: Ib52d9c2e83ae375aad259ddc74138bbc728b3ed0
Reviewed-by: Oswald Buddenhagen <oswald.buddenhagen@nokia.com>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
These defines were there to aid in the commercial
licensing scheme we used long ago, and are no longer needed.
Keep a QT_MODULE(x) define so other modules continue compiling.
Change-Id: I8fd76cd5270df8f14aee746b6cf32ebf7c23fec7
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>
This seems to break a number of tests using the qmltest library.
Functionality will be restored in a future commit, once the breakage has been
identified.
This reverts commit fdedb49b76.
Task-number: QTBUG-23845
Change-Id: I0ca4dbd54132e4250702f06738edbad5e8fc4f36
Reviewed-by: Alan Alpert <alan.alpert@nokia.com>
Previously, the assertation triggered was rather unhelpful:
QFATAL : tst_QHash::qhash_qt4() ASSERT: "d->dataCount <
d->parent->elementCount()" in file qtestdata.cpp, line 88"
We now try a bit harder to be user-friendly.
Change-Id: I2e3a5ae27914d44fc1dc89af2a084e3d798fe221
Reviewed-by: Jonas Gastal <jgastal@profusion.mobi>
Reviewed-by: Giuseppe D'Angelo <dangelog@gmail.com>
Reviewed-by: Jason McDonald <jason.mcdonald@nokia.com>
Be more insistent that tests using the old two-argument version of QSKIP
should be updated. After a grace period the warning will be removed and
incorrect usage of QSKIP will revert to a compilation failure.
Change-Id: Ifa19b856d9f45738bd9d790bb65a8741f965d0f4
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
This change will enable further simplification of QTestResult and
QTestLog in the future, including removing a circular dependency between
the two classes.
The "getter" functions in QTestResult are retained for now, but will be
removed in a future commit, once QtQuickTest has been changed to call
the getters that this commit adds to QTestLog.
This commit is adapted from an unfinished change by Henrik Hartz.
Change-Id: Ife7f80ac6a4310449a4712e96e0bea6c02139a5a
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
The removed function was private API and was not used anywhere in Qt.
Change-Id: Ia262d119ebba30f6968cb7145043475bf88ffa39
Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com>
