qt5base-lts/tests/auto
Edward Welbourne 55ab276700 Major re-write of generate_expected_output.py
Restructured, separated the canonicalising of output lines out (into
an object that prepares the necessary regexes and replacements)
suppress more, changed the path-stripping to strip qtbase's parent
rather than os.getcwd() and took account of shadow builds (so both
source tree and build tree provide prefixes we want to strip from
paths).  Also cope with $PWD potentially having symlinks in it, where
os.getcwd() is canonical.

It's possible some output might name files elsewhere in the source
tree; these won't be filtered by the prior cwd prefix removal; and, in
any case, the problem with cwd is only that the ancestry of qtbase is
apt to vary; paths relative to there should be consistent between test
runs.  This change shall lead to a one-off rewrite of all expected_*
files; but it should now catch all paths.  By stripping both build
root and source root (when different) it also avoids differences for
those doing out-of-source ("shadow") builds.

In our XML formats, any hyphens in root paths (e.g. I had Qt-5.6 in my
build root's path) got represented by a character entity, confounding
the replacement; so also do replacement that catches this.  We may
discover other character entity subsitutions needed along with this.

Now filtering line numbers and timing information, including benchmark
results; these numbers all get replaced with 0 to avoid noisy diffs.
Also purging dangling hspace, to placate sanity-bot.

The module can now be imported - the code it runs is packaged as a
main() function that a __name__ == '__main__' stanza runs - and all
data is localised to where it's needed, rather than held in globals.
Tidied up and organized the existing regexes.  There are doc-strings;
there is a short usage comment.  Data is localised rather than global
and modern pythonic idioms get used where apt.

Regexes are compiled once instead of repeatedly.  An object looks
after the list of patterns to apply and its construction handles all
anticipated problems.  Failures are mediated by an exception.

The output file now gets written once, instead of twice (once before
editing, then over-write to edit), and Popen uses text mode, so that
write can do the same.  Its command is delivered as an array, avoiding
the need to invoke a shell.

Instead of relying on qmake being in our path (which might give us a
bogus QT_VERSION if the one in path doesn't match our build tree), use
the relative path to qmake - we rely on being run in a specific
directory in the build tree, after all.  Escape dots in the version
properly, so that 51730 doesn't get mistaken for 5.7.0 (for example),
and moved this check later in the sequence (matching a smaller target
makes it more likely to falsely match).

Overtly check we are in the right directory and tell the user what we
actually need, if run from the wrong place.  Simplify handling of the
unsupported use-case for MS-Windows (but note what would be needed for
it).

Change-Id: Ibdff8f8cae173f6c31492648148cc345ae29022b
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-08-29 18:02:35 +00:00
..
android Fix typo in test results report for android 2015-07-28 21:16:19 +00:00
bic/data bic: Add 5.{1,2,3,4}.0 bic data for linux-gcc-amd64 2015-02-14 06:54:20 +00:00
cmake Enable cmake auto tests for -no-gui 2016-03-30 07:16:50 +00:00
compilerwarnings/data Fixed license headers 2015-02-17 18:46:09 +00:00
concurrent Fix problem with exception reporting in QFuture::waitForResult() 2016-08-23 14:55:49 +00:00
corelib QMimeType: use default locale rather than system locale 2016-08-24 20:30:13 +00:00
dbus Fix race condition in tst_QDBusConnection::pendingCallWhenDisconnected 2016-06-06 11:39:53 +00:00
gui QTextDocument: fix string backward search 2016-08-28 15:18:55 +00:00
guiapplauncher Fix path to examples part of tst_GuiAppLauncher 2015-02-13 07:28:15 +00:00
installed_cmake Add a unit test for testing Qt in its install location. 2013-07-02 23:08:54 +02:00
network tst_QSslSocket::setLocalCertificateChain(): fix resource leak when test fail 2016-08-11 04:34:19 +00:00
opengl Merge remote-tracking branch 'origin/5.6.1' into 5.6 2016-06-02 06:01:25 +02:00
other Remove unneeded dependency on Qt widgets 2016-08-15 08:40:54 +00:00
printsupport QMacPrintEngine: Really set the printer resolution 2016-02-25 18:42:47 +00:00
shared Update copyright headers 2015-02-11 06:49:51 +00:00
sql Fix UNSIGNED values in QMYSQL 2016-06-14 11:13:11 +00:00
testlib Major re-write of generate_expected_output.py 2016-08-29 18:02:35 +00:00
tools Always generate size hint for spacer items 2016-08-08 07:55:57 +00:00
widgets QAbstractSpinBox: Clear 'cleared' flag on receiving a keypress 2016-08-25 17:29:51 +00:00
xml Fix QDomNamedNodeMap::item crash with negative index 2015-10-29 08:05:45 +00:00
auto.pro Enable cmake auto tests for -no-gui 2016-03-30 07:16:50 +00:00
network-settings.h tst_QTcpSocket, tst_QUdpSocket: Improve diagnostics. 2016-05-02 19:20:33 +00:00
qtest-config.h Update copyright headers 2015-02-11 06:49:51 +00:00
test.pl Update copyright headers 2015-02-11 06:49:51 +00:00