qt5base-lts/tests/auto/testlib
Edward Welbourne 55ab276700 Major re-write of generate_expected_output.py
Restructured, separated the canonicalising of output lines out (into
an object that prepares the necessary regexes and replacements)
suppress more, changed the path-stripping to strip qtbase's parent
rather than os.getcwd() and took account of shadow builds (so both
source tree and build tree provide prefixes we want to strip from
paths).  Also cope with $PWD potentially having symlinks in it, where
os.getcwd() is canonical.

It's possible some output might name files elsewhere in the source
tree; these won't be filtered by the prior cwd prefix removal; and, in
any case, the problem with cwd is only that the ancestry of qtbase is
apt to vary; paths relative to there should be consistent between test
runs.  This change shall lead to a one-off rewrite of all expected_*
files; but it should now catch all paths.  By stripping both build
root and source root (when different) it also avoids differences for
those doing out-of-source ("shadow") builds.

In our XML formats, any hyphens in root paths (e.g. I had Qt-5.6 in my
build root's path) got represented by a character entity, confounding
the replacement; so also do replacement that catches this.  We may
discover other character entity subsitutions needed along with this.

Now filtering line numbers and timing information, including benchmark
results; these numbers all get replaced with 0 to avoid noisy diffs.
Also purging dangling hspace, to placate sanity-bot.

The module can now be imported - the code it runs is packaged as a
main() function that a __name__ == '__main__' stanza runs - and all
data is localised to where it's needed, rather than held in globals.
Tidied up and organized the existing regexes.  There are doc-strings;
there is a short usage comment.  Data is localised rather than global
and modern pythonic idioms get used where apt.

Regexes are compiled once instead of repeatedly.  An object looks
after the list of patterns to apply and its construction handles all
anticipated problems.  Failures are mediated by an exception.

The output file now gets written once, instead of twice (once before
editing, then over-write to edit), and Popen uses text mode, so that
write can do the same.  Its command is delivered as an array, avoiding
the need to invoke a shell.

Instead of relying on qmake being in our path (which might give us a
bogus QT_VERSION if the one in path doesn't match our build tree), use
the relative path to qmake - we rely on being run in a specific
directory in the build tree, after all.  Escape dots in the version
properly, so that 51730 doesn't get mistaken for 5.7.0 (for example),
and moved this check later in the sequence (matching a smaller target
makes it more likely to falsely match).

Overtly check we are in the right directory and tell the user what we
actually need, if run from the wrong place.  Simplify handling of the
unsupported use-case for MS-Windows (but note what would be needed for
it).

Change-Id: Ibdff8f8cae173f6c31492648148cc345ae29022b
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
2016-08-29 18:02:35 +00:00
..
qsignalspy Update copyright headers 2015-02-11 06:49:51 +00:00
selftests Major re-write of generate_expected_output.py 2016-08-29 18:02:35 +00:00
testlib.pro Moved autotests in category 'testlib' into new directory structure 2011-09-12 08:36:15 +02:00