eb52d78e90
Prior to this commit, a benchmark test could report 0..n passes and 0..m fails or skips, where n is the number of accumulation iterations used to collect benchmark data and m is the number of times the test function was invoked. Depending on the type of benchmark measurer being used, this could result in a very large volume of test output and inconsistent pass, fail and skip counts between test runs. This commit changes the behaviour so that each benchmark test reports one pass, fail or skip, regardless of the number of iterations used to collect benchmark data. This commit also prevents benchmark data being reported in the test output if the benchmark test failed or skipped, as any benchmark data is of dubious value in such cases. The latter change in behaviour requires a minor modification to the badxml selftest, which now tests quoting of literal strings in xml test output for both passing and failing benchmarks. Finally, this commit also adds a new selftest specifically for verifying correct behaviour for benchmarks that fail or skip. Task-number: QTBUG-24313 Change-Id: I3426dc659a7511b62fd183a031c7235bc753f497 Reviewed-by: Rohan McGovern <rohan.mcgovern@nokia.com> |
||
|---|---|---|
| .. | ||
| auto | ||
| baselineserver | ||
| benchmarks | ||
| global | ||
| manual | ||
| shared | ||
| README | ||
| tests.pro | ||
This directory contains autotests and benchmarks based on QTestlib. In order
to run the autotests reliably, you need to configure a desktop to match the
test environment that these tests are written for.
Linux X11:
* The user must be logged in to an active desktop; you can't run the
autotests without a valid DISPLAY that allows X11 connections.
* The tests are run against a KDE3 or KDE4 desktop.
* Window manager uses "click to focus", and not "focus follows mouse". Many
tests move the mouse cursor around and expect this to not affect focus
and activation.
* Disable "click to activate", i.e., when a window is opened, the window
manager should automatically activate it (give it input focus) and not
wait for the user to click the window.