v8/tools/testrunner/outproc
Michael Achenbach 3212b90600 [numfuzz] Gracefully ignore contradictory flags
NumFuzz passes various flags to V8 testing randomly, which can lead to
various flag contradictions with existing flags. Up to now the system
ignored the check for contradictions and kept running the test cases,
leading to false positives.

This change adds a new v8 flag --exit-on-contradictory-flags that
exists gracefully when a contradiction is detected. On the numfuzz
side we now filter simple contradictions beforehand.

Measurements showed that ~2% of all numfuzz tests ran into
contradictions. Around half of them are simple contradictions
(repetitions and inversions), which are now filtered beforehand.
The remaining ones (redundant or contradictory implications) are
now ignored.

Bug: v8:11826
Change-Id: I9942e203ba9668a097fabe1343dd1365c9da94c1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3650746
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Tobias Tebbi <tebbi@chromium.org>
Reviewed-by: Almothana Athamneh <almuthanna@chromium.org>
Cr-Commit-Position: refs/heads/main@{#80589}
2022-05-17 13:09:07 +00:00
..
__init__.py [test] Check output on the worker process. 2018-01-04 14:12:31 +00:00
base.py Reland "[tools] Clean up py2 code" 2022-03-25 12:14:26 +00:00
message.py Reland "[tools] Clean up py2 code" 2022-03-25 12:14:26 +00:00
mkgrokdump.py Python2 compatibility: fix flake8 report F632 F821 2019-04-05 16:24:19 +00:00
mozilla.py [test] Check output on the worker process. 2018-01-04 14:12:31 +00:00
test262.py [tools] Generate additional variant for FAIL_PHASE_ONLY tests 2018-12-13 11:28:57 +00:00
webkit.py [numfuzz] Gracefully ignore contradictory flags 2022-05-17 13:09:07 +00:00