This makes test outcomes immutable. This has two benefits:
1) It's more robust. Using the |= operator by mistake
wouldn't lead to unwanted mutation.
2) It's faster as now the set reference can be copied again
when copying test cases. Test case copying happens an order
of magnitude more often than rules do apply.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2212333002
Cr-Commit-Position: refs/heads/master@{#38373}
This adds the possibility to address test cases in the
status file with the variant under which the test is running.
This is only allowed in top-level sections.
Example:
[{
'test-case': [PASS, SLOW],
}]
['variant == foo', {
'test-case': [FAIL],
}]
The test case "test-case" is marked as slow in all variants.
Additionally, in variant foo, it'll be expected to fail.
This CL also exemplifies the new feature with test cases
running under the ignition_turbofan variant. The
corresponding legacy flag is deprecated.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2203013002
Cr-Commit-Position: refs/heads/master@{#38342}
Test case objects were sorted without key function, resulting
in random sort order. On sharded builds, the shards are
determined by the sort order and rely on a deterministic
sorting. This led to random cctest and unittest cases being
dropped or executed twice on sharded testers.
TBR=jkummerow@chromium.org, hablich@chromium.org
Review URL: https://codereview.chromium.org/1842673002
Cr-Commit-Position: refs/heads/master@{#35151}
This adds sanitizer-coverage compilation, test-runner
features and post processing.
Sanitizer coverage is expected to be used together with
asan.
During test runner execution, the produced sancov files
are disambiguated and match the pattern:
<executable name>.test.<test id>.sancov.
Two additional scripts are added for merging raw sancov
files and for generating json data containing all
instrumented lines + all covered lines from merged sancov
files. Both scripts use multiprocessing for speed.
The json data will later be uploaded to google storage
for further use, e.g. to show coverage data in rietveld.
Sancov documentation:
http://clang.llvm.org/docs/SanitizerCoverage.html
BUG=chromium:568949
LOG=n
NOTRY=true
TEST=python -m unittest sancov_formatter_test
TEST=python -m unittest sancov_merger_test
Review URL: https://codereview.chromium.org/1737263003
Cr-Commit-Position: refs/heads/master@{#34578}
This will allow moving the test outcome check beyond the
multi-process boundary in a follow up. It'll allow wrapping
more complex test jobs like predicable mode on the multi-
process side, which will make the code easier to maintain.
BUG=
Review URL: https://codereview.chromium.org/1469833002
Cr-Commit-Position: refs/heads/master@{#32373}
The feature sometimes prevents subsequent swarming bots to
delete the work directory on windows.
The data file is not kept on swarming bots anyways, therefore
this switches off the feature completely.
BUG=chromium:535160
LOG=n
Review URL: https://codereview.chromium.org/1468933002
Cr-Commit-Position: refs/heads/master@{#32168}
Without this change, wildcards always overwrite the outcomes
of more specific rules. Now we always merge.
Review URL: https://codereview.chromium.org/1153073002
Cr-Commit-Position: refs/heads/master@{#28629}
There are separated JS and C stacks on simulators so for some stack
extensive tests (like mozilla/js1_5/extensions/regress-355497) might
cause a C stack overflow and that overflow is not caught by V8. It is
not an issue on real HW. Increasing the C stack also solves the problem
but we have already FLAG_sim_stack_size flag to control the JS stack
size.
This patch makes it possible to add flags to tests conditionally in
.status files.
TEST=mozilla/js1_5/extensions/regress-355497
BUG=v8:3152
LOG=N
Review URL: https://codereview.chromium.org/735723006
Cr-Commit-Position: refs/heads/master@{#25434}
Add random seed to run-tests.py, using either a user supplied
value or a random number generated by random.SystemRandom().
This same random seed is passed to all test cases, making sure
that we can easily reproduce test failures that depend on
random numbers (i.e. bugs related to our handwritten ASLR).
Also fix all uses of rand() to make use of our RNG class
instead.
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/231443002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20637 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
