Tests are queued retaining the order of testing variants and test suites. This reorders the variants and suites to make sure the slowest run first.
Currently the debugger suite contains the slowest tests.
BUG=v8:5861
TBR=jochen@chromium.org,yangguo@chromium.org,jkummerow@chromium.org
Change-Id: Idae349b5e6db2540c6181f9b5e2ffb689733846e
Reviewed-on: https://chromium-review.googlesource.com/439311
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Jochen Eisinger <jochen@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43046}
This enables {FLAG_debug_code} by default in debug builds. The advantage
is that generated code contained within the snapshot will contain such
debug code. Before we would only get coverage for these pieces with the
no-snapshot builds, which have a meager coverage. One can still pass the
inverse --no-debug-code flag to ensure generated code remains readable
within debug builds as well.
R=machenbach@chromium.org
Review-Url: https://codereview.chromium.org/2528913002
Cr-Commit-Position: refs/heads/master@{#41451}
With this change, WebAssembly.Memory objects have backing stores allocated as an
8GB region where everything beyond the size of the Wasm heap is inaccessible.
GrowMemory is now implemented by changing the protection on the guard regions to
make the new portions of the heap accessible.
Guard pages are not enabled by default, but this change adds a flag and a test
variant to make sure we get test coverage on them.
BUG= https://bugs.chromium.org/p/v8/issues/detail?id=5277
Review-Url: https://codereview.chromium.org/2396433008
Cr-Commit-Position: refs/heads/master@{#41089}
The original reason for the extra output on windows is
obsolete since a while. Now the extra output just spams
the logs and causes traffic.
BUG=chromium:485932
Review-Url: https://codereview.chromium.org/2452763003
Cr-Commit-Position: refs/heads/master@{#40580}
This makes auto-detect-build-configs work on bots (i.e.
if --buildbot was passed).
BUG=v8:5533
CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux64_msan_rel,v8_linux64_tsan_rel,v8_linux_arm64_dbg,v8_linux_gc_stress_dbg,v8_linux_noi18n_rel_ng,v8_linux_nosnap_rel
Review-Url: https://chromiumcodereview.appspot.com/2431653002
Cr-Commit-Position: refs/heads/master@{#40424}
This target introduces a dependency on WebKit/platform which is not the way we want it.
Keeping the build files in src/inspector, as they are not used and will be useful later on.
BUG=none
NOPRESUBMIT=true
Review-Url: https://codereview.chromium.org/2239603002
Cr-Commit-Position: refs/heads/master@{#38587}
This removes the test-download method for all but
promises-aplus. They all only contain legacy code for
deleting old archives.
Only test262 needs to prepare sources on swarming, which is
moved to a new method, called unconditionally.
All references to --download-data and --download-data-only
in the infrastructure can be removed after this.
BUG=
Review-Url: https://codereview.chromium.org/2227613002
Cr-Commit-Position: refs/heads/master@{#38439}
This adds the possibility to address test cases in the
status file with the variant under which the test is running.
This is only allowed in top-level sections.
Example:
[{
'test-case': [PASS, SLOW],
}]
['variant == foo', {
'test-case': [FAIL],
}]
The test case "test-case" is marked as slow in all variants.
Additionally, in variant foo, it'll be expected to fail.
This CL also exemplifies the new feature with test cases
running under the ignition_turbofan variant. The
corresponding legacy flag is deprecated.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2203013002
Cr-Commit-Position: refs/heads/master@{#38342}
This prepares for adding the ignition_staging variant to the
default variant set.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2205883002
Cr-Commit-Position: refs/heads/master@{#38257}
It should be enough to specify --variants=... with the
desired set of variants. The stress/nostress shortcuts
might get confusing soon, as we intend to move the stress
option out of the developer default set.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2195373002
Cr-Commit-Position: refs/heads/master@{#38248}
This introduces aliases for testing variants that can be
combined with other variant names. E.g. --variants=dev,foo
would run the three developer default variants and variant
foo.
We'll have three stages: "dev" for variants to be run by
default on developer workstations, "more" for additional
variants, executed on all bots, "extra" for additional
variants executed on a subset of bots (e.g. not on very slow
or otherwise resource-limited bots).
BUG=v8:5238
NOTRY=true
Review-Url: https://codereview.chromium.org/2196223002
Cr-Commit-Position: refs/heads/master@{#38214}
The flag was introduced for ignition development. It can only
be used when running ignition tests in isolation on the bots.
The bots only use ignition_turbo in isolation since a while
and don't pass the --ignition flag anymore.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2197123002
Cr-Commit-Position: refs/heads/master@{#38206}
First step of deprecating the dedicated ignition alias.
Next it will be changed on the bots to use the bot_default
suite. Then we'll delete it.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2194153002
Cr-Commit-Position: refs/heads/master@{#38202}
Having presubmit called from within the test runner often
requires logic to remove the call again.
After the GN transition it would be better if presubmit is
called by a wrapper script if needed at all. It is run
on upload and on the tryservers anyways.
BUG=chromium:474921
Review-Url: https://codereview.chromium.org/2114653002
Cr-Commit-Position: refs/heads/master@{#37447}
This executes an action as part of the build, writing a json
configuration that includes all build flags relevant to v8
testing.
The test runner will derive all build-dependent flags from
the file if it detects it.
BUG=chromium:474921
Review-Url: https://codereview.chromium.org/2106423002
Cr-Commit-Position: refs/heads/master@{#37446}
This adds ignition to the testing variants that are run on all
bots.
Failing tests can only be skipped with the NO_IGNITION
keyword in status files. Existing expectations for the
ignition_turbofan variant are all duplicated and use the
NO_IGNITION keyword as well now.
BUG=v8:4280
LOG=N
Committed: https://crrev.com/132c09ed619f23fb7c6d26a4e3552c703389eabd
Cr-Commit-Position: refs/heads/master@{#35865}
Review-Url: https://codereview.chromium.org/1804003002
Cr-Commit-Position: refs/heads/master@{#35926}
Reason for revert:
Will resubmit with proper fix and more test skips.
Original issue's description:
> [Ignition] Test ignition on all bots.
>
> This adds ignition to the testing variants that are run on all
> bots.
>
> Failing tests can only be skipped with the NO_IGNITION
> keyword in status files. Existing expectations for the
> ignition_turbofan variant are all duplicated and use the
> NO_IGNITION keyword as well now.
>
> BUG=v8:4280
> LOG=N
> NOTRY=true
TBR=rmcilroy@chromium.org,mstarzinger@chromium.org,yangguo@chromium.org,mythria@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=v8:4280
Review-Url: https://codereview.chromium.org/1930943002
Cr-Commit-Position: refs/heads/master@{#35871}
This adds ignition to the testing variants that are run on all
bots.
Failing tests can only be skipped with the NO_IGNITION
keyword in status files. Existing expectations for the
ignition_turbofan variant are all duplicated and use the
NO_IGNITION keyword as well now.
BUG=v8:4280
LOG=N
NOTRY=true
Review-Url: https://codereview.chromium.org/1804003002
Cr-Commit-Position: refs/heads/master@{#35865}
This adds a new ignition_turbofan testing variant that'll
allow switching turbofan off in the other ignition variant.
The switch will happen in a follow up after reconfiguring
the bots.
BUG=v8:4280
LOG=N
Review URL: https://codereview.chromium.org/1922623003
Cr-Commit-Position: refs/heads/master@{#35790}
The slow_archs list had only an effect when no timeout was
passed. Now it will also increase the timeout on the bots.
Also simplified the code a bit.
Review URL: https://codereview.chromium.org/1866953002
Cr-Commit-Position: refs/heads/master@{#35355}
The feature was removed from the bots a while ago. It was
superseeded by the flaky-test detection which reruns tests.
Remaining tests still marked as flaky most certainly pass
since a while.
Referencing all the bugs whose expectations lines get
removed by this.
BUG=v8:3838,v8:3525,v8:3125
LOG=n
Review URL: https://codereview.chromium.org/1802983002
Cr-Commit-Position: refs/heads/master@{#34804}
This also runs the message test suite against Ignition. By now most of
the source positions (and exception messages) are accurate, the failing
ones have been blacklisted.
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/1783773003
Cr-Commit-Position: refs/heads/master@{#34708}
Changes include:
- better test coverage for builds with snapshot
- write snapshot blobs to buffer instead of test serialization files
- renamed tests
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/1777213002
Cr-Commit-Position: refs/heads/master@{#34657}
With this, the test runner automatically merges sancov
files after testing. There's no need to do this by some
external infrastructure.
In a future CL, we could even merge during testing to lift
harddisk pressure.
BUG=chromium:568949
LOG=n
NOTRY=true
Review URL: https://codereview.chromium.org/1776123002
Cr-Commit-Position: refs/heads/master@{#34606}
This adds sanitizer-coverage compilation, test-runner
features and post processing.
Sanitizer coverage is expected to be used together with
asan.
During test runner execution, the produced sancov files
are disambiguated and match the pattern:
<executable name>.test.<test id>.sancov.
Two additional scripts are added for merging raw sancov
files and for generating json data containing all
instrumented lines + all covered lines from merged sancov
files. Both scripts use multiprocessing for speed.
The json data will later be uploaded to google storage
for further use, e.g. to show coverage data in rietveld.
Sancov documentation:
http://clang.llvm.org/docs/SanitizerCoverage.html
BUG=chromium:568949
LOG=n
NOTRY=true
TEST=python -m unittest sancov_formatter_test
TEST=python -m unittest sancov_merger_test
Review URL: https://codereview.chromium.org/1737263003
Cr-Commit-Position: refs/heads/master@{#34578}
Add control for the ignition test suites. This will be
followed by a change on the infra side that targets the
ignition set instead of the hardcoded mjsunit.
After that, cctest can be enabled in a separate CL. This CL
already includes the logic for running cctest for ignition
on swarming.
BUG=v8:4280
LOG=N
Review URL: https://codereview.chromium.org/1559943002
Cr-Commit-Position: refs/heads/master@{#33089}
