The status-file flags and the flags from the test case's source code
must always overwrite extra flags set by bots.
Bug: v8:6924
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I0e2aabb69da7cfb8ba6c1c79bd3851462071a6ac
Reviewed-on: https://chromium-review.googlesource.com/732656
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49001}
This reverts commit 4054cf278f.
Reason for revert: Just exposes existing issues.
Original change's description:
> Revert "[test] Fix win-asan symbolizer path"
>
> This reverts commit 135576ffb6.
>
> Reason for revert: V8 Win32 ASAN failures: https://build.chromium.org/p/client.v8/builders/V8%20Win32%20ASAN/builds/73
>
> It appears these failures were lurking there already, but were hidden because of the bug this CL fixed. Opened https://crbug.com/v8/6953 about these issues.
>
> Original change's description:
> > [test] Fix win-asan symbolizer path
> >
> > This makes the symbolizer path relative, as the absolute paths contain
> > a drive letter + colon on windows. The colon is confused by the
> > sanitizer as an option separator.
> >
> > The test driver changes the cwd to the V8 root dir in each
> > invocation.
> >
> > Bug: chromium:726584
> > Change-Id: Icf4e5a55bba5dec8e59a3dfe3eccdf7224e65c33
> > Reviewed-on: https://chromium-review.googlesource.com/721124
> > Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#48652}
>
> TBR=glider@chromium.org,rnk@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,etienneb@chromium.org
>
> Change-Id: Ic78527950f6a239a03658e042d7244c9781d05db
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: chromium:726584
> Reviewed-on: https://chromium-review.googlesource.com/723825
> Reviewed-by: Eric Holk <eholk@chromium.org>
> Commit-Queue: Eric Holk <eholk@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48653}
TBR=glider@chromium.org,rnk@chromium.org,machenbach@chromium.org,eholk@chromium.org,sergiyb@chromium.org,etienneb@chromium.org
Change-Id: I8ea3b1d74ece09bed4758522f51cbee56a7792e1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:726584
Reviewed-on: https://chromium-review.googlesource.com/725319
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48662}
This reverts commit 135576ffb6.
Reason for revert: V8 Win32 ASAN failures: https://build.chromium.org/p/client.v8/builders/V8%20Win32%20ASAN/builds/73
It appears these failures were lurking there already, but were hidden because of the bug this CL fixed. Opened https://crbug.com/v8/6953 about these issues.
Original change's description:
> [test] Fix win-asan symbolizer path
>
> This makes the symbolizer path relative, as the absolute paths contain
> a drive letter + colon on windows. The colon is confused by the
> sanitizer as an option separator.
>
> The test driver changes the cwd to the V8 root dir in each
> invocation.
>
> Bug: chromium:726584
> Change-Id: Icf4e5a55bba5dec8e59a3dfe3eccdf7224e65c33
> Reviewed-on: https://chromium-review.googlesource.com/721124
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#48652}
TBR=glider@chromium.org,rnk@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,etienneb@chromium.org
Change-Id: Ic78527950f6a239a03658e042d7244c9781d05db
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:726584
Reviewed-on: https://chromium-review.googlesource.com/723825
Reviewed-by: Eric Holk <eholk@chromium.org>
Commit-Queue: Eric Holk <eholk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48653}
This makes the symbolizer path relative, as the absolute paths contain
a drive letter + colon on windows. The colon is confused by the
sanitizer as an option separator.
The test driver changes the cwd to the V8 root dir in each
invocation.
Bug: chromium:726584
Change-Id: Icf4e5a55bba5dec8e59a3dfe3eccdf7224e65c33
Reviewed-on: https://chromium-review.googlesource.com/721124
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48652}
Also fix asan environment for testing on windows.
TBR=sergiyb@chromium.org
Bug: chromium:726584
Change-Id: Ic9e6afa714f4757ad1b0f2ebfa742e742e1c04b9
Reviewed-on: https://chromium-review.googlesource.com/720811
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48583}
The run_test.py tool test selection only expands asterisks at the end of
the test name. This CL introduces glob expansion in test selection
(asterisks are expanded anywhere in the path).
This is useful when tests that belong to the same area have different
prefixes. For example wasm cctests have two different prefixes:
'cctest/test-run-wasm*' and 'cctest/test-wasm*'. With this CL it is
possible to specify the selector 'cctest/*wasm*' to run them all.
R=machenbach@chromium.org
Change-Id: I1c7cc5136b21e71f3eaf69fb98d5dfd77d336e2a
Reviewed-on: https://chromium-review.googlesource.com/609000
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Enrico Bacis <enricobacis@google.com>
Cr-Commit-Position: refs/heads/master@{#47274}
This reverts commit 4fa473cb75.
Reason for revert: Problems when v8 isn't located in a folder called "v8".
Original change's description:
> [test] Don't flatten testcfg globals
>
> This loads each test's testcfg.py as a unique module rather than flattening all into testcfg. Other than accessing LoadTestSuite there should be no references into testcfg files.
>
> Bug: v8:6375
> Change-Id: If863c1b35096b2589111e8091bb7d68f135da674
> Reviewed-on: https://chromium-review.googlesource.com/498807
> Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#45178}
TBR=jkummerow@chromium.org,machenbach@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Bug: v8:6375
Change-Id: I3600b54279c0d98a39475432c5b2163f510153f0
Reviewed-on: https://chromium-review.googlesource.com/500130
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45180}
This loads each test's testcfg.py as a unique module rather than flattening all into testcfg. Other than accessing LoadTestSuite there should be no references into testcfg files.
Bug: v8:6375
Change-Id: If863c1b35096b2589111e8091bb7d68f135da674
Reviewed-on: https://chromium-review.googlesource.com/498807
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#45178}
The test driver fails once in a while with no output when
listing the tests on windows, causing the testing to not
even start.
This should make that less likely if there's a flaky crash
when listing the tests.
BUG=v8:5438
Review-Url: https://codereview.chromium.org/2373043002
Cr-Commit-Position: refs/heads/master@{#39779}
This removes the test-download method for all but
promises-aplus. They all only contain legacy code for
deleting old archives.
Only test262 needs to prepare sources on swarming, which is
moved to a new method, called unconditionally.
All references to --download-data and --download-data-only
in the infrastructure can be removed after this.
BUG=
Review-Url: https://codereview.chromium.org/2227613002
Cr-Commit-Position: refs/heads/master@{#38439}
This makes test outcomes immutable. This has two benefits:
1) It's more robust. Using the |= operator by mistake
wouldn't lead to unwanted mutation.
2) It's faster as now the set reference can be copied again
when copying test cases. Test case copying happens an order
of magnitude more often than rules do apply.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2212333002
Cr-Commit-Position: refs/heads/master@{#38373}
This adds the possibility to address test cases in the
status file with the variant under which the test is running.
This is only allowed in top-level sections.
Example:
[{
'test-case': [PASS, SLOW],
}]
['variant == foo', {
'test-case': [FAIL],
}]
The test case "test-case" is marked as slow in all variants.
Additionally, in variant foo, it'll be expected to fail.
This CL also exemplifies the new feature with test cases
running under the ignition_turbofan variant. The
corresponding legacy flag is deprecated.
BUG=v8:5238
Review-Url: https://codereview.chromium.org/2203013002
Cr-Commit-Position: refs/heads/master@{#38342}
This flag is aiming at shipping the ability to generate optimized code
directly from bytecode (without re-parsing source code). All features
needed to ship such a configuration will be staged behind this flag.
R=hablich@chromium.org,rmcilroy@chromium.org
Review-Url: https://codereview.chromium.org/2174333002
Cr-Commit-Position: refs/heads/master@{#38040}
This adds ignition to the testing variants that are run on all
bots.
Failing tests can only be skipped with the NO_IGNITION
keyword in status files. Existing expectations for the
ignition_turbofan variant are all duplicated and use the
NO_IGNITION keyword as well now.
BUG=v8:4280
LOG=N
Committed: https://crrev.com/132c09ed619f23fb7c6d26a4e3552c703389eabd
Cr-Commit-Position: refs/heads/master@{#35865}
Review-Url: https://codereview.chromium.org/1804003002
Cr-Commit-Position: refs/heads/master@{#35926}
This adds a dedicated flag for enabling the BytecodeGraphBuilder. The
intention is to be explicit when this variant is being tested and to
avoid unnecessary overhead in production code for a configuration that
is not yet shipping.
R=rmcilroy@chromium.org
Review-Url: https://codereview.chromium.org/1925123002
Cr-Commit-Position: refs/heads/master@{#35892}
Reason for revert:
Will resubmit with proper fix and more test skips.
Original issue's description:
> [Ignition] Test ignition on all bots.
>
> This adds ignition to the testing variants that are run on all
> bots.
>
> Failing tests can only be skipped with the NO_IGNITION
> keyword in status files. Existing expectations for the
> ignition_turbofan variant are all duplicated and use the
> NO_IGNITION keyword as well now.
>
> BUG=v8:4280
> LOG=N
> NOTRY=true
TBR=rmcilroy@chromium.org,mstarzinger@chromium.org,yangguo@chromium.org,mythria@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=v8:4280
Review-Url: https://codereview.chromium.org/1930943002
Cr-Commit-Position: refs/heads/master@{#35871}
This adds ignition to the testing variants that are run on all
bots.
Failing tests can only be skipped with the NO_IGNITION
keyword in status files. Existing expectations for the
ignition_turbofan variant are all duplicated and use the
NO_IGNITION keyword as well now.
BUG=v8:4280
LOG=N
NOTRY=true
Review-Url: https://codereview.chromium.org/1804003002
Cr-Commit-Position: refs/heads/master@{#35865}
This adds a new ignition_turbofan testing variant that'll
allow switching turbofan off in the other ignition variant.
The switch will happen in a follow up after reconfiguring
the bots.
BUG=v8:4280
LOG=N
Review URL: https://codereview.chromium.org/1922623003
Cr-Commit-Position: refs/heads/master@{#35790}
Test case objects were sorted without key function, resulting
in random sort order. On sharded builds, the shards are
determined by the sort order and rely on a deterministic
sorting. This led to random cctest and unittest cases being
dropped or executed twice on sharded testers.
TBR=jkummerow@chromium.org, hablich@chromium.org
Review URL: https://codereview.chromium.org/1842673002
Cr-Commit-Position: refs/heads/master@{#35151}
The feature was removed from the bots a while ago. It was
superseeded by the flaky-test detection which reruns tests.
Remaining tests still marked as flaky most certainly pass
since a while.
Referencing all the bugs whose expectations lines get
removed by this.
BUG=v8:3838,v8:3525,v8:3125
LOG=n
Review URL: https://codereview.chromium.org/1802983002
Cr-Commit-Position: refs/heads/master@{#34804}
test262 "negative" test expectations list which exception is thrown. The ES2017
draft specification is very specific about which exception class is thrown
from which path, and V8 works hard to be correct with respect to that spec.
Previously, the test262 test runner would accept any nonzero status code,
such as from a crash, or a FAIL printed out, for a negative test. This
patch makes negative tests check for the right answer using a quick-and-dirty
parsing of the exception printing from d8 to find the exception class.
It invokes d8 in a way to get a status code of 0 from thrown exceptions
so that 'negative' tests aren't actually implemented by negating the output.
Amazingly, this didn't catch any test262 failures, but I verified the extra
checking interactively by changing a negative test to expect a different type
and saw it fail.
BUG=v8:4803
R=machenbach
LOG=Y
Review URL: https://codereview.chromium.org/1766503002
Cr-Commit-Position: refs/heads/master@{#34763}
Changes include:
- better test coverage for builds with snapshot
- write snapshot blobs to buffer instead of test serialization files
- renamed tests
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/1777213002
Cr-Commit-Position: refs/heads/master@{#34657}
The current support for try-catch in the interpreter can handle most of
the cases appearing in our test suite. Also the flag in question did not
detect try-finally constructs. This removes the flag and instead extends
the test expectations.
R=rmcilroy@chromium.org
BUG=v8:4674
LOG=n
Review URL: https://codereview.chromium.org/1631593003
Cr-Commit-Position: refs/heads/master@{#33494}
This removes the above flag definition. The flag is no longer needed as
the default implementation is more than capable of faking presence of
handling of try-catch and try-finally constructs by now.
R=rmcilroy@chromium.org
BUG=v8:4674
LOG=n
Review URL: https://codereview.chromium.org/1603063003
Cr-Commit-Position: refs/heads/master@{#33402}
Now that we support eval in Ignition, remove the fallback for eval checks
and make the flag only fallback on catch blocks.
BUG=v8:4280,v8:4676
LOG=N
Review URL: https://codereview.chromium.org/1595223004
Cr-Commit-Position: refs/heads/master@{#33384}
Adds --turbo to the set of flags run with ignition (making the
configuration pure Ignition+TurboFan except for fallbacks to
full-codegen for catch / eval). Also changes the default
--ignition-filter to allow everything, rather than omit everything.
BUG=v8:4280
LOG=N
Review URL: https://codereview.chromium.org/1537433003
Cr-Commit-Position: refs/heads/master@{#33090}
This will allow moving the test outcome check beyond the
multi-process boundary in a follow up. It'll allow wrapping
more complex test jobs like predicable mode on the multi-
process side, which will make the code easier to maintain.
BUG=
Review URL: https://codereview.chromium.org/1469833002
Cr-Commit-Position: refs/heads/master@{#32373}
This will make sure that message tests cover both the parser and preparser
paths, just as we do for parsing-related cctests.
BUG=v8:4372
LOG=n
Review URL: https://codereview.chromium.org/1469383004
Cr-Commit-Position: refs/heads/master@{#32307}
Adds a blacklist of tests which are currently unsupported or broken in Ignition to
the mjsunit and test262 test status.
Also removes --ignition-script-filter flag, and adds a
--ignition_fallback_on_eval_and_catch flag which fallsback to fullcodegen for
functions which call eval or contain a catch block.
BUG=v8:4280
LOG=N
Review URL: https://codereview.chromium.org/1420963009
Cr-Commit-Position: refs/heads/master@{#31864}
Adds an ignition variant to the test runner and adds support to test262 for
filtering such that only test scripts (not the test harness) get run by
the interpreter.
BUG=v8:4280
LOG=N
Review URL: https://codereview.chromium.org/1379093002
Cr-Commit-Position: refs/heads/master@{#31123}
Now run-tests.py understands "suite/foo/bar" with forward slashes for
command-line test selection on all test suites on all platforms.
Previously, file-based suites like mjsunit also accepted "mjsunit/foo\bar";
that behavior is sacrificed here in favor of unification. For the cctest
suite, OTOH, it wasn't possible on Windows to select specific tests at all.
Original review: https://codereview.chromium.org/1348653003/
This reverts commit 5f44a91059.
NOTRY=true
Review URL: https://codereview.chromium.org/1356613002
Cr-Commit-Position: refs/heads/master@{#30798}
Reason for revert:
mozilla tests are failing on Windows
Original issue's description:
> [test] Fix cctest path separators on Windows
>
> Now run-tests.py understands "suite/foo/bar" with forward slashes for
> command-line test selection on all test suites on all platforms.
>
> Previously, file-based suites like mjsunit also accepted "mjsunit/foo\bar";
> that behavior is sacrificed here in favor of unification. For the cctest
> suite, OTOH, it wasn't possible on Windows to select specific tests at all.
>
> Committed: https://crrev.com/b36cfdb39ae648b49a1396c4f669df9b1f57996c
> Cr-Commit-Position: refs/heads/master@{#30794}
TBR=machenbach@google.com,machenbach@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/1349163002
Cr-Commit-Position: refs/heads/master@{#30795}
Now run-tests.py understands "suite/foo/bar" with forward slashes for
command-line test selection on all test suites on all platforms.
Previously, file-based suites like mjsunit also accepted "mjsunit/foo\bar";
that behavior is sacrificed here in favor of unification. For the cctest
suite, OTOH, it wasn't possible on Windows to select specific tests at all.
Review URL: https://codereview.chromium.org/1348653003
Cr-Commit-Position: refs/heads/master@{#30794}
Use the same path separator for test cases on the cmd-line
on all platforms as in testcase.GetLabel, which is used to
report failures.
BUG=chromium:511215
LOG=n
NOTRY=true
Review URL: https://codereview.chromium.org/1281453003
Cr-Commit-Position: refs/heads/master@{#30045}
This allows variants to be named on test failures (follow
up) and then to be used in the test runner for a repro.
This also speeds up variant iteration for test262 and fixes
a bug with variants for benchmarks.
BUG=chromium:511215
NOTREECHECKS=true
LOG=n
Review URL: https://codereview.chromium.org/1245623005
Cr-Commit-Position: refs/heads/master@{#29899}
There are many test names in the v8 code base that prefix
others, which makes it hard to only run those tests.
BUG=chromium:511215
LOG=n
NOTRY=true
Review URL: https://codereview.chromium.org/1251363002
Cr-Commit-Position: refs/heads/master@{#29809}
Without this change, wildcards always overwrite the outcomes
of more specific rules. Now we always merge.
Review URL: https://codereview.chromium.org/1153073002
Cr-Commit-Position: refs/heads/master@{#28629}
This changes the unittests suite to be run in only the default variant,
because unit tests are (and should be) written to be independent from
global flags.
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/1067863007
Cr-Commit-Position: refs/heads/master@{#28054}
This flag is intended as a staging flag for TurboFan. It serves as a
single flag that always enables a most recent configuration of TurboFan
for test suites and benchmarks, without needing to update test drivers.
R=titzer@chromium.org,machenbach@chromium.org
Review URL: https://codereview.chromium.org/1094573002
Cr-Commit-Position: refs/heads/master@{#27896}
There are separated JS and C stacks on simulators so for some stack
extensive tests (like mozilla/js1_5/extensions/regress-355497) might
cause a C stack overflow and that overflow is not caught by V8. It is
not an issue on real HW. Increasing the C stack also solves the problem
but we have already FLAG_sim_stack_size flag to control the JS stack
size.
This patch makes it possible to add flags to tests conditionally in
.status files.
TEST=mozilla/js1_5/extensions/regress-355497
BUG=v8:3152
LOG=N
Review URL: https://codereview.chromium.org/735723006
Cr-Commit-Position: refs/heads/master@{#25434}
Transformed variant flags into named variants. Now, all combinations of variants can be specified on the command line. The old command-line flags are kept for backwards compatibility on the bots.
Added two new test groups: slow and pass|fail. Both are implemented similar to the flaky test feature and allow to either skip or run tests marked as slow or as pass|fail.
R=yangguo@chromium.org
Review URL: https://codereview.chromium.org/85733003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18062 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
Test expectations can now include outcome: FLAKY.
The test runner can now run a class of tests (flaky|non-flaky|all). All tests are in the non-flaky class that are not marked as FLAKY.
The slash correction for windows is now pulled into the test name method. Currently the progress output on windows contains a mixture of / and \.
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/22381003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16080 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
