This is a reland of 0db74d4974.
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
Bug: v8:6917
Change-Id: I1ea376a4abffce5ab65f4834ea7e6d6011765ffa
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894204
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50978}
This reverts commit 0db74d4974.
Reason for revert: https://chromium-swarm.appspot.com/task?id=3b609f9976bac610&refresh=10&show_raw=1
Original change's description:
> [test] Random seed processor
>
> 1. --total-timeout-sec now available for ./run-tests.py. It can be
> useful with infinite seed stressing
> 2. random seed dropped from the context. Now JSON progress indicator
> gets it from the list of command args.
>
> Bug: v8:6917
> Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
> Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
> Reviewed-on: https://chromium-review.googlesource.com/888524
> Commit-Queue: Michał Majewski <majeski@google.com>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50964}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I2d96ea328cda2d09b01ff455e47c77d567fafe00
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/894522
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50970}
1. --total-timeout-sec now available for ./run-tests.py. It can be
useful with infinite seed stressing
2. random seed dropped from the context. Now JSON progress indicator
gets it from the list of command args.
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I73e535bc8face9b913c696b8d5e3a246fa231004
Reviewed-on: https://chromium-review.googlesource.com/888524
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50964}
It was shipped in Chrome 63.
Bug: v8:5855
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Icc00b8300622d1c7b5662be8ac5e425b9781f666
Reviewed-on: https://chromium-review.googlesource.com/858381
Commit-Queue: Adam Klein <adamk@chromium.org>
Reviewed-by: Sathya Gunasekaran <gsathya@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50558}
This is just a rebased version of
https://chromium-review.googlesource.com/c/v8/v8/+/571453 with
no functional changes
Bug: v8:6889
Cq-Include-Trybots: luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Ia082cc09ca527505b288ac88e68e0b74eae94765
Reviewed-on: https://chromium-review.googlesource.com/849423
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Sathya Gunasekaran <gsathya@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50417}
I added additional exception logging in the execution.py since
errors in processing results were really difficult to debug.
There is a problem on Windows with class serialization when
it comes from dynamically loaded module. To fix it I moved all
output processors to the tools/testrunner/outproc/ and import
them in test/*/testcfg.py.
Bug: v8:6917
Change-Id: Ida604641d659b006e91faf1d56a37769ec47f5f3
Reviewed-on: https://chromium-review.googlesource.com/842784
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50361}
This reverts commit 1685b5d27a.
Reason for revert: Was probably caused by infra change:
https://crrev.com/c/845781
Original change's description:
> Revert "[test] Move has unexpected output to outproc."
>
> This reverts commit 71605b3ea4.
>
> Reason for revert: Seems to break static-initializers step:
> https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
>
> Original change's description:
> > [test] Move has unexpected output to outproc.
> >
> > Expected outcomes optimized to serialize [PASS] as None.
> >
> > Keeping expected outcomes inside output processors should be
> > optimized in the future. Few possible optimizations:
> > - separate classes for tests that are expected to PASS - done as
> > an example in mozilla test suite.
> > - cache output processors inside testcase.
> > - share output processors between copies of the same test - needs
> > some updates to the create_variant to update outproc only if
> > expected outcomes changed.
> >
> > Bug: v8:6917
> > Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> > Reviewed-on: https://chromium-review.googlesource.com/843025
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> > Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#50347}
>
> TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
>
> Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: v8:6917
> Reviewed-on: https://chromium-review.googlesource.com/849572
> Reviewed-by: Bill Budge <bbudge@chromium.org>
> Commit-Queue: Bill Budge <bbudge@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50348}
TBR=bbudge@chromium.org,machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: I7a522b6487de6e96985d223524533493eb9171f0
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/848975
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50350}
This reverts commit 71605b3ea4.
Reason for revert: Seems to break static-initializers step:
https://build.chromium.org/p/client.v8/builders/V8%20Linux64/builds/22156
Original change's description:
> [test] Move has unexpected output to outproc.
>
> Expected outcomes optimized to serialize [PASS] as None.
>
> Keeping expected outcomes inside output processors should be
> optimized in the future. Few possible optimizations:
> - separate classes for tests that are expected to PASS - done as
> an example in mozilla test suite.
> - cache output processors inside testcase.
> - share output processors between copies of the same test - needs
> some updates to the create_variant to update outproc only if
> expected outcomes changed.
>
> Bug: v8:6917
> Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
> Reviewed-on: https://chromium-review.googlesource.com/843025
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#50347}
TBR=machenbach@chromium.org,sergiyb@chromium.org,majeski@google.com
Change-Id: Ice1f3aee0a26f7f38996459d38fd6e0bd964113d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6917
Reviewed-on: https://chromium-review.googlesource.com/849572
Reviewed-by: Bill Budge <bbudge@chromium.org>
Commit-Queue: Bill Budge <bbudge@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50348}
Expected outcomes optimized to serialize [PASS] as None.
Keeping expected outcomes inside output processors should be
optimized in the future. Few possible optimizations:
- separate classes for tests that are expected to PASS - done as
an example in mozilla test suite.
- cache output processors inside testcase.
- share output processors between copies of the same test - needs
some updates to the create_variant to update outproc only if
expected outcomes changed.
Bug: v8:6917
Change-Id: Ie73f1dcdf17fdfc65bce27228f818b1dd1e420c9
Reviewed-on: https://chromium-review.googlesource.com/843025
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50347}
This patch implements https://github.com/tc39/proposal-class-fields/pull/65
and https://github.com/tc39/proposal-static-class-features/ by
splitting out instance and static field declarations into separate
flags for the separate proposals. Instance class fields is currently
at Stage 3 whereas static class fields is currently at Stage 2.
Bug: v8:5367
Change-Id: I133c945fd0b22dc5718c7bb61b10f22348087acd
Reviewed-on: https://chromium-review.googlesource.com/839778
Commit-Queue: Daniel Ehrenberg <littledan@chromium.org>
Reviewed-by: Sathya Gunasekaran <gsathya@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50293}
All get outcome logic moved out from testsuite which makes output
processors serializable. Output processors that are not global
are stored inside testcase to share them with all test copies.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I625466b2b00565d885556f96cab03d8d16fa7ff4
Reviewed-on: https://chromium-review.googlesource.com/837069
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50275}
Making test262's GetExpectedOutcomes independent from a command.
It will enable us to move this function to precompute phase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: Ibb59da29bd30f32427ba6230a22d7d0801954bf0
Reviewed-on: https://chromium-review.googlesource.com/828933
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50127}
- All testcase/testsuite/variant generator subclasses renamed to
just TestCase/TestSuite/VariantGenerator since they're private
implementation.
- All `testcase` variables renamed to `test` to not conflict with
a module name.
- No more two statements in the same line.
- Removed some unused testsuite methods.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I7710f3419f738a5f9ddca73765dd2cad2e35b952
Reviewed-on: https://chromium-review.googlesource.com/823964
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50076}
- create testcase subclass for each test
- move get_command, get_source from suite to test
- promises-aplus tests are broken
- moving expected outcomes etc. is still in progress
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I006e64ecf51dda95c41dff88c68b24f17a638566
Reviewed-on: https://chromium-review.googlesource.com/798331
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50065}
Includes drive-by fix of a small BigInt bug, as caught by
test262/built-ins/BigInt/constructor-from-string-syntax-errors
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: Ic3b78310912f84bbf904a1fcb7ddf2d7eb2df013
Reviewed-on: https://chromium-review.googlesource.com/817775
Reviewed-by: Sathya Gunasekaran (ooo until 12/12) <gsathya@chromium.org>
Reviewed-by: Georg Neis <neis@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Daniel Ehrenberg <littledan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50010}
Filtering by status file split to four parts:
1. Getting outcomes - reads both variant dependent and
independent outcomes, no more need to do it twice.
2. Checking unused rules - has a switch to check only variant
dependent/independent rules.
3. Reading flags - if outcome starts with '--' it is treated as a flag.
4. Actual filtering.
Outcomes removed from the testcase object, can be accessed
by call to its testsuite.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I35762f891010ddda926250452b88656047433daa
Reviewed-on: https://chromium-review.googlesource.com/775160
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49547}
Both of these features were shipped in Chrome 62.
Bug: v8:4545, v8:6172
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: Ie00dcbeded7517a15696d4a78fcfbbf162919923
Reviewed-on: https://chromium-review.googlesource.com/775601
Reviewed-by: Jakob Gruber <jgruber@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49453}
The status-file flags and the flags from the test case's source code
must always overwrite extra flags set by bots.
Bug: v8:6924
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I0e2aabb69da7cfb8ba6c1c79bd3851462071a6ac
Reviewed-on: https://chromium-review.googlesource.com/732656
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49001}
This flag has been on by default since Chrome 61.
Bug: v8:5549
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I81c34d1d3a7dbd219acce2cdf0cf4917eb484002
Reviewed-on: https://chromium-review.googlesource.com/738312
Reviewed-by: Sathya Gunasekaran <gsathya@chromium.org>
Commit-Queue: Adam Klein <adamk@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48945}
This reverts commit d0651bd108.
Reason for revert: Breaks gc stress with embedded snapshot:
https://build.chromium.org/p/client.v8/builders/V8%20Linux64%20GC%20Stress%20-%20custom%20snapshot/builds/15355
Original change's description:
> [language] Implement optional catch binding proposal
>
> This allows the syntax `try {} catch {}` (with no binding after the
> `catch`).
>
> See https://github.com/michaelficarra/optional-catch-binding-proposal/
>
> Currently behind --harmony-optional-catch-binding.
>
> As part of the implementation, this allows TryCatchStatements to not
> have an associated catch scope; various paths which assumed they
> would have been updated to handle this case.
>
> Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
> Change-Id: Ic525b45199eef025eb05da562e10fbd4f3d7465f
> Reviewed-on: https://chromium-review.googlesource.com/571453
> Reviewed-by: Marja Hölttä <marja@chromium.org>
> Reviewed-by: Adam Klein <adamk@chromium.org>
> Reviewed-by: Sathya Gunasekaran <gsathya@chromium.org>
> Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
> Commit-Queue: Kevin Gibbons <bakkot@gmail.com>
> Cr-Commit-Position: refs/heads/master@{#48300}
TBR=rmcilroy@chromium.org,adamk@chromium.org,marja@chromium.org,gsathya@chromium.org,bakkot@gmail.com
Change-Id: I63d68160ec75b87e28d3dcdddca2d8b7d0503b46
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/702334
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48303}
This allows the syntax `try {} catch {}` (with no binding after the
`catch`).
See https://github.com/michaelficarra/optional-catch-binding-proposal/
Currently behind --harmony-optional-catch-binding.
As part of the implementation, this allows TryCatchStatements to not
have an associated catch scope; various paths which assumed they
would have been updated to handle this case.
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: Ic525b45199eef025eb05da562e10fbd4f3d7465f
Reviewed-on: https://chromium-review.googlesource.com/571453
Reviewed-by: Marja Hölttä <marja@chromium.org>
Reviewed-by: Adam Klein <adamk@chromium.org>
Reviewed-by: Sathya Gunasekaran <gsathya@chromium.org>
Reviewed-by: Ross McIlroy <rmcilroy@chromium.org>
Commit-Queue: Kevin Gibbons <bakkot@gmail.com>
Cr-Commit-Position: refs/heads/master@{#48300}
Includes a drive-by fix to a couple of superficial Intl changes
With this roll, test262 starts to look at test262 feature
flags to determine which harmony flags to turn on. There's
still more to do, including adding feature flags to existing
upstream tests and taking advantage of more flags here.
Change-Id: I9cb813e0450be9dc7769ac9c601092bd3572556f
Reviewed-on: https://chromium-review.googlesource.com/471546
Reviewed-by: Adam Klein <adamk@chromium.org>
Commit-Queue: Daniel Ehrenberg <littledan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44495}
A couple infrastructure changes went into this patch:
- test262 changed from expecting $ to $262
- upstream-local-tests.sh gets a command-line parameter for ease of use
- Fixed up the FAIL_SLOPPY infrastructure, which seems to have bit-rotted
- Inserted a terrible hack to get around test262 tests with a $ in the name
Drive-by fix for the length of Intl.DateTimeFormat.prototype.format
R=adamk
Review-Url: https://codereview.chromium.org/2733843002
Cr-Commit-Position: refs/heads/master@{#43749}
This relaxes the check if the test262 archive should be unpacked or not.
A bug in the swarming isolate processor for windows makes the old check for the data directory pass and not unpack.
With this work-around we actually start running tests on windows.
This also fixes the regexp for exception extraction and handles the windows drive letter.
It also strips whitespace to fix a case where a carriage return was included on windows.
BUG=v8:5872
Change-Id: I363925665b0bad7530a1f93a2ea8f39b056d4174
Reviewed-on: https://chromium-review.googlesource.com/445786
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Daniel Ehrenberg <littledan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#43482}
This patch changes the test262 infrastructure to pass individual flags,
specified in the status file, for tests for experimental features, rather
than passing --harmony for all runs. With this change, it should be
easier to run test262 tests in automation when developing new features.
The new workflow would be, when adding a flag, include the flag in the
test expectations file, and when removing the flag, remove the lines from
the test expectations file. This way, the status file does not have to
change when staging or unstaging, and you get the benefit of the automated
tests before staging starts.
R=adamk
CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux_noi18n_rel_ng
Review-Url: https://codereview.chromium.org/2601393002
Cr-Commit-Position: refs/heads/master@{#42249}
This patch provides improved infrastructure for developing test262 tests
together with V8. It has three parts:
- The test262 test runner is updated to look for local versions of tests
in the /test/test262/local-tests directory, which mirrors
/test/test262/data. Additional tests can be added there and are run
together with tests from upstream. Upstream tests can be locally
updated by using the same name in local-tests; if a same-named test
exists, then only the local version will be run. The local-tests
directory is in the V8 repository, unlike the contents of the data
directory, so tests can be added in the same patch as something else.
- The tool /test/test262/upstream-local-tests.sh is added to create
a patch against the test262 respository based on a patch which changes
the local-tests directory.
- The tool /test/test262/prune-local-tests.sh is added to remove
redundant local tests on a test262 roll.
See design doc:
https://docs.google.com/document/d/16bj7AIDgZLv4WOsUEzQ5NzcEN9_xo095e88Pz8FC5rA/edit
Review-Url: https://codereview.chromium.org/2611793002
Cr-Commit-Position: refs/heads/master@{#42117}
- Make testcfg.py ignore files ending in "_FIXTURE.js", which are not supposed
to be tested standalone but only as imports to other modules.
- Refine test262.status to match where we are with the modules implementation.
BUG=v8:1569
R=adamk@chromium.org
Review-Url: https://codereview.chromium.org/2400713004
Cr-Commit-Position: refs/heads/master@{#40134}
The _ParseException function in testcfg.py made incorrect assumptions about how
exceptions are printed (I believe it expected a trace like that produced by
Error).
R=littledan@chromium.org
BUG=v8:5136
Review-Url: https://codereview.chromium.org/2386103010
Cr-Commit-Position: refs/heads/master@{#40078}
This removes the test-download method for all but
promises-aplus. They all only contain legacy code for
deleting old archives.
Only test262 needs to prepare sources on swarming, which is
moved to a new method, called unconditionally.
All references to --download-data and --download-data-only
in the infrastructure can be removed after this.
BUG=
Review-Url: https://codereview.chromium.org/2227613002
Cr-Commit-Position: refs/heads/master@{#38439}
