Commit Graph

22 Commits

Author SHA1 Message Date
machenbach
4fe08abde6 [test] Key variant flags by variant name everywhere.
This allows variants to be named on test failures (follow
up) and then to be used in the test runner for a repro.

This also speeds up variant iteration for test262 and fixes
a bug with variants for benchmarks.

BUG=chromium:511215
NOTREECHECKS=true
LOG=n

Review URL: https://codereview.chromium.org/1245623005

Cr-Commit-Position: refs/heads/master@{#29899}
2015-07-29 07:14:30 +00:00
arv
870ea40a8b [test262-es6] Update to 2015-07-06 which includes the yaml harness fix
Revert "Revert of [test262-es6] Update to 2011-06-29 (patchset #1 id:1 of https://codereview.chromium.org/1220793005/)"

This reverts commit f50fff57b0.

BUG=N
LOG=N
R=littledan@chromium.org, machenbach@chromium.org

Review URL: https://codereview.chromium.org/1212723004

Cr-Commit-Position: refs/heads/master@{#29517}
2015-07-07 13:54:30 +00:00
machenbach
f50fff57b0 Revert of [test262-es6] Update to 2011-06-29 (patchset #1 id:1 of https://codereview.chromium.org/1220793005/)
Reason for revert:
[Sheriff] Runner has some infra failures:
http://build.chromium.org/p/client.v8/builders/V8%20Linux64%20-%20debug/builds/4020/steps/Test262-es6%20-%20no%20variants/logs/stdio

Original issue's description:
> [test262-es6] Update to 2011-06-29
>
> This includes one harness fix but introduces a few new failures. No
> new bugs exposed though.
>
> BUG=N
> LOG=N
> R=littledan@chromium.org, adamk@chromium.org
>
> Committed: https://crrev.com/2e877628803f238dd33c3409413cd7778e36a2b6
> Cr-Commit-Position: refs/heads/master@{#29392}

TBR=littledan@chromium.org,adamk@chromium.org,arv@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=N

Review URL: https://codereview.chromium.org/1212163008

Cr-Commit-Position: refs/heads/master@{#29393}
2015-06-30 20:15:58 +00:00
arv
2e87762880 [test262-es6] Update to 2011-06-29
This includes one harness fix but introduces a few new failures. No
new bugs exposed though.

BUG=N
LOG=N
R=littledan@chromium.org, adamk@chromium.org

Review URL: https://codereview.chromium.org/1220793005

Cr-Commit-Position: refs/heads/master@{#29392}
2015-06-30 18:58:34 +00:00
arv
9f550240b0 Test262-es6 test runner should handle sloppy fail better
This adds a new FAIL_SLOPPY expected output. It then uses this to
determine if the test case has unexpected output.

BUG=v8:4164
LOG=N
R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/1197913002

Cr-Commit-Position: refs/heads/master@{#29194}
2015-06-22 15:08:50 +00:00
arv
0f502f679e Test262 update 2015-06-11 (Take 3)
This reverts commit 72bb369d08.

Always skip slow tests.

Also, some eval tests are failing in strict mode with nosnap

BUG=N
LOG=N
R=adamk@chromium.org, littledan@chromium.org, machenbach@chromium.org

Review URL: https://codereview.chromium.org/1190063002

Cr-Commit-Position: refs/heads/master@{#29164}
2015-06-19 18:58:19 +00:00
machenbach
72bb369d08 Revert of Update test262-es6 to 6/11 (patchset #2 id:40001 of https://codereview.chromium.org/1175313003/)
Reason for revert:
[Sheriff] Please fix test expectations for nosnap before reland. There are 54 tests failing:
http://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20nosnap/builds/3422

Original issue's description:
> Update test262-es6 to 6/11
>
> This reverts commit 67b169199e and picks
> up the fixes to the yaml parser upstream.
>
> BUG=N
> LOG=N
> R=adamk@chromium.org, machenbach@chromium.org
>
> Committed: https://crrev.com/bc847230610d0518a700a69546d23784e6ce3479
> Cr-Commit-Position: refs/heads/master@{#29056}

TBR=adamk@chromium.org,arv@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=N

Review URL: https://codereview.chromium.org/1186093005

Cr-Commit-Position: refs/heads/master@{#29067}
2015-06-17 06:53:04 +00:00
arv
bc84723061 Update test262-es6 to 6/11
This reverts commit 67b169199e and picks
up the fixes to the yaml parser upstream.

BUG=N
LOG=N
R=adamk@chromium.org, machenbach@chromium.org

Review URL: https://codereview.chromium.org/1175313003

Cr-Commit-Position: refs/heads/master@{#29056}
2015-06-16 22:16:08 +00:00
machenbach
71bc7f8793 [test] Fix test archive clobber.
The old version would always clobber.

NOTRY=true
NOTREECHECKS=true

Review URL: https://codereview.chromium.org/1177503005

Cr-Commit-Position: refs/heads/master@{#28895}
2015-06-10 09:09:04 +00:00
machenbach
8f4cd90e8f [test] Clobber unclean test262-es6 checkouts.
Review URL: https://codereview.chromium.org/1173753004

Cr-Commit-Position: refs/heads/master@{#28894}
2015-06-10 08:31:31 +00:00
machenbach
67b169199e Revert of Update Test262 to 5/30 (patchset #4 id:60001 of https://codereview.chromium.org/1136553008/)
Reason for revert:
[Sheriff] Still lots of breakages on some builders. E.g. http://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20nosnap/builds/3313

Original issue's description:
> Update Test262 to 5/30
>
> This updates the test runner to run all tests in both sloppy
> and strict mode.
>
> It also marks the test in the status file as failing in
> sloppy mode even if all it does is marking the test as
> [PASS, FAIL]. A future CL will have to validate that the
> tests pass and fail in the correct mode.
>
> Committed: https://crrev.com/d869f4a4801d4ef6868c266c07f9e29d2e29cba5
> Cr-Commit-Position: refs/heads/master@{#28879}

TBR=rossberg@chromium.org,arv@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/1173933002

Cr-Commit-Position: refs/heads/master@{#28890}
2015-06-10 06:36:55 +00:00
arv
d869f4a480 Update Test262 to 5/30
This updates the test runner to run all tests in both sloppy
and strict mode.

It also marks the test in the status file as failing in
sloppy mode even if all it does is marking the test as
[PASS, FAIL]. A future CL will have to validate that the
tests pass and fail in the correct mode.

Review URL: https://codereview.chromium.org/1136553008

Cr-Commit-Position: refs/heads/master@{#28879}
2015-06-09 17:27:27 +00:00
arv
5639a767ec Update test262-es6 to 2015-03-31
BUG=None
LOG=N
R=adamk, rossberg

Review URL: https://codereview.chromium.org/1052013002

Cr-Commit-Position: refs/heads/master@{#27576}
2015-04-01 21:51:25 +00:00
arv
a56fa150d1 [es6] Update test262 tests
Second try. Disabled the tests that were failing due to
https://github.com/tc39/test262/issues/215

This updates test262 to revision d24fd10 (2015/03/11).

The files moved around in the test repo and a lot of new tests are
failing.

BUG=None
LOG=N
R=adamk, rossberg

Review URL: https://codereview.chromium.org/1040093003

Cr-Commit-Position: refs/heads/master@{#27543}
2015-03-31 13:56:57 +00:00
arv
06a17e54d9 Revert of [es6] Update test262 tests (patchset #4 id:60001 of https://codereview.chromium.org/1025043002/)
Reason for revert:
Bot failed to include verifyNotEnumerable function for some screwed up reason.

Original issue's description:
> [es6] Update test262 tests
>
> This updates test262 to revision d24fd10 (2015/03/11).
>
> The files moved around in the test repo and a lot of new tests are
> failing.
>
> BUG=None
> LOG=N
> R=rossberg
>
> Committed: https://crrev.com/4f2fb3835feff3663146f12be42b01a226d0065e
> Cr-Commit-Position: refs/heads/master@{#27522}

TBR=rossberg@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=None

Review URL: https://codereview.chromium.org/1047713002

Cr-Commit-Position: refs/heads/master@{#27525}
2015-03-30 15:22:54 +00:00
arv
4f2fb3835f [es6] Update test262 tests
This updates test262 to revision d24fd10 (2015/03/11).

The files moved around in the test repo and a lot of new tests are
failing.

BUG=None
LOG=N
R=rossberg

Review URL: https://codereview.chromium.org/1025043002

Cr-Commit-Position: refs/heads/master@{#27522}
2015-03-30 14:28:15 +00:00
arv
36df5747bf Use path in name for test262-es6
For ES5 the file names always included the whole path, like this:

  ch15/15.2/15.2.3/15.2.3.14/15.2.3.14-1-3.js

For ES6 this is no longer true:

  language/computed-property-names/basics/symbol.js
  language/computed-property-names/class/method/symbol.js

To allow the status file to correctly identify the right test we need
to change the reported test name to include the whole path.

BUG=None
LOG=N
R=rossberg@chromium.org, machenbach@chromium.org

Review URL: https://codereview.chromium.org/1001613002

Cr-Commit-Position: refs/heads/master@{#27167}
2015-03-12 16:36:31 +00:00
rossberg@chromium.org
b4a49dfb48 Upgrade test262-es6
TBR=machenbach@chromium.org
BUG=

Review URL: https://codereview.chromium.org/695763003

Cr-Commit-Position: refs/heads/master@{#25036}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@25036 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-31 13:39:44 +00:00
rossberg@chromium.org
d7b5cd0817 Upgrade test262-es6
R=dslomov@chromium.org
BUG=

Review URL: https://codereview.chromium.org/639373005

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24740 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-20 12:17:12 +00:00
rossberg@chromium.org
0d2e95d45b Update test262-es6
R=yangguo@chromium.org
BUG=

Review URL: https://codereview.chromium.org/558013003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23880 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-11 13:39:37 +00:00
rossberg@chromium.org
c987661e0e Activate --harmony switch on test262-es6
R=dslomov@chromium.org
BUG=

Review URL: https://codereview.chromium.org/474423003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23361 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-25 14:31:24 +00:00
rossberg@chromium.org
cc07503bd9 test/test262: update testcfg.py for new test262
testcfg.py:

- update revision and MD5
- remove non-mandatory harness files
- use test parser distributed with test262
- new attribute `suite.harnesspath`
- new method GetIncludesForTest
- GetSourceForTest: cache source on testcase
- IsNegativeTest: use parseTestRecord
- use 7-char sha hash [1]
- DRY setting up paths to test262 suite, harness, etc
- clean up helper fns

harness-adapter.js:

- add $DONE function to adapter [2]

1: github tar file has 7-char sha embedded in dir name
script cannot find directory to rename if they don't match exactly

2: test262 uses a `$DONE` function for async tests with
semantics like those of mocha's `done`.

Briefly: done(arg) => if (arg) { /* failure */ }

Implemented a version of this for v8, using v8-specific
api (`print`, `quit`)

BUG=v8:3513
LOG=N
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/478163002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23313 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-22 13:00:10 +00:00