Commit Graph

56 Commits

Author SHA1 Message Date
Michal Majewski
7f01c9303f [test] Implement variant generators as processors
Bug: v8:6917
Cq-Include-Trybots: luci.v8.try:v8_linux64_fyi_rel_ng;luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: Ib5bfdf4d6fee6102f62c7334a1b22146f1a1fc5b
Reviewed-on: https://chromium-review.googlesource.com/857376
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50525}
2018-01-12 10:39:20 +00:00
Michal Majewski
03165cb93b [test] Store expected outcomes in the testcase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I69d15801b79bf7aa846582367e9f3037b6612431
Reviewed-on: https://chromium-review.googlesource.com/829033
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50163}
2017-12-18 14:51:20 +00:00
Michal Majewski
c8102945af [test] Code cleanup in testcfg.py
- All testcase/testsuite/variant generator subclasses renamed to
  just TestCase/TestSuite/VariantGenerator since they're private
  implementation.
- All `testcase` variables renamed to `test` to not conflict with
  a module name.
- No more two statements in the same line.
- Removed some unused testsuite methods.

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I7710f3419f738a5f9ddca73765dd2cad2e35b952
Reviewed-on: https://chromium-review.googlesource.com/823964
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50076}
2017-12-13 13:36:20 +00:00
Michal Majewski
08a05c2bff [test] Move data access from suite to test.
- create testcase subclass for each test
- move get_command, get_source from suite to test
- promises-aplus tests are broken
- moving expected outcomes etc. is still in progress

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I006e64ecf51dda95c41dff88c68b24f17a638566
Reviewed-on: https://chromium-review.googlesource.com/798331
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50065}
2017-12-13 09:14:47 +00:00
Michal Majewski
22eac2e987 [test] Code cleanup in test suites.
Some refactor moved from
https://chromium-review.googlesource.com/c/v8/v8/+/798331.

Bug: v8:6917
Change-Id: I8cae6cfca7a0d7d8e234052c0ab0bfe252355e60
Reviewed-on: https://chromium-review.googlesource.com/819550
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50020}
2017-12-12 08:54:52 +00:00
Michal Majewski
430e03b3c6 Remove shell info from testcase.
Bug: v8:6917
Change-Id: Ic50ed8aca2ef6b6e60eae194cf46c2264a416657
Reviewed-on: https://chromium-review.googlesource.com/774265
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49417}
2017-11-16 15:42:26 +00:00
Michal Majewski
0a12eaea45 Remove env field from testcase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I09fb05ac6d4b9b1223118494ce2c89e3ab5de109
Reviewed-on: https://chromium-review.googlesource.com/771870
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49415}
2017-11-16 15:26:06 +00:00
Michael Achenbach
0a35d6f985 [test] Let flags from test case have precedence
The status-file flags and the flags from the test case's source code
must always overwrite extra flags set by bots.

Bug: v8:6924
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I0e2aabb69da7cfb8ba6c1c79bd3851462071a6ac
Reviewed-on: https://chromium-review.googlesource.com/732656
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49001}
2017-10-27 13:22:39 +00:00
machenbach
3bf44848c3 [gn] Move build to gypfiles
This prepares for pulling chromium's build as dependency for
gn. After this, the files in build and gypfiles need to stay
in sync until chromium is updated.

BUG=chromium:474921
LOG=n

Review-Url: https://codereview.chromium.org/1848553003
Cr-Commit-Position: refs/heads/master@{#35898}
2016-04-29 10:11:11 +00:00
mstarzinger
a02b7aaa6d [testing] Move the last JS tests out of "preparser".
This moves the last remaining JS file based tests out of the "preparser"
suite. The tests in question all are expected to parse normally and not
throw any exception. This also deprecates the ability of the test suite
to run anything else outside Python templated tests.

R=adamk@chromium.org
TEST=preparser

Review URL: https://codereview.chromium.org/1782173005

Cr-Commit-Position: refs/heads/master@{#34753}
2016-03-14 17:10:49 +00:00
mstarzinger
6a629ff7a9 [testing] Convert "non-alphanum" to be a "message" test.
This converts another test case that is expected to throw a TypeError
but no SyntaxError to have better test coverage (exact message is being
checked now).

R=machenbach@chromium.org
TEST=message,preparser

Review URL: https://codereview.chromium.org/1786623002

Cr-Commit-Position: refs/heads/master@{#34715}
2016-03-11 11:36:40 +00:00
mstarzinger
57ef4f4f3a [testing] Convert some "preparser" tests into "message".
This converts existing "preparser" tests that expect a certain exception
message to be produced into "message" tests. Thereby we get much better
coverage because the former test suite degraded by now to just check
whether each test case threw or not, the exception message was not being
checked at all.

This also deprecates the ability of "preparser" to specify that single
test cases based on JS files are expected to throw, "messages" is far
superior, use that test suite instead.

R=machenbach@chromium.org
TEST=message,preparser

Review URL: https://codereview.chromium.org/1784013003

Cr-Commit-Position: refs/heads/master@{#34713}
2016-03-11 11:00:12 +00:00
machenbach
e1743816d3 Reland [swarming] Isolate v8 testing.
This reverts commit 280a6f8ec8.

Reland of https://codereview.chromium.org/1380593002/

BUG=chromium:535160
LOG=n
CQ_INCLUDE_TRYBOTS=tryserver.v8:v8_linux_nosnap_rel;

Review URL: https://codereview.chromium.org/1390473003

Cr-Commit-Position: refs/heads/master@{#31094}
2015-10-05 09:48:42 +00:00
danno
280a6f8ec8 Revert of [swarming] Isolate v8 testing. (patchset #8 id:140001 of https://codereview.chromium.org/1380593002/ )
Reason for revert:
Prime suspect in breakage of V8 Linux -- no snap

Original issue's description:
> [swarming] Isolate v8 testing.
>
> Add gyp support and isolates for default test suites.
> Add two default isolates, one (default) for using the
> test suite collection we call "default" on the bots. One
> (developer_default) for also supporting the way developers
> call the driver (i.e. without argument, which includes
> the unittests).
>
> BUG=chromium:535160
> LOG=n
>
> Committed: https://crrev.com/9bd83f58f29ab0c7c5b71b00bcb1df3a9e641f05
> Cr-Commit-Position: refs/heads/master@{#31081}

TBR=tandrii@chromium.org,jochen@chromium.org,maruel@chromium.org,machenbach@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:535160

Review URL: https://codereview.chromium.org/1370993008

Cr-Commit-Position: refs/heads/master@{#31084}
2015-10-02 15:45:13 +00:00
machenbach
9bd83f58f2 [swarming] Isolate v8 testing.
Add gyp support and isolates for default test suites.
Add two default isolates, one (default) for using the
test suite collection we call "default" on the bots. One
(developer_default) for also supporting the way developers
call the driver (i.e. without argument, which includes
the unittests).

BUG=chromium:535160
LOG=n

Review URL: https://codereview.chromium.org/1380593002

Cr-Commit-Position: refs/heads/master@{#31081}
2015-10-02 15:15:39 +00:00
machenbach
4fe08abde6 [test] Key variant flags by variant name everywhere.
This allows variants to be named on test failures (follow
up) and then to be used in the test runner for a repro.

This also speeds up variant iteration for test262 and fixes
a bug with variants for benchmarks.

BUG=chromium:511215
NOTREECHECKS=true
LOG=n

Review URL: https://codereview.chromium.org/1245623005

Cr-Commit-Position: refs/heads/master@{#29899}
2015-07-29 07:14:30 +00:00
dslomov
92138c73a7 Remove --harmony-scoping flag.
We have been shipping harmony scoping for 2 Chrome releases now (M41
and M42). Time to remove the flag.

R=rossberg@chromium.org
LOG=Y

Review URL: https://codereview.chromium.org/1007783002

Cr-Commit-Position: refs/heads/master@{#27187}
2015-03-13 15:15:57 +00:00
dslomov
e6198a0fed Update tests in preparation for shipping classes.
R=arv@chromium.org
BUG=v8:3330
LOG=N

Review URL: https://codereview.chromium.org/788773003

Cr-Commit-Position: refs/heads/master@{#25783}
2014-12-11 15:54:09 +00:00
dslomov
4bf531c1ea Update strict mode function declaration tests before block scoping.
R=arv@chromium.org,marja@chromium.org
BUG=v8:2198
LOG=N

Review URL: https://codereview.chromium.org/788143004

Cr-Commit-Position: refs/heads/master@{#25760}
2014-12-10 20:09:42 +00:00
dslomov
008405f0b9 Pass --noharmony-scoping to tests validating that we throw on 'const' in strict mode.
R=marja@chromium.org
BUG=v8:2198
LOG=N

Review URL: https://codereview.chromium.org/784323002

Cr-Commit-Position: refs/heads/master@{#25733}
2014-12-10 08:55:53 +00:00
arv@chromium.org
cfdfbb885e ES6: Duplicate properties are no longer an error
This removes the duplicate property checker and updates the tests.

BUG=v8:3498
LOG=Y
R=marja@chromium.org

Review URL: https://codereview.chromium.org/459463002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23239 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-20 14:25:48 +00:00
mstarzinger@chromium.org
5c7edb7e2b Enable access check tests that no longer fail with TF.
R=verwaest@chromium.org

Review URL: https://codereview.chromium.org/473873002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23130 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-14 13:24:57 +00:00
marja@chromium.org
5bbc92dee0 Throw syntax error when a getter/setter has the wrong number of params
We used to allow any number of parameters in getters and setters to
match JSC. This is a violation of ES5.1 and both SpiderMonkey and
Chakra throw on these syntax errors.

BUG=v8:3371
LOG=Y
R=marja@chromium.org

Review URL: https://codereview.chromium.org/329413002

Patch from Erik Arvidsson <arv@chromium.org>.

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21868 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-06-17 07:23:26 +00:00
ulan@chromium.org
e95bc7eec8 Merge experimental/a64 to bleeding_edge.
BUG=v8:3113
LOG=Y
R=jochen@chromium.org, rmcilroy@chromium.org, rodolph.perfetta@arm.com

Review URL: https://codereview.chromium.org/148293020

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@19311 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-02-12 09:19:30 +00:00
yangguo@chromium.org
aa3518a0f3 Make sure files end with exactly one new line and police this in presubmit.
The changes are (excluding presubmit.py) mechanical. I added the following
lines after the check and iterated the presubmit script until all errors
went away:

f = open(name, "w");
if contents.endswith('\n\n'):
  f.write(contents[0:-1])
else:
  f.write(contents + '\n')

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/82803005

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18017 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-22 13:50:39 +00:00
svenpanne@chromium.org
882778bf09 Make it possible to run a test only in the standard variant.
Use this for mjsunit/unicode-case-overoptimization, which is not
related to Crankshaft at all and takes ages.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/27704002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17255 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-17 13:09:28 +00:00
mstarzinger@chromium.org
63d8abb6c6 Unify and fix checkers for duplicate object literal properties.
R=ulan@chromium.org
TEST=preparser/duplicate-property,mjsunit/regress/regress-parse-object-literal

Review URL: https://codereview.chromium.org/26375004

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17136 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-10 11:58:16 +00:00
mvstanton@chromium.org
fc95207750 Revert "Unify and fix checkers for duplicate object literal properties."
This reverts commit 12c68518bd2c74dc4e44d928c84c17f98ca63359.
(r17114)

R=mstarzinger@chromium.org
TBR=mstarzinger@chromium.org

Review URL: https://codereview.chromium.org/26732006

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17122 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-10 08:45:19 +00:00
mstarzinger@chromium.org
067a266426 Unify and fix checkers for duplicate object literal properties.
R=ulan@chromium.org
TEST=preparser/duplicate-property

Review URL: https://codereview.chromium.org/25755002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17114 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-04 16:58:01 +00:00
mstarzinger@chromium.org
3c8e87d840 Remove obsolete preparser binary.
R=danno@chromium.org

Review URL: https://codereview.chromium.org/25277003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17113 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-04 16:21:23 +00:00
jkummerow@chromium.org
e7130a1e2b Migrate to new test status file syntax
R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/23498058

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16919 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-24 12:08:33 +00:00
jkummerow@chromium.org
5eadc1a428 Remove SCons related files
Review URL: https://codereview.chromium.org/14348002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14328 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-04-18 11:01:14 +00:00
ulan@chromium.org
06a9f51ccb Adjust Android test expectations.
Disable long running tests:
- test-threads/ThreadJoinSelf in release and debug modes.
- regress/regress-1122 in debug mode.

Disable failing test:
- preparser/strict-octal-regexp (v8 issue 2265).

R=jkummerow@chromium.org

Review URL: https://chromiumcodereview.appspot.com/11185073

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12769 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-19 09:28:23 +00:00
jkummerow@chromium.org
975d6e2170 First commit of new tools/run-tests.py
Review URL: https://codereview.chromium.org/10919265

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-24 09:38:46 +00:00
jkummerow@chromium.org
1d0aea7c3a Fix wrong iteration variable in test/preparser/strict-identifiers.pyt
R=mstarzinger@chromium.org
TEST=preparser/strict-identifiers

Review URL: https://chromiumcodereview.appspot.com/10693116

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12009 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-07-09 08:26:17 +00:00
mstarzinger@chromium.org
993d650f15 MIPS: updated test .status files based mostly on the ARM version.
BUG=
TEST=

Review URL: http://codereview.chromium.org/8572032
Patch from Gergely Kis <gergely@homejinni.com>.

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@10155 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-12-05 11:44:17 +00:00
keuchel@chromium.org
ad9290a6ae Disallow strict mode FutureReservedWords as break/continue labels in strict mode
TEST=preparser/strict-identifiers.pyt

Review URL: http://codereview.chromium.org/7987002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@9404 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-09-22 16:38:28 +00:00
lrn@chromium.org
81e7f597b0 Reintroduce duplicate identifier detection in preparser.
Duplicate identifier detection must be an early syntax error in strict code,
so errors in otherwise lazily compiled functions must be caught in the
preparser.

Originally introduced in r8541 and reverted in r8542.
Now really compiles on Windows.

Review URL: http://codereview.chromium.org/7782023

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@9172 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-09-07 12:39:53 +00:00
jkummerow@chromium.org
1a849d2754 Make GYP build usable for day-to-day work (second attempt)
- Introduce a global Makefile that triggers GYP-based building
- Some fixes to .gyp[i] files to make everything work
- tools/test-wrapper-gypbuild.py as a temporary solution for easy testing

Original review URL: http://codereview.chromium.org/7383006/

Review URL: http://codereview.chromium.org/7550030

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8840 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-08-05 09:44:30 +00:00
jkummerow@chromium.org
7ac9dfcc81 Revert "Make GYP build usable for day-to-day work"
This reverts r8674 due to failures on the buildbots.

Review URL: http://codereview.chromium.org/7396022

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8677 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-07-18 16:39:41 +00:00
jkummerow@chromium.org
93c5c5fa23 Make GYP build usable for day-to-day work
- Introduce a global Makefile that triggers GYP-based building
- Some fixes to .gyp[i] files to make everything work
- tools/test-wrapper-gypbuild.py as a temporary solution for easy testing

Review URL: http://codereview.chromium.org/7383006

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8674 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-07-18 14:19:48 +00:00
lrn@chromium.org
57b7a67dba Revert preparser duplicate detection.
Doesn't work on Windows yet.
Crashes some layout-tests.

Review URL: http://codereview.chromium.org/7278039

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8542 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-07-06 09:15:10 +00:00
lrn@chromium.org
57760b8275 Make preparser detect duplicate parameters and object literal properties.
This is a fix and reapply of r8516 with some comments addressed and more
tests added.
The difference from r8516 is that canonicalization of number literals is
no performed using the same methods as in v8, to avoid false positives/negatives
when detecting duplicates.

Review URL: http://codereview.chromium.org/7193045

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8541 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-07-06 08:32:48 +00:00
lrn@chromium.org
4f1e60cc75 Revert r8516.
Revision 8516 contained a temporary hack that doesn't work on Windows.

TBR: ricow

Review URL: http://codereview.chromium.org/7298008

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8519 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-07-01 15:44:21 +00:00
lrn@chromium.org
b6779044c9 Add duplicate parameter detection to preparser.
Add tests for duplicate properties of object initialisers to preparser.

TEST=preparser

Review URL: http://codereview.chromium.org/7168016

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8517 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-07-01 13:46:52 +00:00
keuchel@chromium.org
ab3d4cf7b8 Proper handling of future reserved words in strict and normal mode.
BUG=86442
TEST=mjsunit/keywords-and-reserved_words.js

Review URL: http://codereview.chromium.org/7207007

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8421 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-06-24 14:59:51 +00:00
lrn@chromium.org
b97da90453 Add tests for function statements in strict mode.
Small fixes.
Added test for const declaration in strict mode.

TEST=preparser/strict-function-statement

Review URL: http://codereview.chromium.org/6990056

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8041 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-05-24 14:02:59 +00:00
lrn@chromium.org
dc69639e4c Remove newlines from preparser command line parameters to make Windows happy.
Review URL: http://codereview.chromium.org/6974009

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@7992 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-05-23 12:56:00 +00:00
lrn@chromium.org
b253dd8927 Create template system for strict-mode tests.
This makes it possible to get total coverage without creating thousands
of individual test files.

Review URL: http://codereview.chromium.org/7061008

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@7985 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-05-23 10:35:30 +00:00
lrn@chromium.org
7db97de8e0 Added preparser strict-mode tests.
Now tests for use of eval, arguments, reserved words and with statement.

Review URL: http://codereview.chromium.org/7037024

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@7951 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-05-19 09:01:46 +00:00