Commit Graph

41 Commits

Author SHA1 Message Date
Michal Majewski
c8102945af [test] Code cleanup in testcfg.py
- All testcase/testsuite/variant generator subclasses renamed to
  just TestCase/TestSuite/VariantGenerator since they're private
  implementation.
- All `testcase` variables renamed to `test` to not conflict with
  a module name.
- No more two statements in the same line.
- Removed some unused testsuite methods.

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I7710f3419f738a5f9ddca73765dd2cad2e35b952
Reviewed-on: https://chromium-review.googlesource.com/823964
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#50076}
2017-12-13 13:36:20 +00:00
Michal Majewski
08a05c2bff [test] Move data access from suite to test.
- create testcase subclass for each test
- move get_command, get_source from suite to test
- promises-aplus tests are broken
- moving expected outcomes etc. is still in progress

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I006e64ecf51dda95c41dff88c68b24f17a638566
Reviewed-on: https://chromium-review.googlesource.com/798331
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50065}
2017-12-13 09:14:47 +00:00
Michal Majewski
22eac2e987 [test] Code cleanup in test suites.
Some refactor moved from
https://chromium-review.googlesource.com/c/v8/v8/+/798331.

Bug: v8:6917
Change-Id: I8cae6cfca7a0d7d8e234052c0ab0bfe252355e60
Reviewed-on: https://chromium-review.googlesource.com/819550
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#50020}
2017-12-12 08:54:52 +00:00
Michal Majewski
6056d32b38 [test] Extract expected outcomes. Simpler FAIL_* handling.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I864de452bacb9e34fa1bc70722bf4c2fa3de4204
Reviewed-on: https://chromium-review.googlesource.com/782723
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49614}
2017-11-24 09:23:03 +00:00
Michal Majewski
0f2223c894 [test] Refactor getting test outcomes from statusfile.
Filtering by status file split to four parts:
1. Getting outcomes - reads both variant dependent and
independent outcomes, no more need to do it twice.
2. Checking unused rules - has a switch to check only variant
dependent/independent rules.
3. Reading flags - if outcome starts with '--' it is treated as a flag.
4. Actual filtering.

Outcomes removed from the testcase object, can be accessed
by call to its testsuite.

Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I35762f891010ddda926250452b88656047433daa
Reviewed-on: https://chromium-review.googlesource.com/775160
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49547}
2017-11-21 15:00:24 +00:00
Michal Majewski
0a12eaea45 Remove env field from testcase.
Bug: v8:6917
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I09fb05ac6d4b9b1223118494ce2c89e3ab5de109
Reviewed-on: https://chromium-review.googlesource.com/771870
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michał Majewski <majeski@google.com>
Cr-Commit-Position: refs/heads/master@{#49415}
2017-11-16 15:26:06 +00:00
Michael Achenbach
0a35d6f985 [test] Let flags from test case have precedence
The status-file flags and the flags from the test case's source code
must always overwrite extra flags set by bots.

Bug: v8:6924
Cq-Include-Trybots: master.tryserver.v8:v8_linux_noi18n_rel_ng
Change-Id: I0e2aabb69da7cfb8ba6c1c79bd3851462071a6ac
Reviewed-on: https://chromium-review.googlesource.com/732656
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#49001}
2017-10-27 13:22:39 +00:00
Michael Achenbach
14bfa18b45 [test] Remove deprecated test-download feature
Bug: v8:6917
Change-Id: I3889cd0d059c3473a7b83eb298734a7a6a8a1de5
Reviewed-on: https://chromium-review.googlesource.com/712175
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48467}
2017-10-11 15:18:19 +00:00
Mythri
7371c34b6b Use --opt instead of --crankshaft in tests.
1. Replaces --crankshaft with --opt in tests.
2. Also fixes presubmit to check for --opt flag when
assertOptimized is used.
3. Updates testrunner/local/variants.py and 
v8_foozie.py to use --opt flag.
This would mean, nooptimize variant means there are
no optimizations. Not even with %OptimizeFunctionOnNextCall.

Bug:v8:6325

Change-Id: I638e743d0773a6729c6b9749e2ca1e2537f12ce6
Reviewed-on: https://chromium-review.googlesource.com/490206
Commit-Queue: Mythri Alle <mythria@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#44985}
2017-04-28 14:20:39 +00:00
machenbach
3bf44848c3 [gn] Move build to gypfiles
This prepares for pulling chromium's build as dependency for
gn. After this, the files in build and gypfiles need to stay
in sync until chromium is updated.

BUG=chromium:474921
LOG=n

Review-Url: https://codereview.chromium.org/1848553003
Cr-Commit-Position: refs/heads/master@{#35898}
2016-04-29 10:11:11 +00:00
bmeurer
ccab8be352 [test] Re-enable typescript that is no longer failing on ia32.
R=jarin@chromium.org
BUG=v8:4493
LOG=n

Review URL: https://codereview.chromium.org/1402383005

Cr-Commit-Position: refs/heads/master@{#31545}
2015-10-26 06:00:35 +00:00
machenbach
04a338b1c0 [test] Skip failing test due to --turbo-inlining.
TBR=jarin@chromium.org
BUG=v8:4493
LOG=n
NOTRY=true

Review URL: https://codereview.chromium.org/1417583003

Cr-Commit-Position: refs/heads/master@{#31408}
2015-10-20 12:34:39 +00:00
Benedikt Meurer
b7db7efd15 [test] Skip failing test due to --turbo-inlining.
TBR=jarin@chromium.org
BUG=v8:4493
LOG=n

Review URL: https://codereview.chromium.org/1416863002 .

Cr-Commit-Position: refs/heads/master@{#31407}
2015-10-20 11:56:02 +00:00
machenbach
80df08b59f [swarming] Isolate more tests.
BUG=chromium:535160
LOG=n
NOTRY=true

Review URL: https://codereview.chromium.org/1411213002

Cr-Commit-Position: refs/heads/master@{#31370}
2015-10-19 11:37:26 +00:00
machenbach
3e014cb319 [test] Pull benchmarks as a dependency.
BUG=chromium:535160
LOG=n

Review URL: https://codereview.chromium.org/1411143002

Cr-Commit-Position: refs/heads/master@{#31359}
2015-10-19 08:42:16 +00:00
machenbach
ed5b12a5ba [test] Fix status files.
NOTRY=true

Review URL: https://codereview.chromium.org/1395503002

Cr-Commit-Position: refs/heads/master@{#31151}
2015-10-07 13:14:28 +00:00
machenbach
366dc81143 [Swarming] Presort tests - slowest first.
Swarming won't support an incremental data file. By just
sorting the lowest hanging fruits to the beginning we
already get a big bang for the buck (>80% of the improvement
we get otherwise).

This will require semi-regular manual updates of the
slowest tests.

BUG=chromium:535160
LOG=n

Review URL: https://codereview.chromium.org/1385443002

Cr-Commit-Position: refs/heads/master@{#31061}
2015-10-01 19:19:17 +00:00
machenbach
4fe08abde6 [test] Key variant flags by variant name everywhere.
This allows variants to be named on test failures (follow
up) and then to be used in the test runner for a repro.

This also speeds up variant iteration for test262 and fixes
a bug with variants for benchmarks.

BUG=chromium:511215
NOTREECHECKS=true
LOG=n

Review URL: https://codereview.chromium.org/1245623005

Cr-Commit-Position: refs/heads/master@{#29899}
2015-07-29 07:14:30 +00:00
mstarzinger
8924a9e1b7 [turbofan] Add single --turbo flag.
This flag is intended as a staging flag for TurboFan. It serves as a
single flag that always enables a most recent configuration of TurboFan
for test suites and benchmarks, without needing to update test drivers.

R=titzer@chromium.org,machenbach@chromium.org

Review URL: https://codereview.chromium.org/1094573002

Cr-Commit-Position: refs/heads/master@{#27896}
2015-04-16 16:28:39 +00:00
mstarzinger
9fbe872137 Enable test coverage for --turbo-deoptimization mode.
R=jarin@chromium.org

Review URL: https://codereview.chromium.org/845193004

Cr-Commit-Position: refs/heads/master@{#26102}
2015-01-16 12:53:30 +00:00
jarin@chromium.org
abefd29161 Revert "Enable turbofan deoptimization by default."
This reverts commit 866032692f for breaking
tests.

TBR=mstarzinger@chromium.org
BUG=

Review URL: https://codereview.chromium.org/681713004

Cr-Commit-Position: refs/heads/master@{#24908}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24908 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-27 14:37:10 +00:00
jarin@chromium.org
866032692f Enable turbofan deoptimization by default.
BUG=
R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/638833004

Cr-Commit-Position: refs/heads/master@{#24905}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24905 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-27 12:47:16 +00:00
bmeurer@chromium.org
3dae2a0d3f Run benchmarks tests with --turbo-asm flag.
R=machenbach@chromium.org, mstarzinger@chromium.org

Review URL: https://codereview.chromium.org/646373002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24549 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-13 09:30:27 +00:00
mstarzinger@chromium.org
743d08df79 Skip octane/pdfjs for now in debug mode because slow.
TBR=rossberg@chromium.org
TEST=benchmarks/octane/pdfjs

Review URL: https://codereview.chromium.org/569243002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23950 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-15 14:28:06 +00:00
mstarzinger@chromium.org
2b9e1c54d4 Enable typed pipeline for the benchmark tests.
R=rossberg@chromium.org
TEST=benchmarks

Review URL: https://codereview.chromium.org/567293003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23944 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-15 12:48:55 +00:00
mstarzinger@chromium.org
13f01e65c4 Enable typed pipeline for TurboFan.
R=danno@chromium.org

Review URL: https://codereview.chromium.org/552303003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23830 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-10 09:52:50 +00:00
mstarzinger@chromium.org
1d2a0759f2 Fix predictable mode to make time pass faster.
R=hpayer@chromium.org

Review URL: https://codereview.chromium.org/460413002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23087 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-12 17:54:41 +00:00
mstarzinger@chromium.org
b84dc47259 Reduce redundant phi nodes after graph building.
R=titzer@chromium.org

Review URL: https://codereview.chromium.org/461003002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23075 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-12 12:30:22 +00:00
machenbach@chromium.org
102a17402f Skip richards with TF.
TBR=mstarzinger@chromium.org

Review URL: https://codereview.chromium.org/458483002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23004 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-08 14:07:50 +00:00
machenbach@chromium.org
f263478eed Enable TF for testing benchmarks.
BUG=
R=mstarzinger@chromium.org

Review URL: https://codereview.chromium.org/440383002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22976 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-08-07 15:26:28 +00:00
ulan@chromium.org
e95bc7eec8 Merge experimental/a64 to bleeding_edge.
BUG=v8:3113
LOG=Y
R=jochen@chromium.org, rmcilroy@chromium.org, rodolph.perfetta@arm.com

Review URL: https://codereview.chromium.org/148293020

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@19311 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-02-12 09:19:30 +00:00
yangguo@chromium.org
42f920cfea Update benchmark test suite.
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/67763007

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17934 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-20 15:05:52 +00:00
svenpanne@chromium.org
882778bf09 Make it possible to run a test only in the standard variant.
Use this for mjsunit/unicode-case-overoptimization, which is not
related to Crankshaft at all and takes ages.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/27704002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17255 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-17 13:09:28 +00:00
jkummerow@chromium.org
e7130a1e2b Migrate to new test status file syntax
R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/23498058

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16919 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-24 12:08:33 +00:00
jkummerow@chromium.org
14239ab9fb Add "benchmarks" test suite
R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/21645003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16056 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-08-05 14:54:09 +00:00
mstarzinger@chromium.org
c47af882ba Remove deprecated test suite configurations.
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/13878008

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14326 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-04-18 10:48:27 +00:00
jkummerow@chromium.org
975d6e2170 First commit of new tools/run-tests.py
Review URL: https://codereview.chromium.org/10919265

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-24 09:38:46 +00:00
ricow@chromium.org
950feb6940 Switch from sample shell to d8 for unit test
Landing for yangguo to coordinate with buildbot update.

Original codereview: http://codereview.chromium.org/7282008/
Review URL: http://codereview.chromium.org/7495033

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@8739 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-07-26 05:56:45 +00:00
ricow@chromium.org
4bb55fd341 Change cctests to use variant flags as part of the name for the serilization file.
Because we run all tests three times with different variant flags (to
test crankshaft) we might end up in a situation where we try to write
to the same serilization file from two different threads
simultaneously. The patch concats the variant flags at the end of the
serialization file name.
Review URL: http://codereview.chromium.org/6688068

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@7285 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-03-21 12:57:25 +00:00
ager@chromium.org
e181bd2b6f Use full paths for cctests.
Review URL: http://codereview.chromium.org/6670045

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@7201 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-03-16 11:16:18 +00:00
ager@chromium.org
8a5e4837f8 Add a V8 benchmark suite run to test.py to catch failures on the buildbot.
Review URL: http://codereview.chromium.org/6698029

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@7198 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2011-03-16 09:45:28 +00:00