Commit Graph

27 Commits

Author SHA1 Message Date
jkummerow
9516dccd41 Reland "[test] Fix cctest path separators on Windows"
Now run-tests.py understands "suite/foo/bar" with forward slashes for
command-line test selection on all test suites on all platforms.

Previously, file-based suites like mjsunit also accepted "mjsunit/foo\bar";
that behavior is sacrificed here in favor of unification. For the cctest
suite, OTOH, it wasn't possible on Windows to select specific tests at all.

Original review: https://codereview.chromium.org/1348653003/

This reverts commit 5f44a91059.

NOTRY=true

Review URL: https://codereview.chromium.org/1356613002

Cr-Commit-Position: refs/heads/master@{#30798}
2015-09-17 13:01:12 +00:00
jkummerow
5f44a91059 Revert of [test] Fix cctest path separators on Windows (patchset #2 id:20001 of https://codereview.chromium.org/1348653003/ )
Reason for revert:
mozilla tests are failing on Windows

Original issue's description:
> [test] Fix cctest path separators on Windows
>
> Now run-tests.py understands "suite/foo/bar" with forward slashes for
> command-line test selection on all test suites on all platforms.
>
> Previously, file-based suites like mjsunit also accepted "mjsunit/foo\bar";
> that behavior is sacrificed here in favor of unification. For the cctest
> suite, OTOH, it wasn't possible on Windows to select specific tests at all.
>
> Committed: https://crrev.com/b36cfdb39ae648b49a1396c4f669df9b1f57996c
> Cr-Commit-Position: refs/heads/master@{#30794}

TBR=machenbach@google.com,machenbach@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/1349163002

Cr-Commit-Position: refs/heads/master@{#30795}
2015-09-17 12:00:23 +00:00
jkummerow
b36cfdb39a [test] Fix cctest path separators on Windows
Now run-tests.py understands "suite/foo/bar" with forward slashes for
command-line test selection on all test suites on all platforms.

Previously, file-based suites like mjsunit also accepted "mjsunit/foo\bar";
that behavior is sacrificed here in favor of unification. For the cctest
suite, OTOH, it wasn't possible on Windows to select specific tests at all.

Review URL: https://codereview.chromium.org/1348653003

Cr-Commit-Position: refs/heads/master@{#30794}
2015-09-17 11:24:09 +00:00
machenbach
7ce3afa0df [test] Make test filters platform-independent.
Use the same path separator for test cases on the cmd-line
on all platforms as in testcase.GetLabel, which is used to
report failures.

BUG=chromium:511215
LOG=n
NOTRY=true

Review URL: https://codereview.chromium.org/1281453003

Cr-Commit-Position: refs/heads/master@{#30045}
2015-08-06 12:36:56 +00:00
machenbach
4fe08abde6 [test] Key variant flags by variant name everywhere.
This allows variants to be named on test failures (follow
up) and then to be used in the test runner for a repro.

This also speeds up variant iteration for test262 and fixes
a bug with variants for benchmarks.

BUG=chromium:511215
NOTREECHECKS=true
LOG=n

Review URL: https://codereview.chromium.org/1245623005

Cr-Commit-Position: refs/heads/master@{#29899}
2015-07-29 07:14:30 +00:00
machenbach
3bc1b1346a [test] Let test runner only use exact matches of tests on the cmd-line.
There are many test names in the v8 code base that prefix
others, which makes it hard to only run those tests.

BUG=chromium:511215
LOG=n
NOTRY=true

Review URL: https://codereview.chromium.org/1251363002

Cr-Commit-Position: refs/heads/master@{#29809}
2015-07-23 13:01:38 +00:00
machenbach
a2b6dfb2d1 [test] Correctly merge expected test outcomes.
Without this change, wildcards always overwrite the outcomes
of more specific rules. Now we always merge.

Review URL: https://codereview.chromium.org/1153073002

Cr-Commit-Position: refs/heads/master@{#28629}
2015-05-26 13:22:23 +00:00
mstarzinger
e81ee90018 [test] Limit "unittests" suite to default variant.
This changes the unittests suite to be run in only the default variant,
because unit tests are (and should be) written to be independent from
global flags.

R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/1067863007

Cr-Commit-Position: refs/heads/master@{#28054}
2015-04-27 08:19:55 +00:00
Benedikt Meurer
78f2efe0e4 [mjsunit] Add custom tests based on SQLite 3.8.9.
Also adjust the test runner to remove --always-opt for FAST_VARIANTS.

R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/1056423004

Cr-Commit-Position: refs/heads/master@{#27989}
2015-04-22 07:44:50 +00:00
mstarzinger
8924a9e1b7 [turbofan] Add single --turbo flag.
This flag is intended as a staging flag for TurboFan. It serves as a
single flag that always enables a most recent configuration of TurboFan
for test suites and benchmarks, without needing to update test drivers.

R=titzer@chromium.org,machenbach@chromium.org

Review URL: https://codereview.chromium.org/1094573002

Cr-Commit-Position: refs/heads/master@{#27896}
2015-04-16 16:28:39 +00:00
mstarzinger
9fbe872137 Enable test coverage for --turbo-deoptimization mode.
R=jarin@chromium.org

Review URL: https://codereview.chromium.org/845193004

Cr-Commit-Position: refs/heads/master@{#26102}
2015-01-16 12:53:30 +00:00
balazs.kilvady
fbcfce69aa Conditional flags for tests - set JS stack size for simulators.
There are separated JS and C stacks on simulators so for some stack
extensive tests (like mozilla/js1_5/extensions/regress-355497) might
cause a C stack overflow and that overflow is not caught by V8. It is
not an issue on real HW. Increasing the C stack also solves the problem
but we have already FLAG_sim_stack_size flag to control the JS stack
size.

This patch makes it possible to add flags to tests conditionally in
.status files.

TEST=mozilla/js1_5/extensions/regress-355497
BUG=v8:3152
LOG=N

Review URL: https://codereview.chromium.org/735723006

Cr-Commit-Position: refs/heads/master@{#25434}
2014-11-20 09:47:00 +00:00
machenbach@chromium.org
1b4c25e0b7 Fix fast-variants feature in test driver.
BUG=
R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/701933003

Cr-Commit-Position: refs/heads/master@{#25150}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@25150 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-11-05 13:05:46 +00:00
machenbach@chromium.org
e0801075cd Add fast-variants feature to test driver.
BUG=
R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/704723002

Cr-Commit-Position: refs/heads/master@{#25139}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@25139 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-11-05 10:39:15 +00:00
machenbach@chromium.org
133a7a8647 Let test driver fail if test executable can't list the tests.
TBR=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/605133004

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24253 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-26 13:46:22 +00:00
machenbach@chromium.org
2a92afe1a4 Fix imports in test runner.
TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/532693002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23609 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 12:46:27 +00:00
machenbach@chromium.org
3ca77d4bef Fix test runner imports.
TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/526273003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23597 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 11:18:47 +00:00
machenbach@chromium.org
b4adc1a12b Reland "Refactoring: Make gtest testsuite the default."
BUG=v8:3489
TBR=bmeurer@chromium.org
LOG=n

Review URL: https://codereview.chromium.org/533603002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23585 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:21:03 +00:00
machenbach@chromium.org
b77cb4a793 Revert "Refactoring: Make gtest testsuite the default."
This reverts commit cb8f08658767e7e3182e5deab0473f264f6ce92f.

TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/532673002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23584 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:17:26 +00:00
machenbach@chromium.org
a050734c35 Refactoring: Make gtest testsuite the default.
BUG=v8:3489
R=bmeurer@chromium.org
LOG=n

Review URL: https://codereview.chromium.org/526133003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:11:13 +00:00
machenbach@chromium.org
ba584456e6 Fix result status of rerun flaky tests.
Tests that pass on reruns where wrongly treated as failures. Now the result state can include any of (PASS, FAIL, CRASH, TIMEOUT)

BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/363883003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22186 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-03 09:33:22 +00:00
machenbach@chromium.org
e6e8acd446 Make test runner more flexible for running fast tests.
Transformed variant flags into named variants. Now, all combinations of variants can be specified on the command line. The old command-line flags are kept for backwards compatibility on the bots.

Added two new test groups: slow and pass|fail. Both are implemented similar to the flaky test feature and allow to either skip or run tests marked as slow or as pass|fail.

R=yangguo@chromium.org

Review URL: https://codereview.chromium.org/85733003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18062 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-25 17:34:52 +00:00
svenpanne@chromium.org
882778bf09 Make it possible to run a test only in the standard variant.
Use this for mjsunit/unicode-case-overoptimization, which is not
related to Crankshaft at all and takes ages.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/27704002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17255 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-17 13:09:28 +00:00
machenbach@chromium.org
e7585af65b Add flaky test classification feature to test suites.
Test expectations can now include outcome: FLAKY.

The test runner can now run a class of tests (flaky|non-flaky|all). All tests are in the non-flaky class that are not marked as FLAKY.

The slash correction for windows is now pulled into the test name method. Currently the progress output on windows contains a mixture of / and \.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/22381003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16080 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-08-06 14:39:39 +00:00
jkummerow@chromium.org
7924492ce3 tools/run-tests.py: Fixes for Windows
Review URL: https://codereview.chromium.org/11926015

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13436 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-18 14:55:23 +00:00
jkummerow@chromium.org
70c4d78331 test runner: fix wildcard handling in test status files
Review URL: https://codereview.chromium.org/10986086

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12632 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-28 14:59:00 +00:00
jkummerow@chromium.org
975d6e2170 First commit of new tools/run-tests.py
Review URL: https://codereview.chromium.org/10919265

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-24 09:38:46 +00:00