Commit Graph

141 Commits

Author SHA1 Message Date
arv
bc84723061 Update test262-es6 to 6/11
This reverts commit 67b169199e and picks
up the fixes to the yaml parser upstream.

BUG=N
LOG=N
R=adamk@chromium.org, machenbach@chromium.org

Review URL: https://codereview.chromium.org/1175313003

Cr-Commit-Position: refs/heads/master@{#29056}
2015-06-16 22:16:08 +00:00
machenbach
6a10931820 [test] More debugging output in test runner.
Also use again the taskkill method, recently removed in
https://codereview.chromium.org/1156133006/

BUG=chromium:485932
LOG=n
TBR=jkummerow@chromium.org
NOTREECHECKS=true

Review URL: https://codereview.chromium.org/1177903005

Cr-Commit-Position: refs/heads/master@{#29001}
2015-06-12 13:43:57 +00:00
machenbach
f3e8c11afd [test] Add random seed stress mode to test runner.
NOTRY=true

Review URL: https://codereview.chromium.org/1180473003

Cr-Commit-Position: refs/heads/master@{#28936}
2015-06-11 11:42:37 +00:00
machenbach
816b5b1761 [test] Use generator to accelerate test runner startup.
This saves >1s for mjsunit and >10s for test262.

Review URL: https://codereview.chromium.org/1174923002

Cr-Commit-Position: refs/heads/master@{#28904}
2015-06-10 11:15:19 +00:00
machenbach
f8fe5c6bcf Reland [test] Refactoring - Let runner handle test IDs.
Reason for revert:
All build problems had other reasons.

Original issue's description:
> Revert of [test] Refactoring - Let runner handle test IDs. (patchset #1 id:1 of https://codereview.chromium.org/1168303007/)
>
> Reason for revert:
> [Sheriff] Revert until the tree is in a better state.
>
> Original issue's description:
> > [test] Refactoring - Let runner handle test IDs.
> >
> > This prepares for properly rerunning tests. Currently when
> > tests are rerun, the same test object is reused. This
> > will be changed in a follow up.
> >
> > Committed: https://crrev.com/f41a81b8a513fc360c500c066b74f223bc9c0223
> > Cr-Commit-Position: refs/heads/master@{#28864}
>
> TBR=jkummerow@chromium.org,tandrii@chromium.org
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
>
> Committed: https://crrev.com/f83444a5f24381ec9ccc28b1e18e1e9370415bb2
> Cr-Commit-Position: refs/heads/master@{#28891}

TBR=jkummerow@chromium.org,tandrii@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/1179483002

Cr-Commit-Position: refs/heads/master@{#28903}
2015-06-10 10:15:28 +00:00
machenbach
1a511b0e38 Revert of Revert of [test] Refactoring - Use subject/observer pattern for progress indicators. (patchset #1 id:1 of https://codereview.chromium.org/1163373005/)
Reason for revert:
All build problems had other causes.

Original issue's description:
> Revert of [test] Refactoring - Use subject/observer pattern for progress indicators. (patchset #3 id:40001 of https://codereview.chromium.org/1171943002/)
>
> Reason for revert:
> might break stuff
>
> Original issue's description:
> > [test] Refactoring - Use subject/observer pattern for progress indicators.
> >
> > This should prevent bugs caused by missing super calls in
> > overridden methods. The assumption is that methods of
> > different indicators are independent.
> >
> > Committed: https://crrev.com/fbe973ff1722a6158a5b2babce9c1a32d26a1d3b
> > Cr-Commit-Position: refs/heads/master@{#28866}
>
> TBR=jkummerow@chromium.org,tandrii@chromium.org
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
>
> Committed: https://crrev.com/2a3962d9d2a7415378811f4d56522531332d5a3e
> Cr-Commit-Position: refs/heads/master@{#28869}

TBR=jkummerow@chromium.org,tandrii@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/1176863002

Cr-Commit-Position: refs/heads/master@{#28901}
2015-06-10 09:42:31 +00:00
machenbach
f83444a5f2 Revert of [test] Refactoring - Let runner handle test IDs. (patchset #1 id:1 of https://codereview.chromium.org/1168303007/)
Reason for revert:
[Sheriff] Revert until the tree is in a better state.

Original issue's description:
> [test] Refactoring - Let runner handle test IDs.
>
> This prepares for properly rerunning tests. Currently when
> tests are rerun, the same test object is reused. This
> will be changed in a follow up.
>
> Committed: https://crrev.com/f41a81b8a513fc360c500c066b74f223bc9c0223
> Cr-Commit-Position: refs/heads/master@{#28864}

TBR=jkummerow@chromium.org,tandrii@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/1167343003

Cr-Commit-Position: refs/heads/master@{#28891}
2015-06-10 06:46:45 +00:00
machenbach
67b169199e Revert of Update Test262 to 5/30 (patchset #4 id:60001 of https://codereview.chromium.org/1136553008/)
Reason for revert:
[Sheriff] Still lots of breakages on some builders. E.g. http://build.chromium.org/p/client.v8/builders/V8%20Linux%20-%20nosnap/builds/3313

Original issue's description:
> Update Test262 to 5/30
>
> This updates the test runner to run all tests in both sloppy
> and strict mode.
>
> It also marks the test in the status file as failing in
> sloppy mode even if all it does is marking the test as
> [PASS, FAIL]. A future CL will have to validate that the
> tests pass and fail in the correct mode.
>
> Committed: https://crrev.com/d869f4a4801d4ef6868c266c07f9e29d2e29cba5
> Cr-Commit-Position: refs/heads/master@{#28879}

TBR=rossberg@chromium.org,arv@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/1173933002

Cr-Commit-Position: refs/heads/master@{#28890}
2015-06-10 06:36:55 +00:00
arv
d869f4a480 Update Test262 to 5/30
This updates the test runner to run all tests in both sloppy
and strict mode.

It also marks the test in the status file as failing in
sloppy mode even if all it does is marking the test as
[PASS, FAIL]. A future CL will have to validate that the
tests pass and fail in the correct mode.

Review URL: https://codereview.chromium.org/1136553008

Cr-Commit-Position: refs/heads/master@{#28879}
2015-06-09 17:27:27 +00:00
machenbach
2a3962d9d2 Revert of [test] Refactoring - Use subject/observer pattern for progress indicators. (patchset #3 id:40001 of https://codereview.chromium.org/1171943002/)
Reason for revert:
might break stuff

Original issue's description:
> [test] Refactoring - Use subject/observer pattern for progress indicators.
>
> This should prevent bugs caused by missing super calls in
> overridden methods. The assumption is that methods of
> different indicators are independent.
>
> Committed: https://crrev.com/fbe973ff1722a6158a5b2babce9c1a32d26a1d3b
> Cr-Commit-Position: refs/heads/master@{#28866}

TBR=jkummerow@chromium.org,tandrii@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review URL: https://codereview.chromium.org/1163373005

Cr-Commit-Position: refs/heads/master@{#28869}
2015-06-09 15:32:50 +00:00
machenbach
fbe973ff17 [test] Refactoring - Use subject/observer pattern for progress indicators.
This should prevent bugs caused by missing super calls in
overridden methods. The assumption is that methods of
different indicators are independent.

Review URL: https://codereview.chromium.org/1171943002

Cr-Commit-Position: refs/heads/master@{#28866}
2015-06-09 14:41:26 +00:00
machenbach
f41a81b8a5 [test] Refactoring - Let runner handle test IDs.
This prepares for properly rerunning tests. Currently when
tests are rerun, the same test object is reused. This
will be changed in a follow up.

Review URL: https://codereview.chromium.org/1168303007

Cr-Commit-Position: refs/heads/master@{#28864}
2015-06-09 14:17:08 +00:00
machenbach
e728125131 [test] Fix missing heartbeats in test runner.
TBR=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/1163523008

Cr-Commit-Position: refs/heads/master@{#28808}
2015-06-05 09:51:01 +00:00
machenbach
5effc7131a [test] Fix assert for predictable mode in test runner.
NOTRY=true

Review URL: https://codereview.chromium.org/1156663009

Cr-Commit-Position: refs/heads/master@{#28679}
2015-05-28 13:06:49 +00:00
machenbach
4d6f1abbdb [test] Remove default for zero test cases.
BUG=

Review URL: https://codereview.chromium.org/1153103003

Cr-Commit-Position: refs/heads/master@{#28654}
2015-05-27 13:50:48 +00:00
machenbach
2dda8c3d4e [test] Verbose test runner output on windows.
TBR=jkummerow@chromium.org
NOTRY=true

Review URL: https://codereview.chromium.org/1156133006

Cr-Commit-Position: refs/heads/master@{#28643}
2015-05-26 19:50:54 +00:00
machenbach
a2b6dfb2d1 [test] Correctly merge expected test outcomes.
Without this change, wildcards always overwrite the outcomes
of more specific rules. Now we always merge.

Review URL: https://codereview.chromium.org/1153073002

Cr-Commit-Position: refs/heads/master@{#28629}
2015-05-26 13:22:23 +00:00
machenbach
0b81f67b12 [test-runner] Improve test execution without tmp files for output.
BUG=chromium:485932
LOG=n

Review URL: https://codereview.chromium.org/1134703002

Cr-Commit-Position: refs/heads/master@{#28334}
2015-05-11 09:26:35 +00:00
mstarzinger
e81ee90018 [test] Limit "unittests" suite to default variant.
This changes the unittests suite to be run in only the default variant,
because unit tests are (and should be) written to be independent from
global flags.

R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/1067863007

Cr-Commit-Position: refs/heads/master@{#28054}
2015-04-27 08:19:55 +00:00
Benedikt Meurer
78f2efe0e4 [mjsunit] Add custom tests based on SQLite 3.8.9.
Also adjust the test runner to remove --always-opt for FAST_VARIANTS.

R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/1056423004

Cr-Commit-Position: refs/heads/master@{#27989}
2015-04-22 07:44:50 +00:00
mstarzinger
8924a9e1b7 [turbofan] Add single --turbo flag.
This flag is intended as a staging flag for TurboFan. It serves as a
single flag that always enables a most recent configuration of TurboFan
for test suites and benchmarks, without needing to update test drivers.

R=titzer@chromium.org,machenbach@chromium.org

Review URL: https://codereview.chromium.org/1094573002

Cr-Commit-Position: refs/heads/master@{#27896}
2015-04-16 16:28:39 +00:00
machenbach
99be3e82bf [test-runner] Pass slowest test durations to buildbot.
BUG=

Review URL: https://codereview.chromium.org/1070753003

Cr-Commit-Position: refs/heads/master@{#27696}
2015-04-09 09:56:03 +00:00
machenbach
a54f22d418 Make test runner more chatty to avoid it getting killed by buildbot.
NOTRY=true
NOTREECHECKS=true

Review URL: https://codereview.chromium.org/1064043002

Cr-Commit-Position: refs/heads/master@{#27653}
2015-04-08 09:53:39 +00:00
arv
a56fa150d1 [es6] Update test262 tests
Second try. Disabled the tests that were failing due to
https://github.com/tc39/test262/issues/215

This updates test262 to revision d24fd10 (2015/03/11).

The files moved around in the test repo and a lot of new tests are
failing.

BUG=None
LOG=N
R=adamk, rossberg

Review URL: https://codereview.chromium.org/1040093003

Cr-Commit-Position: refs/heads/master@{#27543}
2015-03-31 13:56:57 +00:00
arv
06a17e54d9 Revert of [es6] Update test262 tests (patchset #4 id:60001 of https://codereview.chromium.org/1025043002/)
Reason for revert:
Bot failed to include verifyNotEnumerable function for some screwed up reason.

Original issue's description:
> [es6] Update test262 tests
>
> This updates test262 to revision d24fd10 (2015/03/11).
>
> The files moved around in the test repo and a lot of new tests are
> failing.
>
> BUG=None
> LOG=N
> R=rossberg
>
> Committed: https://crrev.com/4f2fb3835feff3663146f12be42b01a226d0065e
> Cr-Commit-Position: refs/heads/master@{#27522}

TBR=rossberg@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=None

Review URL: https://codereview.chromium.org/1047713002

Cr-Commit-Position: refs/heads/master@{#27525}
2015-03-30 15:22:54 +00:00
arv
4f2fb3835f [es6] Update test262 tests
This updates test262 to revision d24fd10 (2015/03/11).

The files moved around in the test repo and a lot of new tests are
failing.

BUG=None
LOG=N
R=rossberg

Review URL: https://codereview.chromium.org/1025043002

Cr-Commit-Position: refs/heads/master@{#27522}
2015-03-30 14:28:15 +00:00
Weiliang Lin
a5fa093d0b introduce android_x64 build target
BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/966773005

Cr-Commit-Position: refs/heads/master@{#27055}
2015-03-09 03:23:12 +00:00
michael_dawson
f1ba8d8f86 Contribution of PowerPC port (continuation of 422063005) - AIX Common1
Contribution of PowerPC port (continuation of 422063005 and 817143002). This patch covers
the key changes needed to the common files needed to support AIX. Subsequent
patches will cover:
- changes to update the ppc directories so they are current with the changes
in the rest of the project.
- remaining AIX changes not resolved by 4.8 compiler
- individual optimizations for PPC

This is based off of the GitHub repository
https://github.com/andrewlow/v8ppc

R=danno@chromium.org, svenpanne@chromium.org
BUG=

Review URL: https://codereview.chromium.org/866843003

Cr-Commit-Position: refs/heads/master@{#26343}
2015-01-30 08:02:08 +00:00
mstarzinger
9fbe872137 Enable test coverage for --turbo-deoptimization mode.
R=jarin@chromium.org

Review URL: https://codereview.chromium.org/845193004

Cr-Commit-Position: refs/heads/master@{#26102}
2015-01-16 12:53:30 +00:00
Sven Panne
e4c5b84652 Contribution of PowerPC port (continuation of 422063005)
Contribution of PowerPC port (continuation of 422063005). The inital patch
covers the core changes to the common files.  Subsequent patches will cover
changes to common files to support AIX and to update the ppc directories so
they are current with the changes in the rest of the project.

This is based off of the GitHub repository
https://github.com/andrewlow/v8ppc

BUG=
R=svenpanne@chromium.org, danno@chromium.org, sevnpanne@chromium.org

Review URL: https://codereview.chromium.org/817143002

Cr-Commit-Position: refs/heads/master@{#26091}
2015-01-16 07:42:15 +00:00
machenbach
e04727e720 Double timeout for slow tests.
TBR=svenpanne@chromium.org

Review URL: https://codereview.chromium.org/752733003

Cr-Commit-Position: refs/heads/master@{#25512}
2014-11-26 09:35:02 +00:00
balazs.kilvady
fbcfce69aa Conditional flags for tests - set JS stack size for simulators.
There are separated JS and C stacks on simulators so for some stack
extensive tests (like mozilla/js1_5/extensions/regress-355497) might
cause a C stack overflow and that overflow is not caught by V8. It is
not an issue on real HW. Increasing the C stack also solves the problem
but we have already FLAG_sim_stack_size flag to control the JS stack
size.

This patch makes it possible to add flags to tests conditionally in
.status files.

TEST=mozilla/js1_5/extensions/regress-355497
BUG=v8:3152
LOG=N

Review URL: https://codereview.chromium.org/735723006

Cr-Commit-Position: refs/heads/master@{#25434}
2014-11-20 09:47:00 +00:00
machenbach@chromium.org
0df953c9e1 Fix test driver json output.
TBR=tandrii@chromium.org

Review URL: https://codereview.chromium.org/716713002

Cr-Commit-Position: refs/heads/master@{#25254}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@25254 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-11-11 06:06:42 +00:00
machenbach@chromium.org
3aaaf5c522 Fix serialization of expected outcomes in test driver.
The outcomes are saved as a set and need to be converted into
a list to be json-serializable.

TBR=tandrii@chromium.org

Review URL: https://codereview.chromium.org/710073002

Cr-Commit-Position: refs/heads/master@{#25246}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@25246 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-11-10 14:18:20 +00:00
machenbach@chromium.org
e140bbcafa Add expected outcomes to json test output.
BUG=
R=tandrii@chromium.org

Review URL: https://codereview.chromium.org/713463002

Cr-Commit-Position: refs/heads/master@{#25235}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@25235 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-11-10 10:26:40 +00:00
machenbach@chromium.org
1b4c25e0b7 Fix fast-variants feature in test driver.
BUG=
R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/701933003

Cr-Commit-Position: refs/heads/master@{#25150}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@25150 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-11-05 13:05:46 +00:00
machenbach@chromium.org
e0801075cd Add fast-variants feature to test driver.
BUG=
R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/704723002

Cr-Commit-Position: refs/heads/master@{#25139}
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@25139 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-11-05 10:39:15 +00:00
machenbach@chromium.org
133a7a8647 Let test driver fail if test executable can't list the tests.
TBR=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/605133004

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24253 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-26 13:46:22 +00:00
machenbach@chromium.org
2a92afe1a4 Fix imports in test runner.
TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/532693002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23609 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 12:46:27 +00:00
machenbach@chromium.org
3ca77d4bef Fix test runner imports.
TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/526273003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23597 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 11:18:47 +00:00
machenbach@chromium.org
b4adc1a12b Reland "Refactoring: Make gtest testsuite the default."
BUG=v8:3489
TBR=bmeurer@chromium.org
LOG=n

Review URL: https://codereview.chromium.org/533603002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23585 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:21:03 +00:00
machenbach@chromium.org
b77cb4a793 Revert "Refactoring: Make gtest testsuite the default."
This reverts commit cb8f08658767e7e3182e5deab0473f264f6ce92f.

TBR=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/532673002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23584 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:17:26 +00:00
machenbach@chromium.org
a050734c35 Refactoring: Make gtest testsuite the default.
BUG=v8:3489
R=bmeurer@chromium.org
LOG=n

Review URL: https://codereview.chromium.org/526133003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:11:13 +00:00
machenbach@chromium.org
500b76d5be Only compare allocation output in predictable mode.
This also adds an assert that makes sure allocations are printed.

BUG=391747
LOG=n
R=ishell@chromium.org

Review URL: https://codereview.chromium.org/377213005

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22313 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-10 09:48:31 +00:00
machenbach@chromium.org
562274b60d Fix number of times failures are rerun.
BUG=374134
LOG=n
R=jkummerow@chromium.org
TBR=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/379833003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22303 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 12:48:08 +00:00
machenbach@chromium.org
13f4c91276 Fix test driver output for passes in predictable mode.
Passes in predictable mode were wrongly reported as failures by the json progress indicator. This moves control about what is reported completely to execution.py.

BUG=391747
LOG=n
R=ishell@chromium.org, jkummerow@chromium.org

Review URL: https://codereview.chromium.org/382443004

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22299 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 11:37:36 +00:00
dusan.milosavljevic@rt-rk.com
a0f6878a06 Add mips64 port.
Summary:

- Changes in common code are mainly boilerplate changes,
gyp and test status files updates.

- On mips64 simulator all tests pass from all test units.

- Current issues: mjsunit JS debugger tests fail randomly on HW in release mode.
Corresponding tests are skipped on HW.

- Skipped tests on mips64: test-heap/ReleaseOverReservedPages, mjsunit/debug-*

TEST=
BUG=
R=danno@chromium.org, plind44@gmail.com, ulan@chromium.org

Review URL: https://codereview.chromium.org/371923006

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22297 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 11:08:26 +00:00
machenbach@chromium.org
e3e03d80ef Add predictable mode to test driver.
BUG=391747
LOG=n
R=ishell@chromium.org

Review URL: https://codereview.chromium.org/371363003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22285 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-08 15:59:23 +00:00
machenbach@chromium.org
ba584456e6 Fix result status of rerun flaky tests.
Tests that pass on reruns where wrongly treated as failures. Now the result state can include any of (PASS, FAIL, CRASH, TIMEOUT)

BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/363883003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22186 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-03 09:33:22 +00:00
machenbach@chromium.org
cf88d3c953 Fix rerun barrier for flaky tests.
- The rerun flag doesn't include the first run.
- Timeout is an int. The division makes it 0 for small
numbers.

BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/364793007

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22172 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 15:33:21 +00:00
machenbach@chromium.org
4d55c22827 Add information about rerun tests to json output.
BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/364793004

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22147 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 09:12:07 +00:00
machenbach@chromium.org
ae017be69c Let test runner rerun failures to test for flakes.
When enabled, this dynamically adds jobs that had failures back to the pool. Special json output for flakes will be handled in a separate CL.

BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/360113003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22143 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 08:15:44 +00:00
machenbach@chromium.org
9d0988b0fe Let test driver nuke test perf data on errors.
BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/350913004

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22055 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-06-27 09:51:08 +00:00
mstarzinger@chromium.org
f1204c433b Use non-copying sorting in test harness.
R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/300873006

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21569 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 13:01:47 +00:00
mstarzinger@chromium.org
18ac428252 Add flag to test harness to stop sorting test cases.
R=jkummerow@chromium.org, machenbach@chromium.org

Review URL: https://codereview.chromium.org/307553003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21561 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 10:49:10 +00:00
danno@chromium.org
9c485e182b Introduce x87 port
Support x87-only platform (ia32 without SSE)

R=danno@chromium.org

Review URL: https://codereview.chromium.org/293743005

Patch from Weiliang Lin <weiliang.lin@intel.com>.

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21469 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-23 16:37:27 +00:00
jkummerow@chromium.org
d4d4d30204 Stop using urllib.urlretrieve() directly.
Using urllib for SSL connections when behind a proxy is known to be
broken, so apply the same fix from depot_tools r149742 and use a wrapper
around urllib2 instead.

R=jkummerow@chromium.org
TEST=run test262 behind corporate proxy

Review URL: https://codereview.chromium.org/297663003

Patch from Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com>.

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21402 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-21 09:25:05 +00:00
machenbach@chromium.org
f737580c65 Let test driver export json results.
BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/285193009

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21358 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-19 08:36:53 +00:00
machenbach@chromium.org
04043b3352 Fix keyboard interrupt in test driver.
Drain the queues to prevent failures when queues are garbage collected. Fails when interrupting test262 otherwise.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/286973004

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21334 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 15:01:27 +00:00
machenbach@chromium.org
1d0db84277 Fix deopt fuzzer after test runner changes.
Use the same keyboard interrupt logic as run-tests.
Close the perf database explicitly after one run, as it has two runs on the same architecture. Currently the first run closes on garbage collection, which might corrupt the second run.

BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/284203002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21330 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 12:40:40 +00:00
machenbach@chromium.org
743e07bad0 Run tests sorted by expected runtime.
BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/273613005

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21328 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 12:01:34 +00:00
machenbach@chromium.org
fc437f4007 Introduce a dynamic process pool for the local test driver
The new process pool allows adding jobs after testing has been started. It will also allow to restructure building the job queue (in a follow up CL), so that testing can start instantly while the queue is being built.

Also attempts to clean up the keyboard-interrupt logic. Idea: Only catch keyboard interrupt once per process at the outermost level. Use proper "finally" clauses to clean up everywhere where a keyboard interrupt might occur. Never turn named exceptions into none-exceptions using anonymous "raise".

TEST=python -m unittest pool_unittest
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/275093002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21310 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-14 13:30:57 +00:00
plind44@gmail.com
5a016958c6 MIPS: Add big-endian support for MIPS.
Important notices:

- The snapshot cannot be created for big-endian target in cross-compilation
  environment on little-endian host using simulator.

- In order to have i18n support working on big-endian target, the icudt46b.dat and
  icudt46b_dat.S files should be generated and upstreamed to ICU repo.

- The mjsunit 'nans' test is endian dependent, it is skipped for mips target.

- The zlib and Mandreel from Octane 2.0 benchmark are endian dependent due to
  use of typed arrays.

TEST=
BUG=
R=jkummerow@chromium.org, plind44@gmail.com

Review URL: https://codereview.chromium.org/228943009

Patch from Dusan Milosavljevic <Dusan.Milosavljevic@rt-rk.com>.

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20778 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-15 16:39:21 +00:00
bmeurer@chromium.org
5a564648dd Improve reproducibility of test runs.
Add random seed to run-tests.py, using either a user supplied
value or a random number generated by random.SystemRandom().
This same random seed is passed to all test cases, making sure
that we can easily reproduce test failures that depend on
random numbers (i.e. bugs related to our handwritten ASLR).

Also fix all uses of rand() to make use of our RNG class
instead.

R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/231443002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20637 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-10 07:25:49 +00:00
ulan@chromium.org
5eabc4b802 Run tests on android_arm64.
R=rmcilroy@chromium.org

Review URL: https://codereview.chromium.org/210773003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20295 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-03-27 10:06:53 +00:00
jochen@chromium.org
2ce0bebba1 Rename A64 port to ARM64 port
BUG=354405
R=ulan@chromium.org, rodolph.perfetta@arm.com
LOG=y

Review URL: https://codereview.chromium.org/207823003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20148 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-03-21 09:28:26 +00:00
ulan@chromium.org
e95bc7eec8 Merge experimental/a64 to bleeding_edge.
BUG=v8:3113
LOG=Y
R=jochen@chromium.org, rmcilroy@chromium.org, rodolph.perfetta@arm.com

Review URL: https://codereview.chromium.org/148293020

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@19311 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-02-12 09:19:30 +00:00
machenbach@chromium.org
5be6daa942 Fix deopt fuzzer variable set up.
Adds the simulator variable configuration from r18959 to the deopt fuzzer.

BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/143403004

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18975 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-01-31 09:10:56 +00:00
jkummerow@chromium.org
fa4faff962 'make quickcheck': Assorted improvements.
'make ia32' should not build ia32.optdebug.
'make ia32.clean' should delete ia32.optdebug output.
'make quickcheck' should be terminatable by hitting Ctrl+C just once.

R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/106443002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18259 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-12-05 12:37:24 +00:00
ulan@chromium.org
7f885c709c Print exit code on crash in run-tests.py
R=jkummerow@chromium.org

Review URL: https://chromiumcodereview.appspot.com/99363002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18204 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-12-02 15:46:41 +00:00
machenbach@chromium.org
e6e8acd446 Make test runner more flexible for running fast tests.
Transformed variant flags into named variants. Now, all combinations of variants can be specified on the command line. The old command-line flags are kept for backwards compatibility on the bots.

Added two new test groups: slow and pass|fail. Both are implemented similar to the flaky test feature and allow to either skip or run tests marked as slow or as pass|fail.

R=yangguo@chromium.org

Review URL: https://codereview.chromium.org/85733003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18062 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-25 17:34:52 +00:00
yangguo@chromium.org
aa3518a0f3 Make sure files end with exactly one new line and police this in presubmit.
The changes are (excluding presubmit.py) mechanical. I added the following
lines after the check and iterated the presubmit script until all errors
went away:

f = open(name, "w");
if contents.endswith('\n\n'):
  f.write(contents[0:-1])
else:
  f.write(contents + '\n')

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/82803005

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18017 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-22 13:50:39 +00:00
machenbach@chromium.org
230b47a63b [Sheriff] Mark flaky test.
BUG=v8:2921
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/54423002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17444 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-31 11:48:31 +00:00
svenpanne@chromium.org
882778bf09 Make it possible to run a test only in the standard variant.
Use this for mjsunit/unicode-case-overoptimization, which is not
related to Crankshaft at all and takes ages.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/27704002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17255 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-17 13:09:28 +00:00
machenbach@chromium.org
0cb81e8e62 Add windows to status variable list.
This will allow to modify test expectation for system==windows.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/24528003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16941 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-25 12:37:00 +00:00
jkummerow@chromium.org
e7130a1e2b Migrate to new test status file syntax
R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/23498058

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16919 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-24 12:08:33 +00:00
jochen@chromium.org
a3cef1a9db Skip tests failing on the no-i18n bot
R=machenbach@chromium.org, mstarzinger@chromium.org, jkummerow@chromium.org

Review URL: https://codereview.chromium.org/23708055

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16867 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-20 13:13:57 +00:00
machenbach@chromium.org
e7585af65b Add flaky test classification feature to test suites.
Test expectations can now include outcome: FLAKY.

The test runner can now run a class of tests (flaky|non-flaky|all). All tests are in the non-flaky class that are not marked as FLAKY.

The slash correction for windows is now pulled into the test name method. Currently the progress output on windows contains a mixture of / and \.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/22381003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16080 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-08-06 14:39:39 +00:00
machenbach@chromium.org
f3f14fc7ed Migrate blink tests that are not relevant to blink into a new V8 test suite called 'blink'.
This initial CL contains the new test suite code and two tests for demonstration.

Other tests will follow in a separate CL.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/17260002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@15339 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-06-26 14:23:30 +00:00
machenbach@chromium.org
6a6790be37 HasUnexpectedOutput() is evaluated two times for each test run.
This fix removes the redundant call to allow heavy text comparisons through overwriting of that method.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/17089003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@15162 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-06-14 14:57:14 +00:00
palfia@homejinni.com
17f2177cb3 Add support for JUnit compatible XML output in the new test runner.
Implement a new output method in run-tests.py which
outputs JUnit compatible XML status information.

BUG=

Review URL: https://codereview.chromium.org/13813003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14217 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-04-11 00:22:26 +00:00
ulan@chromium.org
74839e86d7 Modifications to tests and test tools for Native Client V8.
BUG=2614

Review URL: https://chromiumcodereview.appspot.com/13638013
Patch from Brad Chen <bradchen@google.com>.

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14178 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-04-09 08:39:01 +00:00
mstarzinger@chromium.org
366bf736d6 Fix test harness to correctly split extra flags.
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/12094032

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13539 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-29 13:27:23 +00:00
jkummerow@chromium.org
5318c181cf tools/run-tests.py: shlex.split() the value of --command-prefix
BUG=171553

Review URL: https://codereview.chromium.org/12049034

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13469 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-23 11:41:56 +00:00
jkummerow@chromium.org
7924492ce3 tools/run-tests.py: Fixes for Windows
Review URL: https://codereview.chromium.org/11926015

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13436 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-18 14:55:23 +00:00
jkummerow@chromium.org
900720454d tools/runtests.py: Fix cctests on Windows; properly return error code when there were Python exceptions
Review URL: https://codereview.chromium.org/11929020

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13430 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-18 12:44:36 +00:00
ulan@chromium.org
c6feb5065c Fix test runner for Android.
R=jkummerow@chromium.org

Review URL: https://chromiumcodereview.appspot.com/11193055

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12770 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-19 09:55:27 +00:00
jkummerow@chromium.org
230bec1274 Use tools/run-tests.py for "check" targets in the top-level Makefile.
Bonus content: a few minor fixes for run-tests.py

Review URL: https://codereview.chromium.org/11184039

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12758 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-18 14:21:35 +00:00
jkummerow@chromium.org
f10f4fc49c Test runner: More fixes.
- handle Ctrl+C better
- actually show error messages sent by network peers
- gracefully handle missing test cases
- pull test.py and utils.py during server setup (temporary fix for testcfg import issue)

Review URL: https://codereview.chromium.org/11036005

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12648 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-02 08:50:02 +00:00
jkummerow@chromium.org
70c4d78331 test runner: fix wildcard handling in test status files
Review URL: https://codereview.chromium.org/10986086

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12632 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-28 14:59:00 +00:00
jkummerow@chromium.org
975d6e2170 First commit of new tools/run-tests.py
Review URL: https://codereview.chromium.org/10919265

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-24 09:38:46 +00:00