Commit Graph

46 Commits

Author SHA1 Message Date
machenbach@chromium.org
f737580c65 Let test driver export json results.
BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/285193009

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21358 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-19 08:36:53 +00:00
jkummerow@chromium.org
43c170eb44 Move check for existence of generated tests from run-tests.py to presubmit.py
Inspired by https://codereview.chromium.org/275143002#msg3

R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/290023002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21341 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-16 13:18:13 +00:00
machenbach@chromium.org
fc437f4007 Introduce a dynamic process pool for the local test driver
The new process pool allows adding jobs after testing has been started. It will also allow to restructure building the job queue (in a follow up CL), so that testing can start instantly while the queue is being built.

Also attempts to clean up the keyboard-interrupt logic. Idea: Only catch keyboard interrupt once per process at the outermost level. Use proper "finally" clauses to clean up everywhere where a keyboard interrupt might occur. Never turn named exceptions into none-exceptions using anonymous "raise".

TEST=python -m unittest pool_unittest
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/275093002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21310 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-14 13:30:57 +00:00
jkummerow@chromium.org
9866670c26 Add test case generator for runtime functions
R=dslomov@chromium.org, machenbach@chromium.org

Review URL: https://codereview.chromium.org/250923002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21199 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-08 13:11:59 +00:00
jkummerow@chromium.org
e1bbd26794 Refactor mjsunit/fuzz-natives-* into a separate test suite.
R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/252143002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21190 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-07 12:31:26 +00:00
plind44@gmail.com
5a016958c6 MIPS: Add big-endian support for MIPS.
Important notices:

- The snapshot cannot be created for big-endian target in cross-compilation
  environment on little-endian host using simulator.

- In order to have i18n support working on big-endian target, the icudt46b.dat and
  icudt46b_dat.S files should be generated and upstreamed to ICU repo.

- The mjsunit 'nans' test is endian dependent, it is skipped for mips target.

- The zlib and Mandreel from Octane 2.0 benchmark are endian dependent due to
  use of typed arrays.

TEST=
BUG=
R=jkummerow@chromium.org, plind44@gmail.com

Review URL: https://codereview.chromium.org/228943009

Patch from Dusan Milosavljevic <Dusan.Milosavljevic@rt-rk.com>.

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20778 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-15 16:39:21 +00:00
bmeurer@chromium.org
5a564648dd Improve reproducibility of test runs.
Add random seed to run-tests.py, using either a user supplied
value or a random number generated by random.SystemRandom().
This same random seed is passed to all test cases, making sure
that we can easily reproduce test failures that depend on
random numbers (i.e. bugs related to our handwritten ASLR).

Also fix all uses of rand() to make use of our RNG class
instead.

R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/231443002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20637 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-10 07:25:49 +00:00
machenbach@chromium.org
a430449c68 Add the ability to disable tests in nosnap mode.
BUG=v8:3216
LOG=n
R=dcarney@chromium.org

Review URL: https://codereview.chromium.org/230743002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20617 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-09 12:57:43 +00:00
ulan@chromium.org
5eabc4b802 Run tests on android_arm64.
R=rmcilroy@chromium.org

Review URL: https://codereview.chromium.org/210773003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20295 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-03-27 10:06:53 +00:00
jochen@chromium.org
2ce0bebba1 Rename A64 port to ARM64 port
BUG=354405
R=ulan@chromium.org, rodolph.perfetta@arm.com
LOG=y

Review URL: https://codereview.chromium.org/207823003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20148 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-03-21 09:28:26 +00:00
jochen@chromium.org
9fccfc37c8 Prepare switch from a64 to arm64
BUG=354405
R=machenbach@chromium.org
LOG=n

Review URL: https://codereview.chromium.org/207793002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20147 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-03-21 08:47:03 +00:00
dcarney@chromium.org
499b31e222 Crash like chrome. Currently, most chrome crashes do not yield useful stack traces as v8 does a silent abort and chrome's crash symbolization does not kick in.
R=svenpanne@chromium.org

BUG=

Review URL: https://codereview.chromium.org/179793004

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@19544 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-02-25 13:53:06 +00:00
ulan@chromium.org
e95bc7eec8 Merge experimental/a64 to bleeding_edge.
BUG=v8:3113
LOG=Y
R=jochen@chromium.org, rmcilroy@chromium.org, rodolph.perfetta@arm.com

Review URL: https://codereview.chromium.org/148293020

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@19311 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-02-12 09:19:30 +00:00
machenbach@chromium.org
5be6daa942 Fix deopt fuzzer variable set up.
Adds the simulator variable configuration from r18959 to the deopt fuzzer.

BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/143403004

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18975 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-01-31 09:10:56 +00:00
plind44@gmail.com
51107bad1c Skip webkit/function-apply-aliased.js when running on simulators.
The webkit/function-apply-aliased.js test fails on simulators (both MIPS
and ARM) as the printed output does not match to the expected. The
failing test forces a stack overflow exception and the ToString()
operation of the exception object fails because of an other stack
overflow and returns an empty string.

The problem is that on hardware a common JS and C stack is used so the
stack overflow can be caught in C functions also while on simulator
separated JS and C stacks are used.

This patch adds a "sim" condition to test .status files to skip tests
only on simulator.

LOG=N
BUG=v8:3124
R=jkummerow@chromium.org, plind44@gmail.com

Review URL: https://codereview.chromium.org/139233005

Patch from Balazs Kilvady <kilvadyb@homejinni.com>.

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18959 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-01-30 18:04:33 +00:00
machenbach@chromium.org
5d8e2f3435 Make 'ASAN' configurable in test status file.
BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/149173006

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18922 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-01-29 15:30:41 +00:00
machenbach@chromium.org
1864f7388e Add infrastructure for skipping tests in GC stress mode.
Also move the GC stress configuration from the buildbot to the test runner.

BUG=
R=jkummerow@chromium.org, mvstanton@chromium.org

Review URL: https://codereview.chromium.org/141653008

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18708 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-01-21 12:41:25 +00:00
jkummerow@chromium.org
fa4faff962 'make quickcheck': Assorted improvements.
'make ia32' should not build ia32.optdebug.
'make ia32.clean' should delete ia32.optdebug output.
'make quickcheck' should be terminatable by hitting Ctrl+C just once.

R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/106443002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18259 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-12-05 12:37:24 +00:00
jkummerow@chromium.org
5e535ae6c1 'make quickcheck': don't overwrite debug output.
Achieved by turning optdebug into a first-class compilation mode.

R=bmeurer@chromium.org, machenbach@chromium.org

Review URL: https://codereview.chromium.org/98963002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18236 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-12-03 14:24:40 +00:00
machenbach@chromium.org
59ff4ec657 Add fast make and test all target.
The new target called "quickcheck" builds and tests with several configurations using recent test runner speed improvements.

Runtime from scratch (after make clean): < 8 min
Runtime tests only: < 3 min

TEST=make quickcheck
R=jkummerow@chromium.org, yangguo@chromium.org

Review URL: https://codereview.chromium.org/94703002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18120 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-28 13:18:04 +00:00
machenbach@chromium.org
c95173b2eb Increase test runner speed.
Let the test runner preserve the order of test suites to let suites with long running tests run first.

Mark some tests as slow that can now be skipped via --slow-tests=skip.

BUG=
R=yangguo@chromium.org

Review URL: https://codereview.chromium.org/88343002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18086 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-26 16:53:04 +00:00
machenbach@chromium.org
e6e8acd446 Make test runner more flexible for running fast tests.
Transformed variant flags into named variants. Now, all combinations of variants can be specified on the command line. The old command-line flags are kept for backwards compatibility on the bots.

Added two new test groups: slow and pass|fail. Both are implemented similar to the flaky test feature and allow to either skip or run tests marked as slow or as pass|fail.

R=yangguo@chromium.org

Review URL: https://codereview.chromium.org/85733003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18062 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-25 17:34:52 +00:00
machenbach@chromium.org
3cb584c2b4 Add no-variants option to test runner.
BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/48883003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17440 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-31 10:14:02 +00:00
svenpanne@chromium.org
882778bf09 Make it possible to run a test only in the standard variant.
Use this for mjsunit/unicode-case-overoptimization, which is not
related to Crankshaft at all and takes ages.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/27704002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17255 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-17 13:09:28 +00:00
jochen@chromium.org
a3cef1a9db Skip tests failing on the no-i18n bot
R=machenbach@chromium.org, mstarzinger@chromium.org, jkummerow@chromium.org

Review URL: https://codereview.chromium.org/23708055

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16867 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-20 13:13:57 +00:00
machenbach@chromium.org
1a3f3b0569 Skip intl tests from test262 when intl support is disabled.
Skips the whole intl directory if intl support is disabled through the test driver.

One test outside the intl directory is skipped separately. It will be handled in an extra CL.

R=jochen@chromium.org, mstarzinger@chromium.org

Review URL: https://codereview.chromium.org/23792008

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16854 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-20 09:26:25 +00:00
jochen@chromium.org
416f2a9ba7 Turn on i18n support by default
This reverts commit r16789

BUG=none
R=machenbach@chromium.org, mstarzinger@chromium.org

Review URL: https://codereview.chromium.org/23522063

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16827 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-19 12:07:03 +00:00
machenbach@chromium.org
eca42c42cb Revert "Turn on i18n support by default", "Update test262 expectations, now that 15.5.4.9_CE passes", "Make testing i18n the default.", "Mark two failing mozilla tests as failing.", "Mark mozilla tests as flaky instead of failing" and "Fix test expectations for i18n."
This reverts commit 16783-16788 due to several crashers on nosnap builder.

TBR=jochen@chromium.org

Review URL: https://codereview.chromium.org/23455055

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16789 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-18 13:50:00 +00:00
machenbach@chromium.org
7e0cc85765 Make testing i18n the default.
If i18n support is on when building with make, the intl test suite is automatically run with "check".

On the bots, we'll turn it off explicitly, where required.

R=jochen@chromium.org

Review URL: https://codereview.chromium.org/23702044

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16785 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-18 08:58:01 +00:00
machenbach@chromium.org
9a11424a6a Revert "Increase test timeout on slow architectures."
This reverts commit 200c7cbdbc90ec2001ee4e0da7470cb508585e0d.

BUG=
R=bmeurer@chromium.org

Review URL: https://codereview.chromium.org/23475036

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16666 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-12 07:39:44 +00:00
machenbach@chromium.org
119b59ac5d Increase test timeout on slow architectures.
BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/24037002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16582 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-08 19:50:55 +00:00
machenbach@chromium.org
e7585af65b Add flaky test classification feature to test suites.
Test expectations can now include outcome: FLAKY.

The test runner can now run a class of tests (flaky|non-flaky|all). All tests are in the non-flaky class that are not marked as FLAKY.

The slash correction for windows is now pulled into the test name method. Currently the progress output on windows contains a mixture of / and \.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/22381003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16080 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-08-06 14:39:39 +00:00
machenbach@chromium.org
a0734ba3af Add deopt fuzzer tool.
Can be run as a stand-alone script like run-tests.

Executes first all tests of a given test suite to collect the maximum number of possible deopt points. Runs then a fuzzing phase with artificial deoptimizations triggered during testing.

Works for now with mjsunit and ia32 only.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/19931005

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@15855 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-07-24 12:04:29 +00:00
palfia@homejinni.com
17f2177cb3 Add support for JUnit compatible XML output in the new test runner.
Implement a new output method in run-tests.py which
outputs JUnit compatible XML status information.

BUG=

Review URL: https://codereview.chromium.org/13813003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14217 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-04-11 00:22:26 +00:00
ulan@chromium.org
74839e86d7 Modifications to tests and test tools for Native Client V8.
BUG=2614

Review URL: https://chromiumcodereview.appspot.com/13638013
Patch from Brad Chen <bradchen@google.com>.

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14178 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-04-09 08:39:01 +00:00
yangguo@chromium.org
0d63cef35b Constant fold math and string operations.
R=jkummerow@chromium.org
BUG=

Review URL: https://chromiumcodereview.appspot.com/12315005

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13705 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-02-21 11:40:37 +00:00
mstarzinger@chromium.org
366bf736d6 Fix test harness to correctly split extra flags.
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/12094032

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13539 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-29 13:27:23 +00:00
jkummerow@chromium.org
5318c181cf tools/run-tests.py: shlex.split() the value of --command-prefix
BUG=171553

Review URL: https://codereview.chromium.org/12049034

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13469 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-23 11:41:56 +00:00
mstarzinger@chromium.org
70c95681e1 Fix test harness to accept camel-case modes.
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/12026018

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13428 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-18 12:20:59 +00:00
jkummerow@chromium.org
5677ee6565 Fix tools/run-tests.py --report when tests are specified on the command line
Review URL: https://codereview.chromium.org/11189136

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12797 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-23 08:59:53 +00:00
jkummerow@chromium.org
5ea870f855 tools/run-tests.py: A few timeout-related fixes
Review URL: https://codereview.chromium.org/11230029

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12789 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-22 14:13:01 +00:00
ulan@chromium.org
c6feb5065c Fix test runner for Android.
R=jkummerow@chromium.org

Review URL: https://chromiumcodereview.appspot.com/11193055

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12770 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-19 09:55:27 +00:00
jkummerow@chromium.org
230bec1274 Use tools/run-tests.py for "check" targets in the top-level Makefile.
Bonus content: a few minor fixes for run-tests.py

Review URL: https://codereview.chromium.org/11184039

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12758 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-18 14:21:35 +00:00
svenpanne@chromium.org
f03fd70d9e Added a simple dead code removal phase.
We iteratively remove all dead Hydrogen instruction until we reach a fixed point. We consider an instruction dead if it is unused, has no observable side effects and is deletable. The last part of the condition is currently not very nice: We basically have to whitelist "safe" instructions, because we are missing more detailed dependencies and/or more detailed tracking of side effects.

We disable dead code elimination for now in our test runners, because we have tons of poorly written tests which wouldn't test anymore what they are supposed to test with this phase enabled. To get test coverage for dead code elimination itself, we should enable it on a few build bots. This is not really a perfect state, but the best we can do for now.

This patch includes a few const-correctness fixes, most of them were necessary for this CL.

Review URL: https://codereview.chromium.org/11088027

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12697 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-11 10:52:58 +00:00
mstarzinger@chromium.org
09e73bc79b Test runner: remove noisy debug output.
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/10993080

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12634 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-28 15:53:46 +00:00
jkummerow@chromium.org
975d6e2170 First commit of new tools/run-tests.py
Review URL: https://codereview.chromium.org/10919265

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-24 09:38:46 +00:00