Commit Graph

23 Commits

Author SHA1 Message Date
machenbach
a54f22d418 Make test runner more chatty to avoid it getting killed by buildbot.
NOTRY=true
NOTREECHECKS=true

Review URL: https://codereview.chromium.org/1064043002

Cr-Commit-Position: refs/heads/master@{#27653}
2015-04-08 09:53:39 +00:00
machenbach
e04727e720 Double timeout for slow tests.
TBR=svenpanne@chromium.org

Review URL: https://codereview.chromium.org/752733003

Cr-Commit-Position: refs/heads/master@{#25512}
2014-11-26 09:35:02 +00:00
machenbach@chromium.org
500b76d5be Only compare allocation output in predictable mode.
This also adds an assert that makes sure allocations are printed.

BUG=391747
LOG=n
R=ishell@chromium.org

Review URL: https://codereview.chromium.org/377213005

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22313 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-10 09:48:31 +00:00
machenbach@chromium.org
562274b60d Fix number of times failures are rerun.
BUG=374134
LOG=n
R=jkummerow@chromium.org
TBR=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/379833003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22303 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 12:48:08 +00:00
machenbach@chromium.org
13f4c91276 Fix test driver output for passes in predictable mode.
Passes in predictable mode were wrongly reported as failures by the json progress indicator. This moves control about what is reported completely to execution.py.

BUG=391747
LOG=n
R=ishell@chromium.org, jkummerow@chromium.org

Review URL: https://codereview.chromium.org/382443004

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22299 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 11:37:36 +00:00
machenbach@chromium.org
e3e03d80ef Add predictable mode to test driver.
BUG=391747
LOG=n
R=ishell@chromium.org

Review URL: https://codereview.chromium.org/371363003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22285 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-08 15:59:23 +00:00
machenbach@chromium.org
cf88d3c953 Fix rerun barrier for flaky tests.
- The rerun flag doesn't include the first run.
- Timeout is an int. The division makes it 0 for small
numbers.

BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/364793007

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22172 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 15:33:21 +00:00
machenbach@chromium.org
ae017be69c Let test runner rerun failures to test for flakes.
When enabled, this dynamically adds jobs that had failures back to the pool. Special json output for flakes will be handled in a separate CL.

BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/360113003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22143 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 08:15:44 +00:00
machenbach@chromium.org
9d0988b0fe Let test driver nuke test perf data on errors.
BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/350913004

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22055 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-06-27 09:51:08 +00:00
mstarzinger@chromium.org
f1204c433b Use non-copying sorting in test harness.
R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/300873006

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21569 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 13:01:47 +00:00
mstarzinger@chromium.org
18ac428252 Add flag to test harness to stop sorting test cases.
R=jkummerow@chromium.org, machenbach@chromium.org

Review URL: https://codereview.chromium.org/307553003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21561 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 10:49:10 +00:00
machenbach@chromium.org
1d0db84277 Fix deopt fuzzer after test runner changes.
Use the same keyboard interrupt logic as run-tests.
Close the perf database explicitly after one run, as it has two runs on the same architecture. Currently the first run closes on garbage collection, which might corrupt the second run.

BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/284203002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21330 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 12:40:40 +00:00
machenbach@chromium.org
743e07bad0 Run tests sorted by expected runtime.
BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/273613005

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21328 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 12:01:34 +00:00
machenbach@chromium.org
fc437f4007 Introduce a dynamic process pool for the local test driver
The new process pool allows adding jobs after testing has been started. It will also allow to restructure building the job queue (in a follow up CL), so that testing can start instantly while the queue is being built.

Also attempts to clean up the keyboard-interrupt logic. Idea: Only catch keyboard interrupt once per process at the outermost level. Use proper "finally" clauses to clean up everywhere where a keyboard interrupt might occur. Never turn named exceptions into none-exceptions using anonymous "raise".

TEST=python -m unittest pool_unittest
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/275093002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21310 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-14 13:30:57 +00:00
bmeurer@chromium.org
5a564648dd Improve reproducibility of test runs.
Add random seed to run-tests.py, using either a user supplied
value or a random number generated by random.SystemRandom().
This same random seed is passed to all test cases, making sure
that we can easily reproduce test failures that depend on
random numbers (i.e. bugs related to our handwritten ASLR).

Also fix all uses of rand() to make use of our RNG class
instead.

R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/231443002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20637 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-10 07:25:49 +00:00
machenbach@chromium.org
6a6790be37 HasUnexpectedOutput() is evaluated two times for each test run.
This fix removes the redundant call to allow heavy text comparisons through overwriting of that method.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/17089003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@15162 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-06-14 14:57:14 +00:00
mstarzinger@chromium.org
366bf736d6 Fix test harness to correctly split extra flags.
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/12094032

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13539 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-29 13:27:23 +00:00
jkummerow@chromium.org
5318c181cf tools/run-tests.py: shlex.split() the value of --command-prefix
BUG=171553

Review URL: https://codereview.chromium.org/12049034

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13469 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-23 11:41:56 +00:00
jkummerow@chromium.org
900720454d tools/runtests.py: Fix cctests on Windows; properly return error code when there were Python exceptions
Review URL: https://codereview.chromium.org/11929020

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13430 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-18 12:44:36 +00:00
ulan@chromium.org
c6feb5065c Fix test runner for Android.
R=jkummerow@chromium.org

Review URL: https://chromiumcodereview.appspot.com/11193055

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12770 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-19 09:55:27 +00:00
jkummerow@chromium.org
230bec1274 Use tools/run-tests.py for "check" targets in the top-level Makefile.
Bonus content: a few minor fixes for run-tests.py

Review URL: https://codereview.chromium.org/11184039

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12758 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-18 14:21:35 +00:00
jkummerow@chromium.org
f10f4fc49c Test runner: More fixes.
- handle Ctrl+C better
- actually show error messages sent by network peers
- gracefully handle missing test cases
- pull test.py and utils.py during server setup (temporary fix for testcfg import issue)

Review URL: https://codereview.chromium.org/11036005

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12648 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-02 08:50:02 +00:00
jkummerow@chromium.org
975d6e2170 First commit of new tools/run-tests.py
Review URL: https://codereview.chromium.org/10919265

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-24 09:38:46 +00:00