machenbach
a54f22d418
Make test runner more chatty to avoid it getting killed by buildbot.
...
NOTRY=true
NOTREECHECKS=true
Review URL: https://codereview.chromium.org/1064043002
Cr-Commit-Position: refs/heads/master@{#27653}
2015-04-08 09:53:39 +00:00
machenbach
e04727e720
Double timeout for slow tests.
...
TBR=svenpanne@chromium.org
Review URL: https://codereview.chromium.org/752733003
Cr-Commit-Position: refs/heads/master@{#25512}
2014-11-26 09:35:02 +00:00
machenbach@chromium.org
500b76d5be
Only compare allocation output in predictable mode.
...
This also adds an assert that makes sure allocations are printed.
BUG=391747
LOG=n
R=ishell@chromium.org
Review URL: https://codereview.chromium.org/377213005
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22313 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-10 09:48:31 +00:00
machenbach@chromium.org
562274b60d
Fix number of times failures are rerun.
...
BUG=374134
LOG=n
R=jkummerow@chromium.org
TBR=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/379833003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22303 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 12:48:08 +00:00
machenbach@chromium.org
13f4c91276
Fix test driver output for passes in predictable mode.
...
Passes in predictable mode were wrongly reported as failures by the json progress indicator. This moves control about what is reported completely to execution.py.
BUG=391747
LOG=n
R=ishell@chromium.org , jkummerow@chromium.org
Review URL: https://codereview.chromium.org/382443004
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22299 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 11:37:36 +00:00
machenbach@chromium.org
e3e03d80ef
Add predictable mode to test driver.
...
BUG=391747
LOG=n
R=ishell@chromium.org
Review URL: https://codereview.chromium.org/371363003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22285 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-08 15:59:23 +00:00
machenbach@chromium.org
cf88d3c953
Fix rerun barrier for flaky tests.
...
- The rerun flag doesn't include the first run.
- Timeout is an int. The division makes it 0 for small
numbers.
BUG=374134
LOG=n
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/364793007
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22172 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 15:33:21 +00:00
machenbach@chromium.org
ae017be69c
Let test runner rerun failures to test for flakes.
...
When enabled, this dynamically adds jobs that had failures back to the pool. Special json output for flakes will be handled in a separate CL.
BUG=374134
LOG=n
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/360113003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22143 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 08:15:44 +00:00
machenbach@chromium.org
9d0988b0fe
Let test driver nuke test perf data on errors.
...
BUG=
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/350913004
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22055 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-06-27 09:51:08 +00:00
mstarzinger@chromium.org
f1204c433b
Use non-copying sorting in test harness.
...
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/300873006
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21569 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 13:01:47 +00:00
mstarzinger@chromium.org
18ac428252
Add flag to test harness to stop sorting test cases.
...
R=jkummerow@chromium.org , machenbach@chromium.org
Review URL: https://codereview.chromium.org/307553003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21561 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 10:49:10 +00:00
machenbach@chromium.org
1d0db84277
Fix deopt fuzzer after test runner changes.
...
Use the same keyboard interrupt logic as run-tests.
Close the perf database explicitly after one run, as it has two runs on the same architecture. Currently the first run closes on garbage collection, which might corrupt the second run.
BUG=
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/284203002
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21330 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 12:40:40 +00:00
machenbach@chromium.org
743e07bad0
Run tests sorted by expected runtime.
...
BUG=
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/273613005
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21328 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 12:01:34 +00:00
machenbach@chromium.org
fc437f4007
Introduce a dynamic process pool for the local test driver
...
The new process pool allows adding jobs after testing has been started. It will also allow to restructure building the job queue (in a follow up CL), so that testing can start instantly while the queue is being built.
Also attempts to clean up the keyboard-interrupt logic. Idea: Only catch keyboard interrupt once per process at the outermost level. Use proper "finally" clauses to clean up everywhere where a keyboard interrupt might occur. Never turn named exceptions into none-exceptions using anonymous "raise".
TEST=python -m unittest pool_unittest
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/275093002
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21310 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-14 13:30:57 +00:00
bmeurer@chromium.org
5a564648dd
Improve reproducibility of test runs.
...
Add random seed to run-tests.py, using either a user supplied
value or a random number generated by random.SystemRandom().
This same random seed is passed to all test cases, making sure
that we can easily reproduce test failures that depend on
random numbers (i.e. bugs related to our handwritten ASLR).
Also fix all uses of rand() to make use of our RNG class
instead.
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/231443002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20637 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-10 07:25:49 +00:00
machenbach@chromium.org
6a6790be37
HasUnexpectedOutput() is evaluated two times for each test run.
...
This fix removes the redundant call to allow heavy text comparisons through overwriting of that method.
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/17089003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@15162 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-06-14 14:57:14 +00:00
mstarzinger@chromium.org
366bf736d6
Fix test harness to correctly split extra flags.
...
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/12094032
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13539 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-29 13:27:23 +00:00
jkummerow@chromium.org
5318c181cf
tools/run-tests.py: shlex.split() the value of --command-prefix
...
BUG=171553
Review URL: https://codereview.chromium.org/12049034
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13469 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-23 11:41:56 +00:00
jkummerow@chromium.org
900720454d
tools/runtests.py: Fix cctests on Windows; properly return error code when there were Python exceptions
...
Review URL: https://codereview.chromium.org/11929020
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13430 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-18 12:44:36 +00:00
ulan@chromium.org
c6feb5065c
Fix test runner for Android.
...
R=jkummerow@chromium.org
Review URL: https://chromiumcodereview.appspot.com/11193055
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12770 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-19 09:55:27 +00:00
jkummerow@chromium.org
230bec1274
Use tools/run-tests.py for "check" targets in the top-level Makefile.
...
Bonus content: a few minor fixes for run-tests.py
Review URL: https://codereview.chromium.org/11184039
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12758 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-18 14:21:35 +00:00
jkummerow@chromium.org
f10f4fc49c
Test runner: More fixes.
...
- handle Ctrl+C better
- actually show error messages sent by network peers
- gracefully handle missing test cases
- pull test.py and utils.py during server setup (temporary fix for testcfg import issue)
Review URL: https://codereview.chromium.org/11036005
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12648 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-10-02 08:50:02 +00:00
jkummerow@chromium.org
975d6e2170
First commit of new tools/run-tests.py
...
Review URL: https://codereview.chromium.org/10919265
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@12583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2012-09-24 09:38:46 +00:00