machenbach@chromium.org
133a7a8647
Let test driver fail if test executable can't list the tests.
...
TBR=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/605133004
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24253 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-26 13:46:22 +00:00
machenbach@chromium.org
2a92afe1a4
Fix imports in test runner.
...
TBR=bmeurer@chromium.org
Review URL: https://codereview.chromium.org/532693002
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23609 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 12:46:27 +00:00
machenbach@chromium.org
3ca77d4bef
Fix test runner imports.
...
TBR=bmeurer@chromium.org
Review URL: https://codereview.chromium.org/526273003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23597 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 11:18:47 +00:00
machenbach@chromium.org
b4adc1a12b
Reland "Refactoring: Make gtest testsuite the default."
...
BUG=v8:3489
TBR=bmeurer@chromium.org
LOG=n
Review URL: https://codereview.chromium.org/533603002
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23585 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:21:03 +00:00
machenbach@chromium.org
b77cb4a793
Revert "Refactoring: Make gtest testsuite the default."
...
This reverts commit cb8f08658767e7e3182e5deab0473f264f6ce92f.
TBR=bmeurer@chromium.org
Review URL: https://codereview.chromium.org/532673002
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23584 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:17:26 +00:00
machenbach@chromium.org
a050734c35
Refactoring: Make gtest testsuite the default.
...
BUG=v8:3489
R=bmeurer@chromium.org
LOG=n
Review URL: https://codereview.chromium.org/526133003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@23583 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-09-02 09:11:13 +00:00
machenbach@chromium.org
500b76d5be
Only compare allocation output in predictable mode.
...
This also adds an assert that makes sure allocations are printed.
BUG=391747
LOG=n
R=ishell@chromium.org
Review URL: https://codereview.chromium.org/377213005
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22313 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-10 09:48:31 +00:00
machenbach@chromium.org
562274b60d
Fix number of times failures are rerun.
...
BUG=374134
LOG=n
R=jkummerow@chromium.org
TBR=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/379833003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22303 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 12:48:08 +00:00
machenbach@chromium.org
13f4c91276
Fix test driver output for passes in predictable mode.
...
Passes in predictable mode were wrongly reported as failures by the json progress indicator. This moves control about what is reported completely to execution.py.
BUG=391747
LOG=n
R=ishell@chromium.org , jkummerow@chromium.org
Review URL: https://codereview.chromium.org/382443004
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22299 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 11:37:36 +00:00
dusan.milosavljevic@rt-rk.com
a0f6878a06
Add mips64 port.
...
Summary:
- Changes in common code are mainly boilerplate changes,
gyp and test status files updates.
- On mips64 simulator all tests pass from all test units.
- Current issues: mjsunit JS debugger tests fail randomly on HW in release mode.
Corresponding tests are skipped on HW.
- Skipped tests on mips64: test-heap/ReleaseOverReservedPages, mjsunit/debug-*
TEST=
BUG=
R=danno@chromium.org , plind44@gmail.com , ulan@chromium.org
Review URL: https://codereview.chromium.org/371923006
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22297 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-09 11:08:26 +00:00
machenbach@chromium.org
e3e03d80ef
Add predictable mode to test driver.
...
BUG=391747
LOG=n
R=ishell@chromium.org
Review URL: https://codereview.chromium.org/371363003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22285 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-08 15:59:23 +00:00
machenbach@chromium.org
ba584456e6
Fix result status of rerun flaky tests.
...
Tests that pass on reruns where wrongly treated as failures. Now the result state can include any of (PASS, FAIL, CRASH, TIMEOUT)
BUG=374134
LOG=n
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/363883003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22186 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-03 09:33:22 +00:00
machenbach@chromium.org
cf88d3c953
Fix rerun barrier for flaky tests.
...
- The rerun flag doesn't include the first run.
- Timeout is an int. The division makes it 0 for small
numbers.
BUG=374134
LOG=n
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/364793007
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22172 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 15:33:21 +00:00
machenbach@chromium.org
4d55c22827
Add information about rerun tests to json output.
...
BUG=374134
LOG=n
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/364793004
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22147 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 09:12:07 +00:00
machenbach@chromium.org
ae017be69c
Let test runner rerun failures to test for flakes.
...
When enabled, this dynamically adds jobs that had failures back to the pool. Special json output for flakes will be handled in a separate CL.
BUG=374134
LOG=n
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/360113003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22143 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 08:15:44 +00:00
machenbach@chromium.org
9d0988b0fe
Let test driver nuke test perf data on errors.
...
BUG=
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/350913004
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22055 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-06-27 09:51:08 +00:00
mstarzinger@chromium.org
f1204c433b
Use non-copying sorting in test harness.
...
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/300873006
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21569 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 13:01:47 +00:00
mstarzinger@chromium.org
18ac428252
Add flag to test harness to stop sorting test cases.
...
R=jkummerow@chromium.org , machenbach@chromium.org
Review URL: https://codereview.chromium.org/307553003
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21561 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 10:49:10 +00:00
danno@chromium.org
9c485e182b
Introduce x87 port
...
Support x87-only platform (ia32 without SSE)
R=danno@chromium.org
Review URL: https://codereview.chromium.org/293743005
Patch from Weiliang Lin <weiliang.lin@intel.com>.
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21469 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-23 16:37:27 +00:00
jkummerow@chromium.org
d4d4d30204
Stop using urllib.urlretrieve() directly.
...
Using urllib for SSL connections when behind a proxy is known to be
broken, so apply the same fix from depot_tools r149742 and use a wrapper
around urllib2 instead.
R=jkummerow@chromium.org
TEST=run test262 behind corporate proxy
Review URL: https://codereview.chromium.org/297663003
Patch from Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com>.
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21402 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-21 09:25:05 +00:00
machenbach@chromium.org
f737580c65
Let test driver export json results.
...
BUG=374134
LOG=n
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/285193009
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21358 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-19 08:36:53 +00:00
machenbach@chromium.org
04043b3352
Fix keyboard interrupt in test driver.
...
Drain the queues to prevent failures when queues are garbage collected. Fails when interrupting test262 otherwise.
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/286973004
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21334 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 15:01:27 +00:00
machenbach@chromium.org
1d0db84277
Fix deopt fuzzer after test runner changes.
...
Use the same keyboard interrupt logic as run-tests.
Close the perf database explicitly after one run, as it has two runs on the same architecture. Currently the first run closes on garbage collection, which might corrupt the second run.
BUG=
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/284203002
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21330 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 12:40:40 +00:00
machenbach@chromium.org
743e07bad0
Run tests sorted by expected runtime.
...
BUG=
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/273613005
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21328 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 12:01:34 +00:00
machenbach@chromium.org
fc437f4007
Introduce a dynamic process pool for the local test driver
...
The new process pool allows adding jobs after testing has been started. It will also allow to restructure building the job queue (in a follow up CL), so that testing can start instantly while the queue is being built.
Also attempts to clean up the keyboard-interrupt logic. Idea: Only catch keyboard interrupt once per process at the outermost level. Use proper "finally" clauses to clean up everywhere where a keyboard interrupt might occur. Never turn named exceptions into none-exceptions using anonymous "raise".
TEST=python -m unittest pool_unittest
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/275093002
git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21310 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-14 13:30:57 +00:00
plind44@gmail.com
5a016958c6
MIPS: Add big-endian support for MIPS.
...
Important notices:
- The snapshot cannot be created for big-endian target in cross-compilation
environment on little-endian host using simulator.
- In order to have i18n support working on big-endian target, the icudt46b.dat and
icudt46b_dat.S files should be generated and upstreamed to ICU repo.
- The mjsunit 'nans' test is endian dependent, it is skipped for mips target.
- The zlib and Mandreel from Octane 2.0 benchmark are endian dependent due to
use of typed arrays.
TEST=
BUG=
R=jkummerow@chromium.org , plind44@gmail.com
Review URL: https://codereview.chromium.org/228943009
Patch from Dusan Milosavljevic <Dusan.Milosavljevic@rt-rk.com>.
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20778 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-15 16:39:21 +00:00
bmeurer@chromium.org
5a564648dd
Improve reproducibility of test runs.
...
Add random seed to run-tests.py, using either a user supplied
value or a random number generated by random.SystemRandom().
This same random seed is passed to all test cases, making sure
that we can easily reproduce test failures that depend on
random numbers (i.e. bugs related to our handwritten ASLR).
Also fix all uses of rand() to make use of our RNG class
instead.
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/231443002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20637 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-10 07:25:49 +00:00
ulan@chromium.org
5eabc4b802
Run tests on android_arm64.
...
R=rmcilroy@chromium.org
Review URL: https://codereview.chromium.org/210773003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20295 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-03-27 10:06:53 +00:00
jochen@chromium.org
2ce0bebba1
Rename A64 port to ARM64 port
...
BUG=354405
R=ulan@chromium.org , rodolph.perfetta@arm.com
LOG=y
Review URL: https://codereview.chromium.org/207823003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20148 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-03-21 09:28:26 +00:00
ulan@chromium.org
e95bc7eec8
Merge experimental/a64 to bleeding_edge.
...
BUG=v8:3113
LOG=Y
R=jochen@chromium.org , rmcilroy@chromium.org , rodolph.perfetta@arm.com
Review URL: https://codereview.chromium.org/148293020
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@19311 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-02-12 09:19:30 +00:00
machenbach@chromium.org
5be6daa942
Fix deopt fuzzer variable set up.
...
Adds the simulator variable configuration from r18959 to the deopt fuzzer.
BUG=
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/143403004
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18975 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-01-31 09:10:56 +00:00
jkummerow@chromium.org
fa4faff962
'make quickcheck': Assorted improvements.
...
'make ia32' should not build ia32.optdebug.
'make ia32.clean' should delete ia32.optdebug output.
'make quickcheck' should be terminatable by hitting Ctrl+C just once.
R=bmeurer@chromium.org
Review URL: https://codereview.chromium.org/106443002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18259 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-12-05 12:37:24 +00:00
ulan@chromium.org
7f885c709c
Print exit code on crash in run-tests.py
...
R=jkummerow@chromium.org
Review URL: https://chromiumcodereview.appspot.com/99363002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18204 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-12-02 15:46:41 +00:00
machenbach@chromium.org
e6e8acd446
Make test runner more flexible for running fast tests.
...
Transformed variant flags into named variants. Now, all combinations of variants can be specified on the command line. The old command-line flags are kept for backwards compatibility on the bots.
Added two new test groups: slow and pass|fail. Both are implemented similar to the flaky test feature and allow to either skip or run tests marked as slow or as pass|fail.
R=yangguo@chromium.org
Review URL: https://codereview.chromium.org/85733003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18062 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-25 17:34:52 +00:00
yangguo@chromium.org
aa3518a0f3
Make sure files end with exactly one new line and police this in presubmit.
...
The changes are (excluding presubmit.py) mechanical. I added the following
lines after the check and iterated the presubmit script until all errors
went away:
f = open(name, "w");
if contents.endswith('\n\n'):
f.write(contents[0:-1])
else:
f.write(contents + '\n')
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/82803005
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18017 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-11-22 13:50:39 +00:00
machenbach@chromium.org
230b47a63b
[Sheriff] Mark flaky test.
...
BUG=v8:2921
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/54423002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17444 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-31 11:48:31 +00:00
svenpanne@chromium.org
882778bf09
Make it possible to run a test only in the standard variant.
...
Use this for mjsunit/unicode-case-overoptimization, which is not
related to Crankshaft at all and takes ages.
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/27704002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17255 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-10-17 13:09:28 +00:00
jkummerow@chromium.org
a01e029882
Delete references to old_statusfile.py
...
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/24576002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@17000 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-30 09:52:21 +00:00
machenbach@chromium.org
0cb81e8e62
Add windows to status variable list.
...
This will allow to modify test expectation for system==windows.
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/24528003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16941 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-25 12:37:00 +00:00
jkummerow@chromium.org
e7130a1e2b
Migrate to new test status file syntax
...
R=machenbach@chromium.org
Review URL: https://codereview.chromium.org/23498058
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16919 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-24 12:08:33 +00:00
jochen@chromium.org
a3cef1a9db
Skip tests failing on the no-i18n bot
...
R=machenbach@chromium.org , mstarzinger@chromium.org , jkummerow@chromium.org
Review URL: https://codereview.chromium.org/23708055
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16867 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-20 13:13:57 +00:00
machenbach@chromium.org
1a3f3b0569
Skip intl tests from test262 when intl support is disabled.
...
Skips the whole intl directory if intl support is disabled through the test driver.
One test outside the intl directory is skipped separately. It will be handled in an extra CL.
R=jochen@chromium.org , mstarzinger@chromium.org
Review URL: https://codereview.chromium.org/23792008
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16854 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-20 09:26:25 +00:00
machenbach@chromium.org
e7585af65b
Add flaky test classification feature to test suites.
...
Test expectations can now include outcome: FLAKY.
The test runner can now run a class of tests (flaky|non-flaky|all). All tests are in the non-flaky class that are not marked as FLAKY.
The slash correction for windows is now pulled into the test name method. Currently the progress output on windows contains a mixture of / and \.
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/22381003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16080 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-08-06 14:39:39 +00:00
machenbach@chromium.org
f3f14fc7ed
Migrate blink tests that are not relevant to blink into a new V8 test suite called 'blink'.
...
This initial CL contains the new test suite code and two tests for demonstration.
Other tests will follow in a separate CL.
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/17260002
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@15339 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-06-26 14:23:30 +00:00
machenbach@chromium.org
6a6790be37
HasUnexpectedOutput() is evaluated two times for each test run.
...
This fix removes the redundant call to allow heavy text comparisons through overwriting of that method.
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/17089003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@15162 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-06-14 14:57:14 +00:00
palfia@homejinni.com
17f2177cb3
Add support for JUnit compatible XML output in the new test runner.
...
Implement a new output method in run-tests.py which
outputs JUnit compatible XML status information.
BUG=
Review URL: https://codereview.chromium.org/13813003
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14217 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-04-11 00:22:26 +00:00
ulan@chromium.org
74839e86d7
Modifications to tests and test tools for Native Client V8.
...
BUG=2614
Review URL: https://chromiumcodereview.appspot.com/13638013
Patch from Brad Chen <bradchen@google.com>.
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@14178 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-04-09 08:39:01 +00:00
mstarzinger@chromium.org
366bf736d6
Fix test harness to correctly split extra flags.
...
R=jkummerow@chromium.org
Review URL: https://codereview.chromium.org/12094032
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13539 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-29 13:27:23 +00:00
jkummerow@chromium.org
5318c181cf
tools/run-tests.py: shlex.split() the value of --command-prefix
...
BUG=171553
Review URL: https://codereview.chromium.org/12049034
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13469 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-23 11:41:56 +00:00
jkummerow@chromium.org
7924492ce3
tools/run-tests.py: Fixes for Windows
...
Review URL: https://codereview.chromium.org/11926015
git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@13436 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-01-18 14:55:23 +00:00