Commit Graph

44 Commits

Author SHA1 Message Date
Michal Majewski
d5be2832e3 TestRunner initial refactor
Preparation for sharing more code between deopt fuzzer and normal test runner.

Bug: v8:6917
Change-Id: Id8022c7597544ffddf5d1b0a0d73bb4a9d163fc3
Reviewed-on: https://chromium-review.googlesource.com/715417
Commit-Queue: Michał Majewski <majeski@google.com>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48521}
2017-10-13 08:52:28 +00:00
Michael Achenbach
14bfa18b45 [test] Remove deprecated test-download feature
Bug: v8:6917
Change-Id: I3889cd0d059c3473a7b83eb298734a7a6a8a1de5
Reviewed-on: https://chromium-review.googlesource.com/712175
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#48467}
2017-10-11 15:18:19 +00:00
Michael Achenbach
dbfe008b43 [test] Add test isolation and parameters for ubsan
This adds a missing swarming config to the ubsan builders to fix
test isolation.

This also adds ubsan_vptr to the inferred options from build metadata.

For ubsan_vptr builds, the test runner makes sure to set up the
required options.

TBR=ishell@chromium.org

Bug: chromium:726584
Change-Id: I9667ba2b6d9d0f363f7cc94d23308e09da002d99
Reviewed-on: https://chromium-review.googlesource.com/577689
Reviewed-by: Igor Sheludko <ishell@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46866}
2017-07-25 11:42:01 +00:00
Michael Starzinger
05f3b27cdd Reland "[flags] Remove some dead Crankshaft flags."
This is a reland of ec6da23bfe
Original change's description:
> [flags] Remove some dead Crankshaft flags.
> 
> R=bmeurer@chromium.org
> BUG=v8:6408
> 
> Change-Id: I34abbcdc2fc47df44938bac0e59f9982c935c657
> Reviewed-on: https://chromium-review.googlesource.com/569963
> Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
> Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#46631}

Bug: v8:6408
Change-Id: I8a856e25d56e27bccb79588b2e5ee4369d7c5fe5
Reviewed-on: https://chromium-review.googlesource.com/570050
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46637}
2017-07-13 13:15:34 +00:00
Michael Starzinger
5d16e866e7 Revert "[flags] Remove some dead Crankshaft flags."
This reverts commit ec6da23bfe.

Reason for revert: Requires infrastructure changes first. Will reland after changes have happened.

Original change's description:
> [flags] Remove some dead Crankshaft flags.
> 
> R=​bmeurer@chromium.org
> BUG=v8:6408
> 
> Change-Id: I34abbcdc2fc47df44938bac0e59f9982c935c657
> Reviewed-on: https://chromium-review.googlesource.com/569963
> Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
> Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#46631}

TBR=mstarzinger@chromium.org,bmeurer@chromium.org

Change-Id: Iee077911ae7d877c6a9d2edb548e3c04345b47ce
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: v8:6408
Reviewed-on: https://chromium-review.googlesource.com/570049
Reviewed-by: Michael Starzinger <mstarzinger@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46632}
2017-07-13 12:24:56 +00:00
Michael Starzinger
ec6da23bfe [flags] Remove some dead Crankshaft flags.
R=bmeurer@chromium.org
BUG=v8:6408

Change-Id: I34abbcdc2fc47df44938bac0e59f9982c935c657
Reviewed-on: https://chromium-review.googlesource.com/569963
Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
Commit-Queue: Michael Starzinger <mstarzinger@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46631}
2017-07-13 11:56:14 +00:00
Camillo Bruni
34874b3b19 Reland "[mjsunit] Improve mjsunit stracktrace readability"
This is a reland of f720d024dc
Original change's description:
> [mjsunit] Improve mjsunit stracktrace readability
> 
> Format the function name and file-position into proper columns to easily spot
> where the test code ends and the mjsunit framework code starts.
> 
> BEFORE:
> Stack: Error
>     at new MjsUnitAssertionError (test/mjsunit/mjsunit.js:36:18)
>     at failWithMessage (test/mjsunit/mjsunit.js:310:11)
>     at fail (test/mjsunit/mjsunit.js:327:12)
>     at assertEquals (test/mjsunit/mjsunit.js:398:7)
>     at closure (test/mjsunit/regress/regress-4121.js:20:7)
>     at literals_sharing_test (test/mjsunit/regress/regress-4121.js:27:3)
>     at test (test/mjsunit/regress/regress-4121.js:37:5)
>     at eval (eval at <anonymous> (test/mjsunit/regress/regress-4121.js:49:6), <anonymous>:1:1)
>     at test/mjsunit/regress/regress-4121.js:49:6
>     at Array.forEach.call (test/mjsunit/regress/regress-4121.js:50:7)
>     throw new MjsUnitAssertionError(message);
> 
> AFTER:
> Stack: MjsUnitAssertionError
>     at assertEquals          test/mjsunit/mjsunit.js 398:7
>     at closure               test/mjsunit/regress/regress-4121.js 20:7
>     at literals_sharing_test test/mjsunit/regress/regress-4121.js 27:3
>     at test                  test/mjsunit/regress/regress-4121.js 37:5
>     at eval                  eval at <anonymous> (test/mjsunit/regress/regress-4121.js:49:6)
>     at                       test/mjsunit/regress/regress-4121.js 49:6
>     at Array.forEach.call    test/mjsunit/regress/regress-4121.js 50:7
>     throw new MjsUnitAssertionError(message);
> 
> 
> Change-Id: Iad3460a648e26effb43c00426ab043743ee6a138
> Reviewed-on: https://chromium-review.googlesource.com/563627
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Igor Sheludko <ishell@chromium.org>
> Commit-Queue: Camillo Bruni <cbruni@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#46589}

Change-Id: I44bf07f7be4114369315605542cafd17345b4397
Reviewed-on: https://chromium-review.googlesource.com/567063
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#46602}
2017-07-12 19:01:21 +00:00
mstarzinger
651c1b86a2 [compiler] Make --debug-code the default in debug builds.
This enables {FLAG_debug_code} by default in debug builds. The advantage
is that generated code contained within the snapshot will contain such
debug code. Before we would only get coverage for these pieces with the
no-snapshot builds, which have a meager coverage. One can still pass the
inverse --no-debug-code flag to ensure generated code remains readable
within debug builds as well.

R=machenbach@chromium.org

Review-Url: https://codereview.chromium.org/2528913002
Cr-Commit-Position: refs/heads/master@{#41451}
2016-12-02 11:36:55 +00:00
machenbach
03f5124822 [test] Enable test status filtering by variant
This adds the possibility to address test cases in the
status file with the variant under which the test is running.
This is only allowed in top-level sections.

Example:
[{
  'test-case': [PASS, SLOW],
}]

['variant == foo', {
  'test-case': [FAIL],
}]

The test case "test-case" is marked as slow in all variants.
Additionally, in variant foo, it'll be expected to fail.

This CL also exemplifies the new feature with test cases
running under the ignition_turbofan variant. The
corresponding legacy flag is deprecated.

BUG=v8:5238

Review-Url: https://codereview.chromium.org/2203013002
Cr-Commit-Position: refs/heads/master@{#38342}
2016-08-04 14:42:16 +00:00
machenbach
a23a74a7be [test] Remove deprecated --ignition test-runner flag
The flag was introduced for ignition development. It can only
be used when running ignition tests in isolation on the bots.

The bots only use ignition_turbo in isolation since a while
and don't pass the --ignition flag anymore.

BUG=v8:5238

Review-Url: https://codereview.chromium.org/2197123002
Cr-Commit-Position: refs/heads/master@{#38206}
2016-08-01 12:11:43 +00:00
mstarzinger
6e38f4270a [test] Remove NaCl support from test runner harness.
R=machenbach@chromium.org

Review-Url: https://codereview.chromium.org/2141013002
Cr-Commit-Position: refs/heads/master@{#37678}
2016-07-12 13:18:49 +00:00
machenbach
4ffdce65dd [Interpreter] Prepare switching to new ignition testing variant.
This adds a new ignition_turbofan testing variant that'll
allow switching turbofan off in the other ignition variant.

The switch will happen in a follow up after reconfiguring
the bots.

BUG=v8:4280
LOG=N

Review URL: https://codereview.chromium.org/1922623003

Cr-Commit-Position: refs/heads/master@{#35790}
2016-04-26 11:24:34 +00:00
jyan
3b5e84ab0e S390: Initial test changes and files checkin.
R=danno@chromium.org,jkummerow@chromium.org,jochen@chromium.org,joransiu@ca.ibm.com,michael_dawson@ca.ibm.com,mbrandy@us.ibm.com
BUG=

Review URL: https://codereview.chromium.org/1805533002

Cr-Commit-Position: refs/heads/master@{#34795}
2016-03-15 18:09:36 +00:00
yangguo
a65edb8be6 Improve test-serialize test cases.
Changes include:
 - better test coverage for builds with snapshot
 - write snapshot blobs to buffer instead of test serialization files
 - renamed tests

R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/1777213002

Cr-Commit-Position: refs/heads/master@{#34657}
2016-03-10 09:58:26 +00:00
machenbach
33ffced5cc [coverage] Enable sanitizer coverage.
This adds sanitizer-coverage compilation, test-runner
features and post processing.

Sanitizer coverage is expected to be used together with
asan.

During test runner execution, the produced sancov files
are disambiguated and match the pattern:
<executable name>.test.<test id>.sancov.

Two additional scripts are added for merging raw sancov
files and for generating json data containing all
instrumented lines + all covered lines from merged sancov
files. Both scripts use multiprocessing for speed.

The json data will later be uploaded to google storage
for further use, e.g. to show coverage data in rietveld.

Sancov documentation:
http://clang.llvm.org/docs/SanitizerCoverage.html

BUG=chromium:568949
LOG=n
NOTRY=true
TEST=python -m unittest sancov_formatter_test
TEST=python -m unittest sancov_merger_test

Review URL: https://codereview.chromium.org/1737263003

Cr-Commit-Position: refs/heads/master@{#34578}
2016-03-08 10:48:35 +00:00
machenbach
504796e916 [Swarming] Isolate deopt fuzzer.
This also ports a few changes from run-test.py to make sure
cwd = v8 root directory.

BUG=chromium:535160
LOG=n

Review URL: https://codereview.chromium.org/1707963003

Cr-Commit-Position: refs/heads/master@{#34110}
2016-02-18 10:48:21 +00:00
machenbach
a8e4eecf61 [test] Skip some tests on the coverage bot.
BUG=chromium:568949
LOG=n
NOTRY=true

Review URL: https://codereview.chromium.org/1520163003

Cr-Commit-Position: refs/heads/master@{#32840}
2015-12-14 14:38:19 +00:00
machenbach
780077f6d1 [test-runner] Move test case processing beyond the multi-process boundary.
This will allow moving the test outcome check beyond the
multi-process boundary in a follow up. It'll allow wrapping
more complex test jobs like predicable mode on the multi-
process side, which will make the code easier to maintain.

BUG=

Review URL: https://codereview.chromium.org/1469833002

Cr-Commit-Position: refs/heads/master@{#32373}
2015-11-27 12:52:25 +00:00
machenbach
f39d1cd4aa [test] Switch off perf data feature on swarming.
The feature sometimes prevents subsequent swarming bots to
delete the work directory on windows.

The data file is not kept on swarming bots anyways, therefore
this switches off the feature completely.

BUG=chromium:535160
LOG=n

Review URL: https://codereview.chromium.org/1468933002

Cr-Commit-Position: refs/heads/master@{#32168}
2015-11-23 09:11:17 +00:00
rmcilroy
fb9ad89fea [Interpreter]: Add ignition blacklist to mjsunit.status and test262.status.
Adds a blacklist of tests which are currently unsupported or broken in Ignition to
the mjsunit and test262 test status.

Also removes --ignition-script-filter flag, and adds a
--ignition_fallback_on_eval_and_catch flag which fallsback to fullcodegen for
functions which call eval or contain a catch block.

BUG=v8:4280
LOG=N

Review URL: https://codereview.chromium.org/1420963009

Cr-Commit-Position: refs/heads/master@{#31864}
2015-11-06 18:52:51 +00:00
ishell
d7b78abc2e Fix for deopt fuzzer which was broken by https://codereview.chromium.org/1352803002
Review URL: https://codereview.chromium.org/1347073004

Cr-Commit-Position: refs/heads/master@{#30802}
2015-09-17 14:05:43 +00:00
machenbach
5defb720bd [test] Return variant and random seed on failures.
BUG=chromium:511215
LOG=n

Review URL: https://codereview.chromium.org/1276853002

Cr-Commit-Position: refs/heads/master@{#30057}
2015-08-07 08:36:16 +00:00
jochen
22b691ba0e [test] Teach test runner about whether novfp3 is on or off
BUG=none
R=machenbach@chromium.org
LOG=n

Review URL: https://codereview.chromium.org/1204643003

Cr-Commit-Position: refs/heads/master@{#29221}
2015-06-23 09:43:21 +00:00
michael_dawson
f1ba8d8f86 Contribution of PowerPC port (continuation of 422063005) - AIX Common1
Contribution of PowerPC port (continuation of 422063005 and 817143002). This patch covers
the key changes needed to the common files needed to support AIX. Subsequent
patches will cover:
- changes to update the ppc directories so they are current with the changes
in the rest of the project.
- remaining AIX changes not resolved by 4.8 compiler
- individual optimizations for PPC

This is based off of the GitHub repository
https://github.com/andrewlow/v8ppc

R=danno@chromium.org, svenpanne@chromium.org
BUG=

Review URL: https://codereview.chromium.org/866843003

Cr-Commit-Position: refs/heads/master@{#26343}
2015-01-30 08:02:08 +00:00
machenbach
b925fc56de Add no-test-harness mode to test driver.
BUG=

Review URL: https://codereview.chromium.org/855993003

Cr-Commit-Position: refs/heads/master@{#26150}
2015-01-20 10:17:33 +00:00
Sven Panne
e4c5b84652 Contribution of PowerPC port (continuation of 422063005)
Contribution of PowerPC port (continuation of 422063005). The inital patch
covers the core changes to the common files.  Subsequent patches will cover
changes to common files to support AIX and to update the ppc directories so
they are current with the changes in the rest of the project.

This is based off of the GitHub repository
https://github.com/andrewlow/v8ppc

BUG=
R=svenpanne@chromium.org, danno@chromium.org, sevnpanne@chromium.org

Review URL: https://codereview.chromium.org/817143002

Cr-Commit-Position: refs/heads/master@{#26091}
2015-01-16 07:42:15 +00:00
jochen
05a6d87ee4 Teach the test runner whether it's running on a trybot or not
BUG=none
R=machenbach@chromium.org
LOG=n

Review URL: https://codereview.chromium.org/781763002

Cr-Commit-Position: refs/heads/master@{#25675}
2014-12-05 09:09:21 +00:00
machenbach@chromium.org
e48e3e53ae Add an --msan flag to run-tests.py.
Currently a no-op and added only for consistency with ASan/TSan.

BUG=chromium:425187
R=machenbach@chromium.org
LOG=N

Review URL: https://codereview.chromium.org/667903004

Patch from Sergey Matveev <earthdok@chromium.org>.

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@24805 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-10-22 12:54:35 +00:00
machenbach@chromium.org
f5a4b26061 Add tsan support to v8's gyp and test driver.
Tests only the default variant for now.

BUG=
R=jochen@chromium.org

Review URL: https://codereview.chromium.org/412853002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22561 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-23 13:14:26 +00:00
machenbach@chromium.org
e3e03d80ef Add predictable mode to test driver.
BUG=391747
LOG=n
R=ishell@chromium.org

Review URL: https://codereview.chromium.org/371363003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22285 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-08 15:59:23 +00:00
machenbach@chromium.org
ae017be69c Let test runner rerun failures to test for flakes.
When enabled, this dynamically adds jobs that had failures back to the pool. Special json output for flakes will be handled in a separate CL.

BUG=374134
LOG=n
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/360113003

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@22143 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-07-02 08:15:44 +00:00
machenbach@chromium.org
cbd242f707 Fix deopt fuzzer after adding a no_sorting flag.
BUG=
TBR=mstarzinger@chromium.org

Review URL: https://codereview.chromium.org/305643007

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21568 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-28 13:00:47 +00:00
machenbach@chromium.org
1d0db84277 Fix deopt fuzzer after test runner changes.
Use the same keyboard interrupt logic as run-tests.
Close the perf database explicitly after one run, as it has two runs on the same architecture. Currently the first run closes on garbage collection, which might corrupt the second run.

BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/284203002

git-svn-id: https://v8.googlecode.com/svn/branches/bleeding_edge@21330 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-05-15 12:40:40 +00:00
bmeurer@chromium.org
546d0a0b70 Add random_seed parameter to run-deopt-fuzzer.
R=machenbach@chromium.org

Review URL: https://codereview.chromium.org/226843003

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20647 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-10 10:43:13 +00:00
machenbach@chromium.org
a430449c68 Add the ability to disable tests in nosnap mode.
BUG=v8:3216
LOG=n
R=dcarney@chromium.org

Review URL: https://codereview.chromium.org/230743002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@20617 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-04-09 12:57:43 +00:00
dcarney@chromium.org
499b31e222 Crash like chrome. Currently, most chrome crashes do not yield useful stack traces as v8 does a silent abort and chrome's crash symbolization does not kick in.
R=svenpanne@chromium.org

BUG=

Review URL: https://codereview.chromium.org/179793004

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@19544 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-02-25 13:53:06 +00:00
machenbach@chromium.org
5be6daa942 Fix deopt fuzzer variable set up.
Adds the simulator variable configuration from r18959 to the deopt fuzzer.

BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/143403004

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18975 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-01-31 09:10:56 +00:00
machenbach@chromium.org
5d8e2f3435 Make 'ASAN' configurable in test status file.
BUG=
R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/149173006

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18922 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-01-29 15:30:41 +00:00
machenbach@chromium.org
1864f7388e Add infrastructure for skipping tests in GC stress mode.
Also move the GC stress configuration from the buildbot to the test runner.

BUG=
R=jkummerow@chromium.org, mvstanton@chromium.org

Review URL: https://codereview.chromium.org/141653008

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@18708 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2014-01-21 12:41:25 +00:00
jochen@chromium.org
a3cef1a9db Skip tests failing on the no-i18n bot
R=machenbach@chromium.org, mstarzinger@chromium.org, jkummerow@chromium.org

Review URL: https://codereview.chromium.org/23708055

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16867 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-20 13:13:57 +00:00
machenbach@chromium.org
1a3f3b0569 Skip intl tests from test262 when intl support is disabled.
Skips the whole intl directory if intl support is disabled through the test driver.

One test outside the intl directory is skipped separately. It will be handled in an extra CL.

R=jochen@chromium.org, mstarzinger@chromium.org

Review URL: https://codereview.chromium.org/23792008

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16854 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-09-20 09:26:25 +00:00
yangguo@chromium.org
be48c5ae26 Rename "parallel recompilation" to "concurrent recompilation".
Also introduced macros for flag aliases for temporary backwards compatibility.

R=hpayer@chromium.org
BUG=

Review URL: https://codereview.chromium.org/23014007

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@16280 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-08-22 16:14:37 +00:00
machenbach@chromium.org
2aab929806 Add webkit tests to deopt fuzzer portfolio.
Two failing tests are skipped and added to an existing bug report.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/20708002

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@15904 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-07-26 11:39:58 +00:00
machenbach@chromium.org
a0734ba3af Add deopt fuzzer tool.
Can be run as a stand-alone script like run-tests.

Executes first all tests of a given test suite to collect the maximum number of possible deopt points. Runs then a fuzzing phase with artificial deoptimizations triggered during testing.

Works for now with mjsunit and ia32 only.

R=jkummerow@chromium.org

Review URL: https://codereview.chromium.org/19931005

git-svn-id: http://v8.googlecode.com/svn/branches/bleeding_edge@15855 ce2b1a6d-e550-0410-aec6-3dcde31c8c00
2013-07-24 12:04:29 +00:00