v8/test/benchmarks/csuite
Vadim Gorbachev (bmsdave) 631213f6f6 Reland "Preparing v8 to use with python3 /test"
This is a reland of f8962ae1a2

Original change's description:
> Preparing v8 to use with python3 /test
>
> There are now less that 400 days until the end of life
> of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
> The code compatibility check for python2 and python3
> used the following tools: futurize, flake8
> You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
>
> This CL was uploaded by git cl split.
>
> Bug: v8:8594
> Change-Id: Idbf467daf629a4e808345a6a88036c2a3f259138
> Reviewed-on: https://chromium-review.googlesource.com/c/1470121
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59679}

Bug: v8:8594
Change-Id: I8c1a8d6593a4a927d56d37dada2c704062e842cc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1484300
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60354}
2019-03-20 09:56:06 +00:00
..
benchmark.py Reland "Preparing v8 to use with python3 /test" 2019-03-20 09:56:06 +00:00
compare-baseline.py Reland "Preparing v8 to use with python3 /test" 2019-03-20 09:56:06 +00:00
csuite.py Reland "Preparing v8 to use with python3 /test" 2019-03-20 09:56:06 +00:00
README.md [Test] CSuite benchmark runner 2018-12-21 13:15:57 +00:00
run-kraken.js [Test] CSuite benchmark runner 2018-12-21 13:15:57 +00:00
sunspider-standalone-driver.js [Test] CSuite benchmark runner 2018-12-21 13:15:57 +00:00

CSuite: Local benchmarking help for V8 performance analysis

CSuite helps you make N averaged runs of a benchmark, then compare with a different binary and/or different flags. It knows about the "classic" benchmarks of SunSpider, Kraken and Octane, which are still useful for investigating peak performance scenarios. It offers a default number of runs, by default they are:

  • SunSpider - 100 runs
  • Kraken - 80 runs
  • Octane - 10 runs

Usage

Say you want to see how much optimization buys you:

./csuite.py kraken baseline ~/src/v8/out/d8 -x="--noopt"
./csuite.py kraken compare ~/src/v8/out/d8

Suppose you are comparing two binaries, and want a quick look at results. Normally, Octane should have about 10 runs, but 3 will only take a few minutes:

./csuite.py -r 3 octane baseline ~/src/v8/out-master/d8
./csuite.py -r 3 octane compare ~/src/v8/out-mine/d8

You can run from any place:

../../somewhere-strange/csuite.py sunspider baseline ./d8
../../somewhere-strange/csuite.py sunspider compare ./d8-better

Note that all output files are created in the directory where you run from. A _benchmark_runner_data directory will be created to store run output, and a _results directory as well for scores.

For more detailed documentation, see:

./csuite.py --help

Output from the runners is captured into files and cached, so you can cancel and resume multi-hour benchmark runs with minimal loss of data/time. The -f flag forces re-running even if these cached files still exist.