The csuite.py script does not work correctly on Windows. It runs
correctly in baseline mode, but there are two problems when running in
compare mode:
1. In compare mode the output of benchmark.py is piped to the
compare-baseline.py script, but Windows only execute python files if
python.exe is the default program to open '.py' files, and this is
not the case, by default, when python is installed as part of the
depot_tools.
Fix: explicitly add the 'python' command before compare-baseline.py.
2. By default CSuite prints the results to stdout using escapes codes
that add color highlights. But this does not work on Windows when
compare-baseline.py is launched with a pipe:
python test/benchmarks/csuite/benchmark.py <...> |
python test/benchmarks/csuite/compare-baseline.py <baseline_results>
Fix: Do not use a pipe. Write the benchmark numbers for the
compare-run into a separate file, and pass the path to this file to
compare-baseline.py
Change-Id: Ic22d5bd4b47901f0ba0f35bc2496441346d21c6a
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2656855
Reviewed-by: Michael Stanton <mvstanton@chromium.org>
Commit-Queue: Paolo Severini <paolosev@microsoft.com>
Cr-Commit-Position: refs/heads/master@{#72807}
csuite.py compare command currently throws exception
"_csv.Error: iterator should return strings, not bytes
(did you open the file in text mode?)"
This should fix it.
Change-Id: I69c0ec43575a8c3627dac81dc99e47ba6adf6f61
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2316833
Reviewed-by: Michael Stanton <mvstanton@chromium.org>
Commit-Queue: Z Nguyen-Huu <duongn@microsoft.com>
Cr-Commit-Position: refs/heads/master@{#69050}
Before: running different suites is removing non-related master data
After: different suites have separate master data files
Change-Id: Iab46d663fdf2a1d709aeb7da77a443c2ed1bdcc2
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2071870
Reviewed-by: Michael Stanton <mvstanton@chromium.org>
Commit-Queue: Marcin Wiącek <marcin@mwiacek.com>
Cr-Commit-Position: refs/heads/master@{#66453}
This is a reland of f8962ae1a2
Original change's description:
> Preparing v8 to use with python3 /test
>
> There are now less that 400 days until the end of life
> of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
> The code compatibility check for python2 and python3
> used the following tools: futurize, flake8
> You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
>
> This CL was uploaded by git cl split.
>
> Bug: v8:8594
> Change-Id: Idbf467daf629a4e808345a6a88036c2a3f259138
> Reviewed-on: https://chromium-review.googlesource.com/c/1470121
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59679}
Bug: v8:8594
Change-Id: I8c1a8d6593a4a927d56d37dada2c704062e842cc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/1484300
Reviewed-by: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Cr-Commit-Position: refs/heads/master@{#60354}
This reverts commit f8962ae1a2.
Reason for revert: breaks Arm bots, e.g. https://ci.chromium.org/p/v8/builders/luci.v8.ci/V8%20Arm/9655, task: https://chromium-swarm.appspot.com/task?d=true&id=431dfa503db16d10
Original change's description:
> Preparing v8 to use with python3 /test
>
> There are now less that 400 days until the end of life
> of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
> The code compatibility check for python2 and python3
> used the following tools: futurize, flake8
> You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
>
> This CL was uploaded by git cl split.
>
> Bug: v8:8594
> Change-Id: Idbf467daf629a4e808345a6a88036c2a3f259138
> Reviewed-on: https://chromium-review.googlesource.com/c/1470121
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#59679}
TBR=machenbach@chromium.org,sergiyb@chromium.org,herhut@chromium.org,bmsdave@gmail.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: v8:8594
Change-Id: I17a0a7b203fa2c0ab0f965240ee1415b7513e1cf
Reviewed-on: https://chromium-review.googlesource.com/c/1478692
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59725}
There are now less that 400 days until the end of life
of Python 2(aka _legacy_ Python) https://pythonclock.org/ .
The code compatibility check for python2 and python3
used the following tools: futurize, flake8
You can see the reports here: https://travis-ci.com/bmsdave/v8/builds
This CL was uploaded by git cl split.
Bug: v8:8594
Change-Id: Idbf467daf629a4e808345a6a88036c2a3f259138
Reviewed-on: https://chromium-review.googlesource.com/c/1470121
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Belozorov <sergiyb@chromium.org>
Cr-Commit-Position: refs/heads/master@{#59679}
It's nice to have a quick way to get stable results on the local
machine for the classic d8 benchmarks. CSuite revitalizes an
old tool called BSuite which offers multiple averaged runs,
and a nice ascii display of results.
Sample usage:
./csuite.py octane baseline ~/src/v8/out/d8
./csuite.py octane compare ~/src/v8/out-mine/d8
benchmark: score | master | % |
===================================================+==========+========+
Richards: 26201.0 | 26568.0 | -1.4 |
DeltaBlue: 55744.0 | 58309.0 | -4.4 |
Crypto: 31831.0 | 31815.0 | |
RayTrace: 73481.0 | 73185.0 | 0.4 |
............<snip>............................
Octane: 36388.0 | 35741.0 | 1.8 |
---------------------------------------------------+----------+--------+
See README.md for more documentation.
Change-Id: I182490506ca07fab1240bd485bd1d7a8920db893
Reviewed-on: https://chromium-review.googlesource.com/c/1387487
Commit-Queue: Michael Stanton <mvstanton@chromium.org>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Reviewed-by: Jakob Kummerow <jkummerow@chromium.org>
Cr-Commit-Position: refs/heads/master@{#58434}
