8fd0ccfc9f
The make_lotties_with_assets is a handy tool to take a directory of lottie files and move them into their own folders. This will make it easier to add assets (e.g. fonts, images) if we want to get more accurate benchmarks. This also makes the test not stop on the first failure, but complete all of them, so we can see more easily if multiple things have broken. Change-Id: Idb2ece71376a11e7524cd513a2798884a44bb59f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/421927 Reviewed-by: Florin Malita <fmalita@google.com> |
||
|---|---|---|
| .. | ||
| canvas_perf_assets | ||
| path_translate_assets | ||
| benchmark.js | ||
| canvas_perf.html | ||
| canvas_perf.js | ||
| Makefile | ||
| package-lock.json | ||
| package.json | ||
| path-transform.html | ||
| perf_all_skps.sh | ||
| perf-canvaskit-with-puppeteer.js | ||
| README.md | ||
| render-skp.html | ||
| skottie-frames.html | ||
| skp_data_prep.js | ||
Measuring the performance of CanvasKit using Puppeteer and Chrome.
Initial setup
Run npm ci to install the dependencies need to run the tests. In //modules/canvaskit, run
make release to build the canvaskit that will be used. With modifications to the Makefile,
other builds (e.g. make profile) can be used as well.
If needed, one can download the lottie-samples and/or skp assets from CIPD using the sk tool:
sk asset download lottie-samples ~/Downloads/lottie-samples
sk asset download skps ~/Downloads/skps
The actual location that these assets can be downloaded to is not important - the Makefile assumes them to be in Downloads, but that can be modified by the local user.
Basic Performance Tests
We have a harness for running benchmarks. Benchmark code snippets can be added to canvas_perf.js.
The harness itself is the canvas_perf.html and benchmark.js. It will run the "test" portion of
the code on multiple frames and gather data.
To run the benchmarks, run make perf_js. By default, this will use the most recent release build
of canvaskit done locally. If you want to only run one or a few, modify the
canvas_perf.js file by changing the relevent tests.push to onlytests.push and then run
make perf_js.
On the CI, the results from these tests are uploaded to Perf. For example: https://perf.skia.org/e/?queries=test%3Dcanvas_drawOval We include metrics such as the 90th, 95th, and 99th percentile frame, average frame time, median frame time, and standard deviation. There are three types of measurements: without_flush_ms is the measurement of the test() function; with_flush_ms is the measurement of test() and the subsequent flush() call; total_frame_ms is the frame-to-frame time. Frame-to-frame is important to measure because it accounts for any work the GPU needs to do, even after CanvasKit flushes.
Skottie Frames Performance
There is a harness that gathers data about rendering 600 frames of a skottie animation, cycling through it in a similar fashion to how it would be displayed to a user (e.g. as it is on skottie.skia.org).
To test it locally with a specific skottie animation, feel free to modify the Makefile to adjust the
input_lottie argument and then run make frames. The harness itself is skottie-frames.html and
benchmark.js.
On the CI, the results from these tests are uploaded to Perf. For example: https://perf.skia.org/e/?queries=test%3Dlego_loader We include metrics such as the first 5 frame times, average frame times, 90th, 95th and 99th percentile frame time.
SKP Performance
There is a harness that repeatedly will draw an SKP and measure various metrics. This is handled
by skottie-frames.html and benchmark.js. As before, feel free to modify the Makefile (the
input_skp argument) and run make skp.
On the CI, the results from these tests are uploaded to Perf. For example: https://perf.skia.org/e/?queries=binary%3DCanvasKit%26test%3Ddesk_chalkboard.skp