skia2/tools/skpbench
Kevin Lubick 6ba9f702ba [bazel] Try adding cc_binary rules that use generated rules
To make the atomic rules a bit easier to work with, in many
of the folders, this adds in cc_library rules to group
together the sources from that folder (and subfolders
where prudent). We only needs sources because those atoms
should have their headers as deps.

One issue that was pointed out is that there is currently
no way to restrict the inclusion of certain packages,
a la, `gn check`. For example, there is no mechanism from
stopping a dev from adding
#include "modules/canvaskit/WasmCommon.h"
to something in //src/core (except circular dependencies).
We can probably address that using Bazel's visibility
rules as needed:
https://docs.bazel.build/versions/main/visibility.html
https://docs.bazel.build/versions/main/be/functions.html#package_group

It is recommended to look at this CL patchset by patchset.
PS1: Update gazelle command to generate rules in more folders.
PS2: A few changes to make generation work better.
PS3: The result of running make generate in //bazel
PS4: Adding the rules to build sksllex, the simplest binary I
     could find in the Skia repo.
PS5: Adding the rules to build skdiff, a more complex binary.
     I tried a few approaches, but ended up gravitating back
     towards the layout where we have each folder/package
     group up the sources. I imagine at some point, we'll have
     skdiff depend on skia_core or something, which will
     have things like //src/core, //src/codecs, //src/pathops
     all bundled together.
PS7: Added in the groupings of sources, similar to what we had
     earlier. I liked these for readability. These helped fix
     up the //:skia_core build, and by extension, the CanvasKit
     build.

Change-Id: I3faa7c4e821c876b243617aacf0246efa524cbde
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/476219
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-30 18:18:53 +00:00
..
__init__.py skpbench 2016-09-19 11:03:58 -07:00
_adb_path.py Assert there are some sources when running skpbench 2019-09-11 20:20:33 +00:00
_adb.py skpbench: invoke 'adb wait-for-device' lazily 2019-07-29 16:49:59 +00:00
_benchresult.py skpbench: add "resultsfile" option 2016-11-10 16:08:12 +00:00
_hardware_android.py Move GPU locking logic into generic hardware profile 2021-02-12 23:01:08 +00:00
_hardware_nexus_6p.py [skpbench] experiment with not disabling services 2018-03-01 19:28:44 +00:00
_hardware_pixel2.py skpbench: Add support for Pixel 2 2018-04-30 15:29:13 +00:00
_hardware_pixel_c.py [skpbench] Use android's perf settings for Pixel C 2018-03-19 14:22:55 +00:00
_hardware_pixel.py skpbench: Add support for Pixel 2 2018-04-30 15:29:13 +00:00
_hardware.py skpbench: support pixel phone 2017-10-20 17:52:36 +00:00
_os_path.py Add "AllPathsVolatile" skpbench bots 2020-09-24 19:50:34 +00:00
BUILD.bazel [bazel] Try adding cc_binary rules that use generated rules 2021-11-30 18:18:53 +00:00
README.md [sk] Update asset download references to use SK 2021-09-17 16:55:53 +00:00
sheet.py skpbench: add utility to format results for Skia Perf 2016-11-10 20:00:56 +00:00
skiaperf.py find min and max, separared by comma. not minmax. 2019-11-06 16:35:23 +00:00
skpbench.cpp Use CommonFlags namespace for other flags/functions 2021-10-14 14:34:21 +00:00
skpbench.py Delete mixed samples 2021-04-22 20:11:34 +00:00

skpbench

skpbench is a benchmarking tool for replaying skp or mksp files on android devices. it achieves a lower variance in framerate by controlling the clock speed and stopping all other processes that could cause interference.

Build

skpbench consists of the skpbench binary which must be built for the phone you intend to run on, and skpbench.py which runs on the machine the phone is connected to via ADB and is the entry point.

The to build skia for android are at https://skia.org/user/build#android and reproduced here.

Download the Android NDK

./bin/sk asset download android_ndk_linux /tmp/ndk

After this is set up once, build skpbench for your target cpu (assumed to be arm64 here for a Pixel 3)

bin/gn gen out/arm64 --args='ndk="/tmp/ndk" target_cpu="arm64" is_debug=false'
ninja -C out/arm64 skpbench

Benchmark an SKP on a connected device.

First, copy the built skpbench binary and an example skp file to the device. (or pull a skp corpus using instructions in the section below)

adb push out/arm64/skpbench /data/local/tmp
adb push /home/nifong/Downloads/foo.skp /data/local/tmp/skps/

Run skpbench.py

python tools/skpbench/skpbench.py \
  --adb \
  --config gles \
  /data/local/tmp/skpbench \
  /data/local/tmp/skps/foo.skp

--adb specifies that it should use adb to the only connected device and run skpbench there. --force is necessary because we don't yet have a configuration to monitor vitals on the Pixel 3. --config gles specifies Open GL ES is the backend GPU config to use.

Additional documentation of arguments is printed by python tools/skpbench/skpbench.py --help

Output appears in the following format

   accum    median       max       min   stddev  samples  sample_ms  clock  metric  config    bench
  0.1834    0.1832    0.1897    0.1707    1.59%      101         50  cpu    ms      gles      foo.skp

accum is the time taken to draw all frames, divided by the number of frames. metric specifies that the unit is ms (milliseconds per frame)

MSKP corpus

A manually collected corpus of MSKPs from around 30 top apps (using skia via HWUI) and of about 20 actions in RenderEngine exists in a google cloud storage folder managed by skia/infra/bots/assets/mskp/upload.py

To download the fileset, first determine the highest current version of the fileset

gsutil ls gs://skia-assets/assets/mskp/

Download the latest version.

gsutil cp gs://skia-assets/assets/mskp/5.zip ~/Downloads

Unzip the archive and adb push it to the device.

To upload a new version of the corpus, use the steps above to download and unzip the last version, change the content however you need, then Use the upload tool, passing the directory of the altered archive (not a zip file). Note that you must provide it as an absolute path.

python upload.py --target_dir=/home/nifong/scratch/new_mskps

The upload script should print a version number. Finally, submit something like https://skia-review.googlesource.com/c/skia/+/304376 to point jobs at the new version.

Production

skpbench is run as a tryjob from gerrit, where it uploads the results to perf.skia.org. Once such job name is Perf-Android-Clang-Pixel4XL-GPU-Adreno640-arm64-Release-All-Android_Skpbench

Perf results are available by querying with this or similar. extra_config = Android_Skpbench sub_result = accum_cpu_ms

Example perf query https://perf.skia.org/e/?queries=extra_config%3DAndroid_Skpbench%26sub_result%3Daccum_cpu_ms