AuroraMiddleware/skia2
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
76b80eca64
skia2/tools/perf-canvaskit-puppeteer/Makefile

43 lines
1.9 KiB
Makefile
Raw Normal View History

[canvaskit] Start a generic puppeteer perfing system. IMPORTANT LESSON: when bringing in node (and possibly other executables) via CIPD, add them to the path in gen_tasks_logic so the parent executable (the task driver itself) has the right PATH set. Otherwise, the subprocesses it spawns might grab the wrong version because of how golang handles environments of subprocesses. This is starting as a fork of Skottie WASM. I hope to have a more unified system for creating and running benchmarks. Overall overview: gen_tasks_logic.go creates a task in task.json that compiles CanvasKit and the task drivers and then executes our task (i.e. perf_puppeteer.go) perf_puppeteer runs a node program (perf-with-puppeteer.js) that uses puppeteer to execute benchmarking code on an html page (canvaskit-skottie-frames-load.html). I needed to update the node package so npm could be updated from 3.x to 6.14.4 so it knew about `npm ci`. This may not have been entirely necessary, given the problems of executing the correct npm (see important lesson above), but it hasn't broken things further, so more up-to-date is probably a good thing. Suggested Review Order: - canvaskit-skottie-frames-load.html (note it is similar to skottie-wasm-perf.html, but it waits for a button click to start animating and records times from the main JS thread itself) - perf-with-puppeteer.js (similar to skottie-wasm-perf.js, but has some things made optional [e.g. tracing]) - perf_puppeteer_test.go (shows the inputs/outputs of various steps) - perf_puppeteer.go - Everything else. Change-Id: I380e81b825f36682c257664d488267edaf36369e Reviewed-on: https://skia-review.googlesource.com/c/skia/+/285783 Commit-Queue: Kevin Lubick <kjlubick@google.com> Reviewed-by: Eric Boren <borenet@google.com>
2020-05-01 18:16:27 +00:00
# This shows an example invocation presuming the user has built canvaskit locally.
[canvaskit] Revert raster change. Drawing to a Premul Raster Surface and then copying out Unpremul pixels was 15x slower. Drawing to an Unpremul Raster surface was a little slower than the previous RasterDirect surface. Out of an abundance of caution, I'm going back to the way things were. We can revisit drawing to a Raster surface in the future. This also adds better error handling to the SkottieFrames test which thankfully caught this regression. Bug: skia:10717 Change-Id: Ib8b9d490c92e013c3311adb1d08de5fa0fc0351c Reviewed-on: https://skia-review.googlesource.com/c/skia/+/334356 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-11-12 14:11:47 +00:00
frames_lego:
[canvaskit] Start a generic puppeteer perfing system. IMPORTANT LESSON: when bringing in node (and possibly other executables) via CIPD, add them to the path in gen_tasks_logic so the parent executable (the task driver itself) has the right PATH set. Otherwise, the subprocesses it spawns might grab the wrong version because of how golang handles environments of subprocesses. This is starting as a fork of Skottie WASM. I hope to have a more unified system for creating and running benchmarks. Overall overview: gen_tasks_logic.go creates a task in task.json that compiles CanvasKit and the task drivers and then executes our task (i.e. perf_puppeteer.go) perf_puppeteer runs a node program (perf-with-puppeteer.js) that uses puppeteer to execute benchmarking code on an html page (canvaskit-skottie-frames-load.html). I needed to update the node package so npm could be updated from 3.x to 6.14.4 so it knew about `npm ci`. This may not have been entirely necessary, given the problems of executing the correct npm (see important lesson above), but it hasn't broken things further, so more up-to-date is probably a good thing. Suggested Review Order: - canvaskit-skottie-frames-load.html (note it is similar to skottie-wasm-perf.html, but it waits for a button click to start animating and records times from the main JS thread itself) - perf-with-puppeteer.js (similar to skottie-wasm-perf.js, but has some things made optional [e.g. tracing]) - perf_puppeteer_test.go (shows the inputs/outputs of various steps) - perf_puppeteer.go - Everything else. Change-Id: I380e81b825f36682c257664d488267edaf36369e Reviewed-on: https://skia-review.googlesource.com/c/skia/+/285783 Commit-Queue: Kevin Lubick <kjlubick@google.com> Reviewed-by: Eric Boren <borenet@google.com>
2020-05-01 18:16:27 +00:00
node perf-canvaskit-with-puppeteer.js --canvaskit_js ../../out/canvaskit_wasm/canvaskit.js \
--canvaskit_wasm ../../out/canvaskit_wasm/canvaskit.wasm --use_gpu \
--input_lottie ../../modules/canvaskit/perf/assets/lego_loader.json \
--bench_html skottie-frames.html
Add puppeteer perf for performance degredation related to transforming a complex path. Translations and rotations both have an effect on cache usage. Snapping translations to integer coordinations reduces cache usage. Opacity of path painting does not have an effect on cache usage. Bug: skia:10272 Change-Id: Id5d5f08cb43645c9ec44b9d8e5e96643041727c3 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/292280 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-05-29 20:13:51 +00:00
[canvaskit] Revert raster change. Drawing to a Premul Raster Surface and then copying out Unpremul pixels was 15x slower. Drawing to an Unpremul Raster surface was a little slower than the previous RasterDirect surface. Out of an abundance of caution, I'm going back to the way things were. We can revisit drawing to a Raster surface in the future. This also adds better error handling to the SkottieFrames test which thankfully caught this regression. Bug: skia:10717 Change-Id: Ib8b9d490c92e013c3311adb1d08de5fa0fc0351c Reviewed-on: https://skia-review.googlesource.com/c/skia/+/334356 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-11-12 14:11:47 +00:00
frames_cpu_lego:
node perf-canvaskit-with-puppeteer.js --canvaskit_js ../../out/canvaskit_wasm/canvaskit.js \
--canvaskit_wasm ../../out/canvaskit_wasm/canvaskit.wasm \
--input_lottie ../../modules/canvaskit/perf/assets/lego_loader.json \
--bench_html skottie-frames.html
Add puppeteer perf for performance degredation related to transforming a complex path. Translations and rotations both have an effect on cache usage. Snapping translations to integer coordinations reduces cache usage. Opacity of path painting does not have an effect on cache usage. Bug: skia:10272 Change-Id: Id5d5f08cb43645c9ec44b9d8e5e96643041727c3 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/292280 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-05-29 20:13:51 +00:00
test_path_transform_without_snap:
node perf-canvaskit-with-puppeteer.js --canvaskit_js --canvaskit_js ../../out/canvaskit_wasm/canvaskit.js \
--canvaskit_wasm ../../out/canvaskit_wasm/canvaskit.wasm --use_gpu \
--assets path_translate_assets \
--bench_html path-transform.html \
--query_params translate opacity
test_path_transform_with_snap:
node perf-canvaskit-with-puppeteer.js --canvaskit_js --canvaskit_js ../../out/canvaskit_wasm/canvaskit.js \
--canvaskit_wasm ../../out/canvaskit_wasm/canvaskit.wasm --use_gpu \
--assets path_translate_assets \
--bench_html path-transform.html \
[canvaskit] Add benchmarks on SKPs For each skp in the corpus, we start a fresh instance of Chromium (via puppeteer), draw the skp and measure that time. This process is repeated a fixed amount of repetitions and the median, the average, and the std deviation is reported to perf (as well as the individual datapoints as an FYI). Importantly (and something we'll need to change about SkottieFrames), we measure the average time between frames after unlocking the framerate. This ensures we account for the time needed by the GPU to actually draw (flush() returns after the GPU has all the instructions, but not necessarily has been able to draw). This implementation is very similar to the SkottieFrames code; a notable deviation is the repetitions are handled outside of the html, i.e. a new chrome window per run. I explored using content_shell, but noticed that requires building Chromium, which our infrastructure is not set up to do well. Change-Id: I14fdbdc951604d3fdf06e81a4be7e614d0e53c03 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/295079 Commit-Queue: Kevin Lubick <kjlubick@google.com> Reviewed-by: Nathaniel Nifong <nifong@google.com> Reviewed-by: Eric Boren <borenet@google.com>
2020-06-10 19:10:16 +00:00
--query_params translate opacity snap
skp_with_local:
node perf-canvaskit-with-puppeteer.js --canvaskit_js ../../out/canvaskit_wasm/canvaskit.js \
--canvaskit_wasm ../../out/canvaskit_wasm/canvaskit.wasm --use_gpu \
--input_skp ${HOME}/skps/desk_nytimes.skp \
Add Puppeteer perf for release and experimental_simd builds of CanvasKit against SKPs Adds the command: make skps_release_and_SIMD for perfing builds against a set of SKPs in ~/skps for release and simd builds of CanvasKit. Also outputs a summary of the perf results in a table format. See the document "SIMD CanvasKit Build Performance Testing" for more details: https://docs.google.com/document/d/114kdSGPMnOSQCZ7pFgd3MGMn5mIW562RMoXVmD13e0M/edit# Bug: skia:10453 Change-Id: I311629a1420301dda41f7ec57ce1403b05fd949b Reviewed-on: https://skia-review.googlesource.com/c/skia/+/301982 Reviewed-by: Elliot Evans <elliotevans@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-07-16 17:25:01 +00:00
--bench_html render-skp.html
A puppeteer test that runs karma-like tests Change-Id: Iedd98c03ba9d56b64390ffea23a50c945be9aff3 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/305557 Commit-Queue: Nathaniel Nifong <nifong@google.com> Reviewed-by: Elliot Evans <elliotevans@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-07-29 15:01:55 +00:00
# runs all the tests in canvas_perf.js
perf_js:
node perf-canvaskit-with-puppeteer.js --canvaskit_js ../../out/canvaskit_wasm/canvaskit.js \
--canvaskit_wasm ../../out/canvaskit_wasm/canvaskit.wasm --use_gpu \
allow external files in canvas_perf.js tests Change-Id: Id4d143605bfa73672130c839cf54eec2b23217d7 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/308233 Commit-Queue: Nathaniel Nifong <nifong@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-08-06 19:27:47 +00:00
--assets canvas_perf_assets \
Add more perf tests and other minor improvemnts Change-Id: I2da6d2da2cb0dd938d04d0b872be7fc5f2a91d6d Docs-Preview: https://skia.org/?cl=308787 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/308787 Commit-Queue: Nathaniel Nifong <nifong@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-08-10 14:49:02 +00:00
--bench_html canvas_perf.html \
--timeout 600 # seconds
A puppeteer test that runs karma-like tests Change-Id: Iedd98c03ba9d56b64390ffea23a50c945be9aff3 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/305557 Commit-Queue: Nathaniel Nifong <nifong@google.com> Reviewed-by: Elliot Evans <elliotevans@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-07-29 15:01:55 +00:00
Add Puppeteer perf for release and experimental_simd builds of CanvasKit against SKPs Adds the command: make skps_release_and_SIMD for perfing builds against a set of SKPs in ~/skps for release and simd builds of CanvasKit. Also outputs a summary of the perf results in a table format. See the document "SIMD CanvasKit Build Performance Testing" for more details: https://docs.google.com/document/d/114kdSGPMnOSQCZ7pFgd3MGMn5mIW562RMoXVmD13e0M/edit# Bug: skia:10453 Change-Id: I311629a1420301dda41f7ec57ce1403b05fd949b Reviewed-on: https://skia-review.googlesource.com/c/skia/+/301982 Reviewed-by: Elliot Evans <elliotevans@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-07-16 17:25:01 +00:00
skps_release_and_simd:
./perf_all_skps.sh --release --simd --summary
Reference in New Issue Copy Permalink