AuroraMiddleware/skia2
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
79426194de
skia2/infra/bots/recipes/perf_canvaskit.py

99 lines
3.2 KiB
Python
Raw Normal View History

[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
# Copyright 2018 The Chromium Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
# Recipe which runs the Canvaskit tests using docker
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
DEPS = [
'checkout',
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
'docker',
Update android-skqp Docker image to accept licenses Set DOCKER_CONFIG env var for all docker commands so that the Swarming task uses the config created here: https://skia.googlesource.com/buildbot/+/06aaf21b5d91a68de7c02aafc0fdee8531ba0c21/scripts/run_on_swarming_bots/install_docker.py#65 Bug: skia:8698 Change-Id: I731a99abc274caa498af8dde5142a7e6a6b6eca3 Reviewed-on: https://skia-review.googlesource.com/c/185690 Commit-Queue: Ben Wagner <benjaminwagner@google.com> Reviewed-by: Stephan Altmueller <stephana@google.com>
2019-01-22 23:09:30 +00:00
'env',
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
'infra',
'recipe_engine/file',
'recipe_engine/path',
'recipe_engine/properties',
'recipe_engine/python',
'recipe_engine/step',
'run',
'vars',
]
[canvaskit] Update Chrome version and use npm ci for tests By using npm ci, we can make sure the versions of the helper libraries (e.g. Karma, Jasmine) we are testing with locally is the same as the versions we are using in the continuous integration system. The copying is needed because our docker recipe forces us to run as not root, and this was causing some issues. As a result, I changed the canvaskit test/perf to not re-use the same file as pathkit does so copying was easier and the dependencies between the two modules is broken. Bug: skia:11077 Change-Id: Ib05890d666d3507d4f724a4ae298484629c7932a Reviewed-on: https://skia-review.googlesource.com/c/skia/+/343503 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2020-12-14 14:43:13 +00:00
DOCKER_IMAGE = 'gcr.io/skia-public/perf-karma-chrome-tests:87.0.4280.88_v1'
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
INNER_KARMA_SCRIPT = 'skia/infra/canvaskit/perf_canvaskit.sh'
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
def RunSteps(api):
api.vars.setup()
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
checkout_root = api.path['start_dir']
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
out_dir = api.vars.swarming_out_dir
# The karma script is configured to look in ./canvaskit/bin/ for
# the test files to load, so we must copy them there (see Set up for docker).
[canvaskit] Move from experimental to modules Bug: skia: Change-Id: I2ffd54cf81c974f3a80103e1726a06067cc90d82 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/200044 Reviewed-by: Mike Reed <reed@google.com> Reviewed-by: Florin Malita <fmalita@chromium.org> Commit-Queue: Kevin Lubick <kjlubick@google.com>
2019-03-11 20:11:58 +00:00
copy_dest = checkout_root.join('skia', 'modules', 'canvaskit',
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
'canvaskit', 'bin')
base_dir = api.vars.build_dir
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
copies = {
base_dir.join('canvaskit.js'): copy_dest.join('canvaskit.js'),
base_dir.join('canvaskit.wasm'): copy_dest.join('canvaskit.wasm'),
}
recursive_read = [checkout_root.join('skia')]
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
args = [
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
'--builder', api.vars.builder_name,
'--git_hash', api.properties['revision'],
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
'--buildbucket_build_id', api.properties.get('buildbucket_build_id', ''),
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
'--bot_id', api.vars.swarming_bot_id,
'--task_id', api.vars.swarming_task_id,
'--browser', 'Chrome',
'--config', api.vars.configuration,
'--source_type', 'canvaskit',
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
]
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
if api.vars.is_trybot:
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
args.extend([
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
'--issue', api.vars.issue,
'--patchset', api.vars.patchset,
])
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
api.docker.run(
[canvaskit] Try some things to reduce GPU test flakiness 1) some flags that "The Internet" says may help 2) retry running the test script up to 3 times. I wasn't able to reproduce the crashes with a non-Docker Chrome, only in the Docker container, which was very hard to debug. Change-Id: I87f31c32f63b2770d8d5afa6a8e4b90c35dbf0bb Reviewed-on: https://skia-review.googlesource.com/c/skia/+/255820 Reviewed-by: Eric Boren <borenet@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2019-11-21 17:49:16 +00:00
name='Performance tests of CanvasKit with Docker',
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
docker_image=DOCKER_IMAGE,
src_dir=checkout_root,
out_dir=out_dir,
[infra] Use CIPD packages from infra repo Includes recipe hack because Kitchen introduced a directory level: https://chromium-review.googlesource.com/c/infra/infra/+/1446288 Change-Id: I1afe43ea75964a2940e561f8d2bbfda3dd1f57e2 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/255256 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com> Reviewed-by: Ravi Mistry <rmistry@google.com>
2019-11-20 13:28:20 +00:00
script=checkout_root.join(INNER_KARMA_SCRIPT),
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
args=args,
docker_args=None,
copies=copies,
recursive_read=recursive_read,
[canvaskit] Try some things to reduce GPU test flakiness 1) some flags that "The Internet" says may help 2) retry running the test script up to 3 times. I wasn't able to reproduce the crashes with a non-Docker Chrome, only in the Docker container, which was very hard to debug. Change-Id: I87f31c32f63b2770d8d5afa6a8e4b90c35dbf0bb Reviewed-on: https://skia-review.googlesource.com/c/skia/+/255820 Reviewed-by: Eric Boren <borenet@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2019-11-21 17:49:16 +00:00
attempts=3,
[recipes] CanvasKit, PathKit, some others don't need bot_update Bug: skia:9588 Change-Id: I8dc915f9cedd40b94f37f3d1ae89bb7d4767814f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/252186 Commit-Queue: Eric Boren <borenet@google.com> Reviewed-by: Kevin Lubick <kjlubick@google.com>
2019-11-05 15:08:23 +00:00
)
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
def GenTests(api):
yield (
Changing ~everything (mostly recipe related stuff) to reference Debian10 jobs Bug: skia:9438 Change-Id: Ibea28d454bcc1c44dfb9ab82a2a8216342f3b8a7 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/281878 Reviewed-by: Eric Boren <borenet@google.com> Commit-Queue: Weston Tracey <westont@google.com>
2020-04-07 01:27:14 +00:00
api.test('Perf-Debian10-EMCC-GCE-CPU-AVX2-wasm-Release-All-CanvasKit') +
api.properties(buildername=('Perf-Debian10-EMCC-GCE-CPU-AVX2'
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
'-wasm-Release-All-CanvasKit'),
repository='https://skia.googlesource.com/skia.git',
revision='abc123',
path_config='kitchen',
swarm_out_dir='[SWARM_OUT_DIR]')
)
yield (
api.test('pathkit_trybot') +
Changing ~everything (mostly recipe related stuff) to reference Debian10 jobs Bug: skia:9438 Change-Id: Ibea28d454bcc1c44dfb9ab82a2a8216342f3b8a7 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/281878 Reviewed-by: Eric Boren <borenet@google.com> Commit-Queue: Weston Tracey <westont@google.com>
2020-04-07 01:27:14 +00:00
api.properties(buildername=('Perf-Debian10-EMCC-GCE-GPU-AVX2'
[canvaskit] Add perf jobs Of note, the perf results reported are not directly comparable CPU->GPU because the GPU ones are likely emulated (no real gpu in the Docker container on an GCE VM). Bug: skia: Change-Id: I52259085f4d9e22c45b67f3e8ce1211a5c6c6d3e Reviewed-on: https://skia-review.googlesource.com/c/163126 Reviewed-by: Kevin Lubick <kjlubick@google.com>
2018-10-17 18:59:35 +00:00
'-wasm-Release-All-CanvasKit'),
repository='https://skia.googlesource.com/skia.git',
revision='abc123',
path_config='kitchen',
swarm_out_dir='[SWARM_OUT_DIR]',
patch_ref='89/456789/12',
patch_repo='https://skia.googlesource.com/skia.git',
patch_storage='gerrit',
patch_set=7,
patch_issue=1234,
gerrit_project='skia',
gerrit_url='https://skia-review.googlesource.com/')
)
Reference in New Issue Copy Permalink