Commit Graph

28 Commits

Author SHA1 Message Date
Eric Boren
7bc4c9a8d1 [python3] Reland recipes -> python3 changes
This was enabled by moving the iPhones off of the old RPI2 hosts.

This reverts commit 04cd6fba97.
This reverts commit a726978ae7.

Bug: chromium:1256037
Change-Id: I35069089aa39baf62a18235c8d0514923f327c53
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/477987
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2021-12-01 13:04:03 +00:00
Erik Rose
4fb7dec706 [infra] Change the iOS arch to arm64 so we get the right CIPD packages for the rpi2 machines.
Change-Id: Id44d46f57aa27b2c7e5abc9949444b244c4b7ee7
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/477645
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Erik Rose <erikrose@google.com>
2021-11-30 12:59:55 +00:00
Eric Boren
a726978ae7 Revert "[python3] More Recipes -> Python 3 fixes"
This reverts commit 12e786730f.

Reason for revert: missing CIPD package on armv6l

Original change's description:
> [python3] More Recipes -> Python 3 fixes
>
> - Set environment variables to force usage of Python 3 in more places
> - Fix more compatibility issues
> - Mark recipes as only supporting Python 3
> - Includes a roll of the infra code
>
> Change-Id: I24e3827a6402c454bdc9467d28864d360632f9e6
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/470303
> Reviewed-by: Ravi Mistry <rmistry@google.com>
> Commit-Queue: Eric Boren <borenet@google.com>

Change-Id: If7b6fcb7838ac053af2c5eb45a7a1ac4aed340a5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/472736
Auto-Submit: Eric Boren <borenet@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2021-11-17 14:57:10 +00:00
Eric Boren
12e786730f [python3] More Recipes -> Python 3 fixes
- Set environment variables to force usage of Python 3 in more places
- Fix more compatibility issues
- Mark recipes as only supporting Python 3
- Includes a roll of the infra code

Change-Id: I24e3827a6402c454bdc9467d28864d360632f9e6
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/470303
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2021-11-17 13:51:08 +00:00
Kevin Lubick
e405d7c8e5 [infra] Migrate ChromeOS devices to arm64 RPIs
This is similar to https://skia-review.googlesource.com/c/skia/+/447177

Change-Id: I9189c6994ee02360115f5397d5008e2eb4c3e40e
Bug: skia:12401
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/451916
Reviewed-by: Eric Boren <borenet@google.com>
2021-09-23 15:59:59 +00:00
Joe Gregorio
db2b44e01f Move all Android machines to arm64.
Change-Id: I738fb0c58d1a27397c94df384c8c87c045e36cb6
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/447177
Reviewed-by: Eric Boren <borenet@google.com>
2021-09-09 16:14:47 +00:00
Joe Gregorio
2ee5cb516a [switchboard] Move P30's over to switchboard based RPis.
Bug: skia:12063
Change-Id: I86833ddfac9414348cbc6cf5bfb39e09c02c7d92
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/434857
Auto-Submit: Joe Gregorio <jcgregorio@google.com>
Commit-Queue: Joe Gregorio <jcgregorio@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2021-07-30 16:19:17 +00:00
Joe Gregorio
f1f542cde8 [switchboard] Switch S20s over to arm64 RPis.
Bug: skia:12063
Change-Id: I47257179d38de0452766ce70857eaf0a920b7abc
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/434156
Commit-Queue: Joe Gregorio <jcgregorio@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
Auto-Submit: Joe Gregorio <jcgregorio@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2021-07-28 18:36:40 +00:00
Eric Boren
8f553a769b [infra] Fix RecreateSKPs after assets update
Change-Id: I13ae7034cda4291743f433406ee2e8267dd48d3e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/416576
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
2021-07-19 14:13:09 +00:00
Eric Boren
e76e96adf8 Reland "[infra] Roll Infra dep to update CIPD packages" with fix
This reverts commit 8cb7c3be75.

Just leave Win7 Golo tasks on python 2.

Change-Id: Idf1d6cc6d303107afd8e74cdec94e7a9ec91399e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/416316
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2021-06-22 17:31:55 +00:00
Eric Boren
8cb7c3be75 Revert "[infra] Roll Infra dep to update CIPD packages"
This reverts commit c227bbea02.

Reason for revert: Breaking Win7 Golo

Original change's description:
> [infra] Roll Infra dep to update CIPD packages
>
> This is intended to pick up https://skia-review.googlesource.com/c/buildbot/+/414458 which updates the tasks to use cpython3, but it also updates the luci-auth, kitchen, and git packages.
>
> Also includes fixes needed after https://skia-review.googlesource.com/c/buildbot/+/387896
>
> Bug: chromium:1210114
> Change-Id: I2003be762b59a08a6b972b6563e4f3a9920b082d
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/414438
> Reviewed-by: Kevin Lubick <kjlubick@google.com>
> Reviewed-by: Joe Gregorio <jcgregorio@google.com>
> Commit-Queue: Eric Boren <borenet@google.com>

TBR=borenet@google.com,jcgregorio@google.com,kjlubick@google.com

Change-Id: Ie4cf5932ad163d2085ecd7c08b915cbeaea3f14b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:1210114
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/415899
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2021-06-04 19:39:19 +00:00
Eric Boren
c227bbea02 [infra] Roll Infra dep to update CIPD packages
This is intended to pick up https://skia-review.googlesource.com/c/buildbot/+/414458 which updates the tasks to use cpython3, but it also updates the luci-auth, kitchen, and git packages.

Also includes fixes needed after https://skia-review.googlesource.com/c/buildbot/+/387896

Bug: chromium:1210114
Change-Id: I2003be762b59a08a6b972b6563e4f3a9920b082d
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/414438
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2021-06-04 18:37:27 +00:00
Eric Boren
942db50454 [infra] Remove LenovoYoga jobs and associated code
These devices died in the flood a year ago.

Change-Id: Ibf2561016279b1c91eabffd3a5eb76995466b265
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/388745
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2021-03-25 12:58:45 +00:00
Eric Boren
1f95e6c6b4 Reland "[infra] Switch from isolate to RBE-CAS"
Bug: skia:10883
Change-Id: Ie702a4699cda48603ceff690d2ba946bf7a0d0b2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/344898
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Weston Tracey <westont@google.com>
2021-02-22 14:41:06 +00:00
Eric Boren
32c6099d92 Revert "Reland "[infra] Switch from isolate to RBE-CAS""
This reverts commit 540b51ce8f.

Reason for revert: RPI disk space

Original change's description:
> Reland "[infra] Switch from isolate to RBE-CAS"
>
> Bug: skia:10883
> Change-Id: Iec82a07fdf3c0807a9bb1870309eded85d4f0b1e
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/341679
> Commit-Queue: Eric Boren <borenet@google.com>
> Reviewed-by: Weston Tracey <westont@google.com>

TBR=borenet@google.com,westont@google.com

Change-Id: I0156842b995d275d8deb5bd73c6877de7f8af72e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: skia:10883
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/342596
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2020-12-09 18:09:39 +00:00
Eric Boren
540b51ce8f Reland "[infra] Switch from isolate to RBE-CAS"
Bug: skia:10883
Change-Id: Iec82a07fdf3c0807a9bb1870309eded85d4f0b1e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/341679
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Weston Tracey <westont@google.com>
2020-12-09 17:14:39 +00:00
Eric Boren
0a8ddfbf98 [infra] Roll infra DEPS, update CIPD packages
Change-Id: Ie20d7db4646851e8baaefbe134f2c273b97f7c54
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/342298
Reviewed-by: Weston Tracey <westont@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2020-12-09 15:56:38 +00:00
Eric Boren
4159ecbca9 Revert "[infra] Switch from isolate to RBE-CAS"
This reverts commit 03d69876ff.

Reason for revert: failing

Original change's description:
> [infra] Switch from isolate to RBE-CAS
>
> Bug: skia:10883
> Change-Id: I4bab202c89253248b7c93d23540038f83af97d33
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/340100
> Commit-Queue: Eric Boren <borenet@google.com>
> Reviewed-by: Weston Tracey <westont@google.com>

TBR=borenet@google.com,westont@google.com

Change-Id: Idf99f74579e44bb6b88dcb2c727d8215a3c70c71
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: skia:10883
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/341678
Reviewed-by: Eric Boren <borenet@google.com>
2020-12-08 14:29:10 +00:00
Eric Boren
03d69876ff [infra] Switch from isolate to RBE-CAS
Bug: skia:10883
Change-Id: I4bab202c89253248b7c93d23540038f83af97d33
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/340100
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Weston Tracey <westont@google.com>
2020-12-08 14:04:24 +00:00
Eric Boren
2ff24c64cd [infra] Fix Git package for Housekeeper-PerCommit-InfraTests_Win
Bug: skia:9458
Change-Id: Ib76b235c603af8261411d42fa4a9c72bf03ffba2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/315641
Reviewed-by: Weston Tracey <westont@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2020-09-08 17:09:10 +00:00
Kevin Lubick
5443bb32a2 [canvaskit] Start a generic puppeteer perfing system.
IMPORTANT LESSON: when bringing in node (and possibly other
executables) via CIPD, add them to the path in gen_tasks_logic
so the parent executable (the task driver itself) has the right
PATH set. Otherwise, the subprocesses it spawns might grab the
wrong version because of how golang handles environments of
subprocesses.

This is starting as a fork of Skottie WASM. I hope to have a more unified
system for creating and running benchmarks.

Overall overview:
gen_tasks_logic.go creates a task in task.json that compiles
CanvasKit and the task drivers and then executes our task
(i.e. perf_puppeteer.go)

perf_puppeteer runs a node program (perf-with-puppeteer.js)
that uses puppeteer to execute benchmarking code on an
html page (canvaskit-skottie-frames-load.html).

I needed to update the node package so npm could be updated from
3.x to 6.14.4 so it knew about `npm ci`. This may not have been
entirely necessary, given the problems of executing the correct
npm (see important lesson above), but it hasn't broken things
further, so more up-to-date is probably a good thing.

Suggested Review Order:
 - canvaskit-skottie-frames-load.html (note it is similar to
   skottie-wasm-perf.html, but it waits for a button click
   to start animating and records times from the main JS thread
   itself)
 - perf-with-puppeteer.js (similar to skottie-wasm-perf.js, but
   has some things made optional [e.g. tracing])
 - perf_puppeteer_test.go (shows the inputs/outputs of various steps)
 - perf_puppeteer.go
 - Everything else.


Change-Id: I380e81b825f36682c257664d488267edaf36369e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/285783
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2020-05-01 19:23:08 +00:00
Kevin Lubick
4baa7326cc [infra] Add POC task driver
Change-Id: I0f8972b62c938389772872adea2c96acc29ca503
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/285665
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
2020-04-28 11:50:17 +00:00
Eric Boren
12da772572 [infra] Update infra dep again
https://skia-review.googlesource.com/c/buildbot/+/279043 requires a
manual update to gen_tasks_logic.

Change-Id: I3871ed8940e08019a9248d26b90a10e136d1daf0
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/279144
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2020-03-26 10:40:39 +00:00
Eric Boren
457c1eb913 [infra] Minor refactor for recipe properties in gen_tasks
Change-Id: I76fb1e5da5dd162e8245758ccae8effac2f21c0d
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/277179
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
2020-03-16 18:29:50 +00:00
Eric Boren
fae71fb596 [infra] gen_tasks: fix TODOs in usesDocker()
This adds the "docker_installed=true" dimension for some tasks which are
currently using Docker but do not set the dimension.

Change-Id: I94ede7376e081b861bc691111b58eda37ccf6c41
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/276002
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2020-03-10 10:17:59 +00:00
Eric Boren
23899c64e3 [infra] gen_tasks: Fix TODOs in usesGit()
This removes the "work" cache from tasks which should not need it, and
adds the "git" cache to all tasks which use Git.

Change-Id: Ie3e5e964d77db44db869030ba195a50e27b75e70
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/275996
Reviewed-by: Ben Wagner aka dogben <benjaminwagner@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2020-03-09 16:23:33 +00:00
Eric Boren
8a05d23363 [infra] More gen_tasks tweaks
Behavioral changes:
- CreateDockerImage and PushAppsFrom*DockerImage tasks no longer get go
  or protoc assets and caches.
- Skpbench on desktop uses CIPD packages for [m]skp, rather than the
  isolated versions.

Change-Id: I0532546be9a4a79434b2d2017061322884ba11fb
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/275956
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
2020-03-09 14:37:58 +00:00
Eric Boren
c27e70235f [infra] Overhaul gen_tasks_logic
- Add jobBuilder and taskBuilder for convenience.
- Add parts type with convenience matching funcs.

Change-Id: I8213a1e3742d055515e7b8520b2dec0c06c6f2d3
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/275076
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ben Wagner aka dogben <benjaminwagner@google.com>
2020-03-09 13:02:47 +00:00