This removes the previous task driver dedicated to enforcing
IWYU and adds individual jobs for the binaries we want to
test. This aligns with the current plan of organizing our
Bazel CI jobs - going wide where possible.
Change-Id: I8b163957e34e594c1a80d7b9e6b63c64a277bef1
Bug: skia:12541 skia:13052
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/547019
Reviewed-by: Eric Boren <borenet@google.com>
Needed to fix the Dawn and Harfbuzz Bazel rules.
Change-Id: I21f63c970bdc972b97e42ef85d82d7f478bd45e2
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/545721
Reviewed-by: Eric Boren <borenet@google.com>
gazelle ended up being more liability than asset for our C++ rules.
It required devs to manually run the command frequently (and was
easy to forget until the CQ failed). The fact that we still had to
edit the source files (e.g. the "srcs" cc_libraries) meant that
the mixture between generated and hand-written caused some
tension (see include/third_party/vulkan for a good example).
The combination of gazelle and our IWYU enforcement added several
bits of churn without any real benefit. The generated rules
also didn't help identify cases where we were not keeping tight
boundaries (e.g. non-gpu code and gpu code).
Identifying third_party deps automatically ended up being trickier
than anticipated (see the deleted //third_party/file_map_for_bazel.json)
Using the "maximum set of dependencies" worked ok, but ended up
increasing build time unnecessarily. For example, compiling
CanvasKit for WebGL always needed to compile Dawn because
SkSLCompiler.cpp sometimes needs to include tint/tint.h.
Follow-up CLs will rebuild the BUILD.bazel rules without gazelle.
Note to Reviewers:
- The only file worth manually reviewing here is bazel/Makefile.
Change-Id: I36d6fc3747487fabaf699690780c95f1f6765770
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/543976
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Reviewed-by: Ben Wagner <bungeman@google.com>
This change introduces the use of "Android12" as an OS, so
many checks had to be changed from b.os("Android") to
b.matchOs("Android") which treats "Android" and "Android12"
the same.
Bug: skia:12912
Change-Id: Ib0a5f9d8f1a06b62e529a1567efcde52c7bb44d0
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/527496
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Joe Gregorio <jcgregorio@google.com>
Appeasing IWYU and the Gazelle-generated BUILD.bazel files is currently
annoying for devs working on non-linux environments.
I have been experimenting with removing the gazelle rules (but still
keeping our BUILD.bazel rules organized), which would alleviate
some of the burden here. Until that is finished, we can stop one
of the two jobs.
Change-Id: Id6c43bd98c6ce41e1ae9a6bfc1102ff47a6e5b1f
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/543496
Reviewed-by: Brian Salomon <bsalomon@google.com>
Change-Id: I6db702c2d231bfe527b3fd4bf4f99beb6e58bf8a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/542641
Auto-Submit: Michael Ludwig <michaelludwig@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
Commit-Queue: Michael Ludwig <michaelludwig@google.com>
This is similar to https://skia-review.googlesource.com/c/buildbot/+/538218
BIG CHANGE: If we fail to download uninteresting hashes
(because Gold is down or the script otherwise fails), we
now crash/fail our Test-* tasks. In an early version of
this removal, that lack of failure masked an issue with
the script (urllib2 is not in Python3) and nearly would
have landed. Failing loudly is better, IMO.
This removes the symbolize_stack_trace script, which
stopped working with Python3 for reasons unknown.
If we need the behavior, we can rewrite it.
Relatedly, we removed 4 jobs, the Docker ones, because
we will not get much value out of them as we migrate
towards Bazel and removing the symbolization script
was tricky to get right.
There are a few cleanups around copypasta that I noticed
when combing through the recipes.
Change-Id: I8dfab416e964fd494267800b4ebe216061895f19
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/538636
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Bug: 1320964
Change-Id: I5f05ea469b824b1382f64c1eb3fb9be0498cf329
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/537517
Reviewed-by: Greg Daniel <egdaniel@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Robert Phillips <robertphillips@google.com>
For additional context, see "Codifying Certain Build Options"
and "Building on the CI" in the design doc go/skia-bazel
Suggested review order:
- builder_name_schema.json to see the three required and
one optional part of BazelBuild jobs.
- jobs.json to see one new BazelBuild job added. In an
ideal world, this job would have been named
BazelBuild-//modules/canvaskit:canvaskit_wasm-debug-linux_x64
but Buildbucket (?) requires jobs match the regex
^[a-zA-Z0-9\\-_.\\(\\) ]{1,128}$
so we use spaces instead of slashes or colons.
- gen_tasks_logic.go; noting the makeBazelLabel function
expands most of the spaces to / and the last one to a
colon to make a single-target label. If there are three
dots, then it is a multi-target label, and we do not
need to add a colon.
- bazel_build.go; This is a very simple task driver, and
I do not anticipate getting too much more complex.
The place where we decide which args to augment
a build with depend on the host platform and thus
should be set in gen_tasks_logic.go.
- bazel/buildrc to see some initial configurations set,
one of which, "debug", is used by the new job.
The "release" version of CanvasKit probably works on
3.1.10 which had a bugfix, but we are still on
3.1.9
- .bazelrc to see a rename of the linux-rbe config to
linux_rbe (our configs should have no dashes if
we want to specify them verbatim in our Job names).
It also imports the Skia-specified build configs
from //bazel/buildrc and supports the user-specified
//bazel/user/buildrc file if it exists.
- All other files in any order.
Change-Id: Ib954dd6045100eadcbbf4ffee0888f6fbce65fa7
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/537797
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Jorge Betancourt <jmbetancourt@google.com>
Bug: skia:13040
Bug: skia:9235
Bug: skia:10412
Bug: skia:12437
Change-Id: Ia96f8332b5372ecf65cb20ffb9549bc0dc8f931b
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/537080
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
There are two SKP versions we deal with here:
* The min and current supported versions in src/core/SkPicturePriv.h.
* The SKP asset version that is incremented weekly in infra/bots/assets/skp/VERSION.
We will need to update the asset version used by the bot when the min version in src/core/SkPicturePriv.h is updated.
Instructions on how to do that have been documented in src/core/SkPicturePriv.h.
As noted in https://bugs.chromium.org/p/skia/issues/detail?id=13278#c2 DM currently does not fail when it fails to parse an SKP. Till DM is updated to fail, developers will have to look for blank images in Gold via this new bot to determine when we are failing to support the oldest SKP version.
Bug: skia:13278
Change-Id: I8fff62cc289c3bd6abf5179bcee349baf0a8188a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/536106
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
Reviewed-by: Brian Salomon <bsalomon@google.com>
Added Test-Ubuntu18-Clang-Golo-GPU-QuadroP400-x86_64-Debug-All-Dawn
to exercise the Dawn Vulkan backend.
Change-Id: I3679822a1460eda36654f99c70ba9510e394b40c
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/536736
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Arman Uguray <armansito@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Bug: skia:13272
Change-Id: I6ba8d43a723fddf1ecbc06c6f7d1d8295428d342
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/536436
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Add a Mac Mini to test using GrSlug drawing instead of using
GrTextBlob for drawing.
Change-Id: I28ee402b7f82ae235518ee585d5803857490f8e1
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/535964
Reviewed-by: Brian Osman <brianosman@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Herb Derby <herb@google.com>
This change does the following:
* Removes 2 Golo Test-TSAN_Vulkan jobs because their counterparts already exist in Skolo.
* Moves 1 Golo Test-TSAN_Vulkan job to Skolo.
Bug: skia:13129
Change-Id: I28b40320c831d109aa4f8778ddc1d03be9473592
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/526916
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
This moves the Build-Debian10-BazelClang-x86_64-Release-IWYU
job from experimental to on when a file in one of the
folders that we enforce IWYU is modified (currently
for svg, sksl, and now, debugger).
Change-Id: Ia6fe1e7b30fc486db3eb081b6a64bc4c250cbf0b
Bug: skia:13052
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/525796
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
There are a few driver issues on the Golo machines that we
are trying to work out.
This removes 2 Perf-ASAN jobs, as one is probably sufficient
given that we have Test-ASAN variants too.
Change-Id: Id2aa492a64706562286b820709f082f2374e2222
Bug: 1309590
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/525191
Reviewed-by: Ravi Mistry <rmistry@google.com>
The toolchain now uses extract_ar from
https://skia-review.googlesource.com/c/buildbot/+/524764
which is a static executable to extract the .deb files.
This was necessary because the llvm-ar that had previously
been used requires glibc 2.31+ to run, but our Debian10
machines on Swarming have an older version (2.28).
A longer-term fix is to have Bazel support .ar files,
which I plan to attempt to contribute this week.
The RBE task will be added as an experimental CQ job, to
see how it handles the load of running often. With the
remote execution cache, I hope it performs well, once
the toolchains are cached on both the Swarming
machines and in the RBE workers.
Note: We had to add several files to the CAS spec
(see compile_cas.go) which are required for Bazel to work.
Change-Id: Ie70c70d5f33768c957760f9eeb7835025109b487
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524759
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Change-Id: I7900eec79e5608bb0456ab5681bc8015b768ead5
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/522741
Reviewed-by: Michael Ludwig <michaelludwig@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
Change-Id: If2e7bfd93b916663e568e7d761cd891d0ac2303a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/521838
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Joe Gregorio <jcgregorio@google.com>
Bug: skia:12974
Change-Id: I6ec7736edc871241b0cf78413b0d5f7ff9abd8c5
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/520736
Reviewed-by: Jim Van Verth <jvanverth@google.com>
Commit-Queue: Greg Daniel <egdaniel@google.com>
Bug: skia:13033
Change-Id: I2b4a30f8adb927966ab20e8175a0dcf15fb3ed1e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/518636
Reviewed-by: Erik Rose <erikrose@google.com>
Commit-Queue: Jim Van Verth <jvanverth@google.com>
The LoopInt test fails in Metal on many devices with macOS 12, so it's
currently disabled.
Change-Id: I575b662ae543187d7062e0fba37afa6e80663fc5
Bug: skia:13005
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/514576
Auto-Submit: John Stiles <johnstiles@google.com>
Reviewed-by: Arman Uguray <armansito@google.com>
Commit-Queue: Arman Uguray <armansito@google.com>
That was making Gerrit unable to invoke the jobs; the regex for valid
jobs doesn't contain comma.
Change-Id: I9b8a42ab1d5b8ac204895388844bde88a604491f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/514356
Reviewed-by: Jim Van Verth <jvanverth@google.com>
Commit-Queue: Erik Rose <erikrose@google.com>
These have been failing for 9+ months
Change-Id: I9cc15298ee0ea1f2f1070affeeb5392ced376793
Bug: skia:12268
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/514276
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Change-Id: I51d3f94bcb483e6e3de6f54a2e10683b0ea6d3d0
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/512162
Reviewed-by: Robert Phillips <robertphillips@google.com>
Reviewed-by: Erik Rose <erikrose@google.com>
Back in 47045c9e06 I had meant to "Add a
Debug-All-NativeFonts CPU job for all OS versions on the new Mac Mini 7.1s."
Well, none of those is running 12.
Change-Id: I9dd13ce33c9a4004faf4da5f53d25c5ce5d8ec04
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/509718
Reviewed-by: Ben Wagner <bungeman@google.com>
Commit-Queue: Erik Rose <erikrose@google.com>
Change-Id: Ifcd6e9ced293d51b5c75849ec4a6a9899b2a0459
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/506780
Reviewed-by: Ben Wagner <bungeman@google.com>
Commit-Queue: Erik Rose <erikrose@google.com>
...since the other Debug-All-NativeFonts GPU job is on a different GPU
and CPU ISA.
Add an "i5" extraConfig option so we can distinguish between the 2
differently-CPU'd MacMini7,1s.
Change-Id: I609d5361c313a005d888ca7a847375cad78dec01
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/506778
Reviewed-by: Ben Wagner <bungeman@google.com>
...since we have so many of those machines (which were recently racked).
Change-Id: I61baa5d7b5140633d707411ab3a7a9ecc376b042
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/506777
Reviewed-by: Ben Wagner <bungeman@google.com>
This CL adds the task driver and plumbing necessary to run CI tasks that upload build statistics to https://codesize.skia.org.
For now, a single CodeSize-dm-Debian10-Clang-x86_64-Release task is defined, which as the name suggests:
- Depends on Build-Debian10-Clang-x86_64-Release.
- Takes the "dm" binary built by the aforementioned job.
- Runs Bloaty against said binary.
- Uploads the Bloaty output, along with a JSON file with metadata, to the GCS bucket where https://codesize.skia.org gets its data from.
If one wishes to upload codesize statistics for another binary, the steps are as follows:
- Identify the target binary, for example "fm".
- Identify a compile task that builds said binary with the desired settings, for example "Build-Debian10-Clang-x86_64-Debug".
- Add a new job in //infra/bots/jobs.json named "CodeSize-<binary>-<compile task>", where <compile task> is the name of the compile task without the "Build-" prefix, for example "CodeSize-fm-Debian10-Clang-x86_64-Debug".
- Run "make train" from the //infra/bots directory.
Eventually the codesize.skia.org webserver will automatically pick up the output of any such CodeSize-* task and show it in the UI, with no additional steps needed.
One caveat is that the binary file name, and therefore the "<binary>" part of the task name, cannot contain dashes. This is due to how //infra/bots/gen_tasks_logic.go works. See comments in said Go file for ideas on how to work around this if we ever need to.
Bug: skia:12151
Change-Id: If406944ca7660c4dd15c8e6b8f34e48c65cbbe2f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/502788
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Leandro Lovisolo <lovisolo@google.com>
...based on conversation with jvanverth@ on 2022-02-04 in Skia GChat.
Change-Id: I463737e33e87f2052326f6231781a801ad5c320c
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/506877
Reviewed-by: Jim Van Verth <jvanverth@google.com>
Commit-Queue: Erik Rose <erikrose@google.com>
These Minis have a new CPU (I5-4278U) we've never cared about before, so
add that too. (We had some other Minis of the same model in the Golo,
but we didn't care about their CPUs because we were running only GPU
jobs on them.)
If you want to see a good diff of those, use FileMerge or something that
uses a similar algorithm; git makes a total hash of it.
Change-Id: Id7c4f6bc06f83d2ed4a0034e9174b9e69d3d15b2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/495424
Reviewed-by: Ben Wagner <bungeman@google.com>
Commit-Queue: Erik Rose <erikrose@google.com>
