This adds targets which test our Dawn, GL, and Vulkan backends.
It follows the hierarchical filegroup pattern, as
outlined in https://skia-review.googlesource.com/c/skia/+/543977
Suggested Review order:
- tools/sk_app/BUILD.bazel. For many things in tools, I anticipate
they will depend on //:skia_core and other //tools targets.
sk_app shows this off, as well how to make the target
specific to a given platform and pull in the proper native code.
I'm trying out setting test_only = True, to see if we can
partition Skia's tests and helpers from the actual Skia library.
- other changes to //tools/, especially looking at sk_app's
dependencies.
- //example/BUILD.bazel. This uses the cc_binary_with_flags which
existed previously [1] to make it so people don't have to
specify all the flags for a given binary and can build it as is.
These targets nows how up in //bazel/Makefile
- //include/... and //src/..., where some typos from previous
CLs were fixed and rules expanded.
- Misc changes to .cpp files to remove unnecessary includes
that were assuming the GL backend was being compiled in.
- All other changes
[1] 162dfca340/bazel/cc_binary_with_flags.bzl
Change-Id: Ieacec464d44368cad0da0890c7dc85a6c0b900c9
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/544317
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
The primary goal of this organization structure is to keep
our top level BUILD.bazel file short, with as little logic
as feasible. The logic required to control which files to
include, which third_party deps are needed, what system libraries
should be linked again, etc, should be in the BUILD.bazel
file best should be as close to the affected files as feasible.
In essence, we use filegroup() rules to bubble up the files
needed to build Skia (all as one big cc_library call) and
cc_library rules to bubble up the other components needed to build.
For example, //src/ports/SkFontHost_FreeType.cpp needs FreeType,
but only if we are compiling Skia with that type of font
support. With the new organization structure in this CL,
//src/ports/BUILD.bazel should have the logic that determines
if the cpp file should be included in the build of Skia and
if it is, that the Skia build should depend on //third_party:freetype2
Another example is //src/gpu/ganesh/BUILD.bazel, which
chooses which of the dawn, gl, vulkan, etc backend sources,
and the associated dependencies to include in the build.
It does not specify what those are, but delegates to the
BUILD.bazel files in the subdirectories housing the
backend-specific code.
The structure guidelines for BUILD.bazel files are as follows:
- Have a filegroup() called "hdrs" (for public headers) or
"srcs" (for private headers and all .cpp files) that is
visible to the parent directory. This should list the
files from the containing directory to include in the
build.
See //include/core/BUILD.bazel and //src/effects/BUILD.bazel
as examples.
- filegroup() rules can list a child directory's "hdrs"
or "srcs" in their "srcs" attributes, but should not contain
select statements pertaining to child directory files.
See //include/gpu/BUILD.bazel and //src/gpu/ganesh/BUILD.bazel
as examples.
- May have a cc_library() called "deps". This can specify
dependencies, cc_opts, and linkopts, but not srcs or hdrs. [1]
See //src/codec/BUILD.bazel as an example. These should
be visible to the parent directory.
- "hdrs", "srcs", and "deps" for the primary Skia build
(currently called "skia_core") should bubble up through
//include/BUILD.bazel and //src/BUILD.bazel, one directory
at a time.
This CL demonstrates a very basic build of Skia with many features
turned off (CPU only, no fonts, no codecs). Follow-on CLs will
add to these rules as more targets are supported. See bazel/Makefile
for the builds that work with just this CL.
Suggested Review Order:
- //BUILD.bazel to see the very small skia_core rule which
delegates all the logic down stack. Note that it has a
dependency on //bazel:defines_from_flags which will set
all the defines listed there when compiling all the
.cpp and .h files in skia_core *and* anything that depends
on skia_core, but *not* //src:deps.
- //include/BUILD.bazel and other BUILD.bazel files in the
subdirectories of that folder. Note that the filegroups in
//include/private/... are called "srcs" to be similar to
how Bazel wants "private headers" to be in the "srcs" of
cc_library, cc_binary, etc. and only public headers are
to be in "hdrs" [2].
- //src/BUILD.bazel and other BUILD.bazel files in the
subdirectories of that folder. //src/gpu/ganesh/...
will be filled in for dawn, vulkan, and GL in the next CL.
- //PRESUBMIT.py, which adds a check that runs buildifier [3]
on modified BUILD.bazel files to make sure they stay
consistently formatted.
- //bazel/... to see the new option I added to make sksl
opt-in or opt-out, so one could build Skia with sksl,
but not with a gpu backend.
- Misc .h and .cpp files, whose includes were removed if
unnecessary or #ifdef'd out to make the minimal build
work without GPU or SkSL includes.
- //bazel/Makefile to see the builds that work with this CL.
[1] Setting srcs or hdrs is error-prone at best, because those
files will be compiled with a different set of defines than
the rest of skia_core, because they wouldn't depend on
//bazel:defines_from_flags.
[2] https://bazel.build/reference/be/c-cpp#cc_library.hdrs
[3] https://github.com/bazelbuild/buildtools/releases
Change-Id: I5e0e3ae01ad42d672506d5aad1239f2512188191
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/543977
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Reviewed-by: Ben Wagner <bungeman@google.com>
gazelle ended up being more liability than asset for our C++ rules.
It required devs to manually run the command frequently (and was
easy to forget until the CQ failed). The fact that we still had to
edit the source files (e.g. the "srcs" cc_libraries) meant that
the mixture between generated and hand-written caused some
tension (see include/third_party/vulkan for a good example).
The combination of gazelle and our IWYU enforcement added several
bits of churn without any real benefit. The generated rules
also didn't help identify cases where we were not keeping tight
boundaries (e.g. non-gpu code and gpu code).
Identifying third_party deps automatically ended up being trickier
than anticipated (see the deleted //third_party/file_map_for_bazel.json)
Using the "maximum set of dependencies" worked ok, but ended up
increasing build time unnecessarily. For example, compiling
CanvasKit for WebGL always needed to compile Dawn because
SkSLCompiler.cpp sometimes needs to include tint/tint.h.
Follow-up CLs will rebuild the BUILD.bazel rules without gazelle.
Note to Reviewers:
- The only file worth manually reviewing here is bazel/Makefile.
Change-Id: I36d6fc3747487fabaf699690780c95f1f6765770
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/543976
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Reviewed-by: Ben Wagner <bungeman@google.com>
This reverts commit bfc9b94840.
Reason for revert: new dependency causing Android build error
Original change's description:
> Add Perfetto library (gn & bazel) and bare-bones SkPerfTrace class
>
> First incremental step to incorporating Perfetto tracing into Skia, more CLs to follow
>
> NOTE: The presubmit check is failing. This appears to be due to the known issue where the presubmit check bugs out if the DEPS file is edited, which it was on this CL (modified to include Perfetto).
>
> Bug: skia:13303
> Change-Id: I908be0392b520e8da14b34588b842bf6d955bd93
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/543081
> Reviewed-by: Greg Daniel <egdaniel@google.com>
> Commit-Queue: Nicolette Prevost <nicolettep@google.com>
Bug: skia:13303
Change-Id: Ia2b883bbab1f8fb4f3914b63104a39240cc60e86
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/544239
Reviewed-by: Tyler Denniston <tdenniston@google.com>
Auto-Submit: Tyler Denniston <tdenniston@google.com>
Reviewed-by: Nicolette Prevost <nicolettep@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Tyler Denniston <tdenniston@google.com>
First incremental step to incorporating Perfetto tracing into Skia, more CLs to follow
NOTE: The presubmit check is failing. This appears to be due to the known issue where the presubmit check bugs out if the DEPS file is edited, which it was on this CL (modified to include Perfetto).
Bug: skia:13303
Change-Id: I908be0392b520e8da14b34588b842bf6d955bd93
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/543081
Reviewed-by: Greg Daniel <egdaniel@google.com>
Commit-Queue: Nicolette Prevost <nicolettep@google.com>
- Introduced the SK_ENABLE_WGSL_VALIDATION macro which is currently only
enabled when skslc gets compiled when using the `skia_compile_sksl_tests`
setting.
- SkSLCompiler::toWGSL now validates its output using Tint's WGSL reader
structures based on conditionally compiled code depending on the
SK_ENABLE_WGSL_VALIDATION flag.
- Fixed `warning: use of deprecated language feature: struct members should be separated with commas"
warnings that were generated for HelloWorld.wgsl.
Bug: skia:13092
Change-Id: Ib894457030004966221faf82f61360e390b95e22
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/537802
Commit-Queue: Arman Uguray <armansito@google.com>
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
third_party/**/BUILD.gn have a copyright header, so we'll add one
to our third_party/**/BUILD.bazel files too.
Change-Id: Ifc04d36624af07d91d279a139f3a691e51ce7418
Bug: skia:13323
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/541076
Reviewed-by: Ravi Mistry <rmistry@google.com>
Rather than having a monolithic third_party/BUILD.bazel, this moves
our Dawn rules to their own subdirectory and makes it callable via
@dawn instead of //third_party/dawn.
This will help with the G3 roll and make our rules more organized in
general.
This also rolls Dawn
Roll Dawn from ab9757036bd6 to e831fb61046b (22 revisions)
https://dawn.googlesource.com/dawn.git/+log/ab9757036bd6..e831fb61046b
Suggested Review Order:
- WORKSPACE.bazel, where we define @dawn and its deps
(@vulkan_headers and @vulkan_tools). I initially thought
I needed to define all of Dawn's deps in the workspace_file_content
for new_local_repository, but that WORKSPACE file is
ignored when building Skia rules.
- third_party/dawn/BUILD.bazel, the contents of which were copied
from //third_party/BUILD.bazel and modified largely via
find-and-replace to point to files relative to
//third_party/externals/dawn. One exception is the cpu_wasm
config_setting because @dawn isn't able to see Skia's
//bazel/macros.bzl.
- All other files
Change-Id: Ib2d7bc972ef00b6b68370ce5c2839ffb70ed9a2f
Bug: skia:12541, skia:13211
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/538638
Reviewed-by: Ben Wagner <bungeman@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Change-Id: Iac6b8a0b74fe23959f5ee814f4a30a5458ccf21e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/539936
Reviewed-by: John Stiles <johnstiles@google.com>
Auto-Submit: Michael Ludwig <michaelludwig@google.com>
Commit-Queue: John Stiles <johnstiles@google.com>
Commit-Queue: Michael Ludwig <michaelludwig@google.com>
Change-Id: I2a1fedbbaabd57e4546c2a628f6d297e11f4244b
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/539318
Reviewed-by: John Stiles <johnstiles@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Michael Ludwig <michaelludwig@google.com>
https://dawn.googlesource.com/dawn.git/+log/9483b741a884..be0abb611aed
$ git log 9483b741a..be0abb611 --date=short --no-merges --format='%ad %ae %s'
2022-05-09 dawn-autoroll Roll ANGLE from e316203a6a2f to 78d88796b44b (1 revision)
2022-05-08 dawn-autoroll Roll ANGLE from 225d8f83af71 to e316203a6a2f (1 revision)
2022-05-07 dawn-autoroll Roll SwiftShader from 00efa1913863 to 1dd93361b1d9 (1 revision)
2022-05-07 dawn-autoroll Roll ANGLE from cadb933fdb12 to 225d8f83af71 (2 revisions)
2022-05-07 dawn-autoroll Roll vulkan-deps from e13072c54977 to 626b2fd72bde (2 revisions)
2022-05-07 dawn-autoroll Roll SwiftShader from 74e34ab97aeb to 00efa1913863 (2 revisions)
2022-05-06 dawn-autoroll Roll ANGLE from 1d5d09e4191a to cadb933fdb12 (14 revisions)
2022-05-06 dawn-autoroll Roll vulkan-deps from 4957ae734445 to e13072c54977 (21 revisions)
2022-05-06 dawn-autoroll Roll SwiftShader from 62c1af08ea4d to 74e34ab97aeb (5 revisions)
2022-05-06 bclayton tint: Rename and move builtin_table and builtin-gen
2022-05-06 bclayton dawn: Fix clang warning treated as error
2022-05-06 zhaoming.jiang dawn: Add shader module validation for WGSL extension
2022-05-06 dawn-autoroll Roll ANGLE from 2ce60b52a239 to 1d5d09e4191a (10 revisions)
2022-05-05 jrprice Surface Vulkan validation messages in errors
2022-05-05 bclayton tint: Change all ProgramBuilder literals to 'i' or 'u' suffix
2022-05-05 dawn-autoroll Roll ANGLE from 9865ed8b8117 to 2ce60b52a239 (4 revisions)
2022-05-05 bclayton tint: Castable - support non-default-constructable return types
2022-05-05 dawn-autoroll Roll SwiftShader from 2e793ae08002 to 62c1af08ea4d (1 revision)
2022-05-05 amaiorano tint: add --overrides flag to specify pipeline overrides
2022-05-05 pkasting Fixes for C++20 support.
2022-05-05 bclayton tint: Add Bitcast helper
2022-05-05 bclayton tint: Fix CFI error in BlockAllocator
2022-05-05 dawn-autoroll Roll ANGLE from 9053a641bf5d to 9865ed8b8117 (1 revision)
2022-05-04 senorblanco OpenGLES: disable some end2end tests which require reading from depth textures.
2022-05-04 bclayton tint/reader/spirv: Generate 'i' suffixed literals
2022-05-04 bclayton tint: Merge [S|U]intLiteralExpression
2022-05-04 dawn-autoroll Roll SwiftShader from 643179694ec7 to 2e793ae08002 (1 revision)
2022-05-04 bclayton tint: Lex three types of integer literal
2022-05-04 dsinclair Enable more `gn check` results.
2022-05-04 dawn-autoroll Roll ANGLE from 84e42c3b04da to 9053a641bf5d (12 revisions)
2022-05-04 bclayton webgpu-cts/expectations.txt: Remove expectations that now pass
2022-05-04 amaiorano Disable angle dependency on wayland
2022-05-04 bclayton tint: Chromium-style fixes
2022-05-04 jamessliu2020 AST fuzzer: Change unary expression operator
2022-05-04 dawn-autoroll Roll SwiftShader from c75846ead9a0 to 643179694ec7 (3 revisions)
Created with:
roll-dep third_party/externals/dawn
Change-Id: I6656ab41573ce465e9ced94ad65731be9bc5dafe
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/538716
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Commit-Queue: Leandro Lovisolo <lovisolo@google.com>
Starting with Visual Studio 2019 version 16.9 (msvc++ 14.28,
__MSC_VER 1928) has support for it's own version of AddressSanitizer.
Change-Id: I106b7e765ac80e4fc6eabd5b88500cbec5e38714
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/537461
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Ben Wagner <bungeman@google.com>
Any settings in the FreeType build which may modify the public FreeType
headers must be public_defines. Otherwise FreeType may be built one way
and the dependents on this target may be built with what are essentially
different headers.
This fixes the Skia build with built-in FreeType on Windows.
Change-Id: I6b317fbe491dd6fdd135928b6f50436102a1e281
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/537460
Commit-Queue: Herb Derby <herb@google.com>
Auto-Submit: Ben Wagner <bungeman@google.com>
Reviewed-by: Herb Derby <herb@google.com>
G3 prefers license() first.
This was done mechanically with a big find/replace
Change-Id: I8c33c7bc10a6bec42e966cad81c259954e841811
Bug: skia:13211
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/535898
Reviewed-by: Ben Wagner <bungeman@google.com>
Ran the following commands:
find -name "BUILD.bazel" -exec sed -i -e '1iload("//bazel:macros.bzl", "cc_library", "exports_files_legacy")\nexports_files_legacy()' {} +
buildifier --lint=fix --mode=fix -r .
This had the effect of making sure we can export all of our
files in G3 (until we no longer have legacy targets) and
making all of our cc_libraries shim-able.
bazel/macros.bzl has the human-contributed changes, the rest
were mechanical.
Change-Id: I8e24e30e74b038cfd072cdbe4078bfd1d213dd46
Bug: skia:13211
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/535359
Reviewed-by: Ben Wagner <bungeman@google.com>
To make it easier to use the G3 version of this dep,
we make a reference to it in the WORKSPACE.bazel, so
we can refer to this external dep with the @ notation.
I would like to do this for all of our third party
deps, but one at a time.
Change-Id: I03e0beca124225e0faf1232278dae641da8a3e4d
Bug: skia:13240, skia:13211
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/535358
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Also updated the gn_to_bp.py to handle using the DEPSed vk_mem_alloc.h.
When rolled into Android we will copy the vk_mem_alloc.h from a local
DEPS checkout into a vma_android folder that will be checked into the
skia that lands with Android. We are using vma_android folder instead
of the current third_party/vulkanmemoryallocator folder because we
soon plan to move the latter into src/ instead of third_party. So just
using a different directory allows us to avoid doing an additional
change to the auto roller.
Bug: skia:13240
Bug: skia:13242
Change-Id: Ia344e13e3f7c7efecc2e6a97a96820f3ae58b5c2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/531318
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Greg Daniel <egdaniel@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
Reviewed-by: Leon Scroggins <scroggo@google.com>
Change-Id: Ic93f264fb556ecf271f9c2642f7c60665669b092
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/534499
Commit-Queue: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Salomon <bsalomon@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Auto-Submit: Brian Salomon <bsalomon@google.com>
It's time to better organize //third_party/BUILD.bazel
This CL starts by moving the "third party" stuff we closely own.
Change-Id: I3e0be0044b790794e94f34af6202860ce0a7b7aa
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/531999
Reviewed-by: Ben Wagner <bungeman@google.com>
Change-Id: Ia4dac275a8614d7597e5c17cae77ad3770ba27e2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/532001
Auto-Submit: Michael Ludwig <michaelludwig@google.com>
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
This is a reland of commit f33cf451a2
Original change's description:
> Update VMA to latest version.
>
> This is needed so that we can use non skia supplied vk_mem_alloc.h on
> clients that have already updated to newest version. As we transition
> to Bazel builds this helps to make it so we can set the rules for
> specific clients around VMA without things breaking
>
> Bug: skia:13211
> Change-Id: I5d38a3a91a44f6b3fdf75894a3248ee3991dd5d9
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/531157
> Reviewed-by: Kevin Lubick <kjlubick@google.com>
Bug: skia:13211
Change-Id: Ia4a863819a7d0bb4763f1b516acfa1cc76c47a8a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/531548
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Greg Daniel <egdaniel@google.com>
This reverts commit f33cf451a2.
Reason for revert: breaking the android roller
Original change's description:
> Update VMA to latest version.
>
> This is needed so that we can use non skia supplied vk_mem_alloc.h on
> clients that have already updated to newest version. As we transition
> to Bazel builds this helps to make it so we can set the rules for
> specific clients around VMA without things breaking
>
> Bug: skia:13211
> Change-Id: I5d38a3a91a44f6b3fdf75894a3248ee3991dd5d9
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/531157
> Reviewed-by: Kevin Lubick <kjlubick@google.com>
Bug: skia:13211
Change-Id: Ibd621aa2f07359f9b72e1a35df0f50ca3c063e86
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/531543
Auto-Submit: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This is needed so that we can use non skia supplied vk_mem_alloc.h on
clients that have already updated to newest version. As we transition
to Bazel builds this helps to make it so we can set the rules for
specific clients around VMA without things breaking
Bug: skia:13211
Change-Id: I5d38a3a91a44f6b3fdf75894a3248ee3991dd5d9
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/531157
Reviewed-by: Kevin Lubick <kjlubick@google.com>
For importing into G3, this will simplify the regex needed
and allow us to stub out things not supported (e.g. gazelle)
Change-Id: I770f5dee6f29e555356742dae36212ad6cfb713a
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/529836
Reviewed-by: Ben Wagner <bungeman@google.com>
This ports the GN skia_executable [1] and adds third_party
Bazel rules for spirv_cross, translated from [2]. spirv_cross,
unlike spirv_tools, did not have pre-made Bazel rules.
This binary compiles and links with
bazelisk build //tools/skslc --config=clang
[1] ad324a31e6/BUILD.gn (L712)
[2] ad324a31e6/third_party/spirv-cross/BUILD.gn (L10)
Change-Id: I4abd51889552153fc7e64a5f7f3d9f0f597524e7
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/528044
Reviewed-by: John Stiles <johnstiles@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Includes a manual roll of Dawn, as well as tweaks to the Tint build
overrides.
Dawn needs git in order to generate a header that has the
version of Dawn compiled in. If we use Dawn's GN rules to
build it, we need git to generate that file.
Our (now updated) Bazel rules to build Dawn do not need to
do this because we (Skia) do not use this file [1]
[1] https://dawn.googlesource.com/dawn/+/088a600b03679cd20991f145173a573ed9c03480/src/dawn/common/BUILD.gn#177
Change-Id: Id19fdbb9b53fdad2397a0044a4a314b1444faeb0
Cq-Include-Trybots: luci.skia.skia.primary:Build-Debian10-Clang-x86_64-Debug-Dawn,Build-Win-Clang-x86_64-Release-Dawn
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/528157
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
A new RBE worker-pool called gce_linux was created in
conjunction with this CL. See
https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit#
for some details on that.
Note: everything under bazel/rbe/gce_linux was autogenerated
and can be ignored from manual review. It basically specifies
what files are on the RBE image that are necessary for running
Bazel.
Testing it out can be done by authenticating for RBE
gcloud auth application-default login --no-browser
Then, run make -C bazel rbe_known_good_builds
to test it out.
On my 4 core laptop with an empty local cache, but a
warm remote cache, the build took <2 min instead of the
10+ minutes it would have [1].
The folder structure in //bazel/rbe is meant to let us
have multiple remote configurations there, e.g.
//bazel/rbe/gce_windows.
Suggested Review Order:
- bazel/rbe/README.md
- bazel/rbe/gce_linux_container/Dockerfile to see the
bare-bones RBE image.
- bazel/rbe/BUILD.bazel to see a custom platform defined.
It is nearly identical to the autogenerated one
in bazel/rbe/gce_linux/config/BUILD, with one extra
field to force the gce_linux pool to be used.
- .bazelrc to see the settings needed to make
--config=linux-rbe work. The naming convention was
inspired by SkCMS's setup [2], and allows us to have
some common RBE settings (i.e. config:remote) and
some specialized ones for the given host machine
(e.g. config:linux-rbe) A very important, but subtle
configuration, is on line 86 of .bazelrc where we say
to use our hermetic toolchain and not whatever C++
compiler and headers are on the host machine (aka
the RBE container).
- toolchain/build_toolchain.bzl to see some additional
dependencies needed in the toolchain (to run IWYU) which
I had installed locally but didn't realize were important.
- third_party/BUILD.bazel to see an example of how failing
to specify all files can result in something that works
locally, but fails remotely.
--execution_log_json_file=/tmp/execlog.json helped debug
these issues.
- All other files.
[1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA
[2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20
Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
The file generation logic that dawn [1] uses to make some
source files requires jinja2, which also requires MarkupSafe.
The GN build handles this by specifying those repos in
DEPS, checking them out at a certain git hash, and then
providing them via a command line arg [2].
We do not have to do it this way in Bazel to have reproducible
builds. This CL specifies an exact version (verified by sha256)
of those two deps and then uses a hermetic version of
Python 3.9 to run all py_binary commands.
Previously, we would rely on the system Python (and installed
libraries). That happened to work on my machine, but not on
other machines without jinja2 and MarkupSafe installed. After
this CL, it should work on machines that do not have python
even installed.
I chose the same jinja2 version used by Dawn [3], which was
2.11.3. Then I chose the newest version of MarkupSafe that
was compatible with jinja2 (2.0.1).
If we have other python scripts that need external deps, we
should be able to specify them in the py_binary that needs
them and in requirements.txt. Then, the pip_install() step
in WORKSPACE.bazel will download them and make them available.
[1] https://dawn.googlesource.com/dawn.git/+/refs/heads/main/docs/dawn/overview.md
[2] https://dawn.googlesource.com/dawn.git/+/e45ff6a4b3c2f06dade68ec0f01ddc3bfd70c282/generator/generator_lib.gni#77
[3] ee69aa00ee
Change-Id: I3d0074f3003de179400e239e00107c34f35f4901
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524217
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
