fed97e8f40
A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
67 lines
3.7 KiB
Markdown
67 lines
3.7 KiB
Markdown
RBE configurations
|
|
==================
|
|
|
|
Some subdirectories of this folder are generated. For example, `gce_linux` was generated by running
|
|
`make generate_linux_config`. Those generated files describe the C++ and Java toolchain that are
|
|
in the RBE Docker image; these toolchains are required to run Bazel, but are **not** the toolchains
|
|
that we use to compile our code.
|
|
|
|
We build our own, bare-bones, Docker image to use on RBE. We intend to use a hermetic toolchain
|
|
(see //toolchain) that specifies everything necessary to compile and link Skia. Use of the
|
|
hermetic toolchain on and off RBE makes the build reproducible and consistent across machines,
|
|
and not require internet access (assuming the toolchain has been cached at least once). This setup
|
|
has the desirable property of not needing to change and upload RBE Docker images if we need to
|
|
change a small detail of our toolchain.
|
|
|
|
The only requirement we have of our Docker image (beyond the minimum requirements to run Bazel)
|
|
are that it have sufficient runtime libraries to run our toolchain. For example, this means that
|
|
the Linux RBE image has at least glibc 2.32, which is the current minimum requirement of the
|
|
Linux binaries in our toolchain. This is the same requirement of any developer who tries to
|
|
build Skia using Bazel locally.
|
|
|
|
Getting rbe_configs_gen
|
|
-----------------------
|
|
It is suggested to download a prebuilt binary from
|
|
[GitHub](https://github.com/bazelbuild/bazel-toolchains/releases/tag/v5.1.1) and put that onto
|
|
your PATH.
|
|
|
|
Creating/Updating the RBE image
|
|
-------------------------------
|
|
In accordance with SLSA level 1, we want to be able to have a scripted way of building our image
|
|
and specify exactly what artifacts are in it. To accommodate this, we specify the exact sha256
|
|
hash of the base Docker image we use and the exact versions of the packages we install on top of
|
|
that. If we need to add a package or update things, it is best build the image without these
|
|
qualifiers to see what was actually used, and then respecify them so if someone were to build
|
|
the docker image again, they are likely to get the same image.
|
|
|
|
This process is:
|
|
1) Modify the appropriate Dockerfile (e.g. gce_linux_container/Dockerfile) to not have the
|
|
version or hash qualifiers. Also increment the appropriate VERSION variable in `Makefile`.
|
|
2) Add any new packages or make any changes.
|
|
3) Run `make build_linux_container` to build the image locally. One may verify it works by running
|
|
something like `docker run -it gcr.io/skia-public/rbe_linux:v2 /bin/bash`.
|
|
4) Note the versions and base image hash that were used. Modify the Dockerfile to use these.
|
|
5) Run `make push_linux_container` to rebuild the container and push it to GCS where it can
|
|
be used by our RBE workers. Note the sha256 hash of this created container
|
|
6) Modify the appropriate generate step in `Makefile` (e.g. `generate_linux_config`) to refer
|
|
to the correct toolchain_container. Then, run that step.
|
|
7) Modify the RBE platform in `./BUILD.bazel` to refer to the new `container_image`.
|
|
|
|
We chose not to use Bazel rules for this container step, as that could be difficult to bootstrap
|
|
without Bazel already setup. Additionally, Make is a simple and sufficient way to script the steps
|
|
for SLSA purposes.
|
|
|
|
Defining our own Bazel RBE platforms
|
|
------------------------------------
|
|
While the generated files *do* have a platform we can use (e.g.
|
|
`//bazel/rbe/gce_linux/config:platform`), we do not use it because we cannot easily customize it
|
|
without a risk that the changes will be lost when we update the image. Thankfully, we can specify
|
|
our own platforms, which we do in `./BUILD.bazel`.
|
|
|
|
More details
|
|
------------
|
|
https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit
|
|
|
|
RBE Metrics
|
|
-----------
|
|
http://go/skia-rbe-metrics |