AuroraMiddleware/skia2
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
421cff8320
skia2/.bazelrc

92 lines
4.5 KiB
Plaintext
Raw Normal View History

[infra] Add hermetic toolchain for C/C++ using Clang+Musl This can successfully build a C library: bazel build --config=clang //third_party:libpng This can build and run a statically-linked executable: bazel test --config=clang //:bazel_test For more verbose compile and linking output, add the `--features diagnostic` flag to a Bazel command (see _make_diagnostic_flags() in toolchain/clang_toolchain_config.bzl. Similarly, a `--features print_search_dirs` can be used to show where clang is looking for libraries etc to link against. These features are made available for easier debugging. Suggested review order: - Read https://docs.bazel.build/versions/4.2.1/tutorial/cc-toolchain-config.html if unfamiliar with setting up C++ toolchains in Bazel - .bazelrc and WORKSPACE.bazel that configure use and download of the toolchain (Clang 13, musl 1.2.2) - toolchain/build_toolchain.bzl which downloads and assembles the toolchain (w/o installing anything on the host machine) - toolchain/BUILD.bazel and toolchain/*trampoline.sh to see the setup of the toolchain rules. - toolchain/clang_toolchain_config.bzl to see the configuration of the toolchain. Pay special attention to the various command line flags that are set. - See that tools/bazel_test.cc has made a new home in experimental/bazel_test/bazel_test.cpp, with a companion BUILD.bazel. Note the addition of some function calls that test use of the C++ standard library. The number being used to test the PNG library is the latest and greatest that verifies we are compiling the one brought in via DEPS (and not a local one). - third_party/* to see how png (and its dependent zlib) have been built. Pay special attention to the musl_compat hack to fix static linking (any idea what the real cause is?) - //BUILD.bazel to see definition of the bazel_test executable. Change-Id: I7b0922d0d45cb9be8df2fd5fa5a1f48492654d5f Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/461178 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-10-20 20:20:42 +00:00
build:clang --crosstool_top=//toolchain:clang_suite
build:clang --compiler=clang
[infra] Add initial Bazel rules and files These rules can be used to build our GMs on WASM+WebGL and libskia.a with just the CPU backend (and most other features turned off). This can be done with the following commands: - bazel build //modules/canvaskit:gm-bindings-wasm --gpu_backend=gl_backend --with_gl_standard=webgl_standard - bazel build :skia-core --config clang This pivots slightly from http://review.skia.org/463517 by using config_settings [1] instead of platforms for the optional features that we control. This pivot was suggested in [2] We have BUILD.bazel files in many of the subdirectories that specify filegroups for the appropriate files. In an effort to make //BUILD.bazel more readable, it is the responsibility of these subfolders to deal with conditionally including certain .h or .cpp files. This is done using select statements and config_settings or platform constraints as necessary. For example, src/gpu/BUILD.bazel will different private filegroups for each of the supported gpu backends [3] and a more-visible filegroup called "srcs" that has the right selection of the private files to be used for compilation. An effort has been made to avoid using glob() in our BUILD.bazel files. These file lists were made by using `ls -1` and some regex to add in quotes. We might want to make a helper script to assist with that, if necessary. To specify which options we have, the settings in //bazel/common_config_settings/BUILD.bazel have been redesigned. They make use of a macro `string_flag_with_values` that removes the boilerplate. Patchset 36 shows what the file looks like w/o the macro. The top level BUILD.bazel file will still need to use some logic to handle defines, because local_defines is a list of strings, not a list of labels [4]. Suggested Review Order: - WORKSPACE.bazel to see the new dependencies on the emsdk toolchain and bazel_skylib - bazel/common_config_settings/* to see the few settings defined (we have more to define, see BUILD.gn and //gn/skia.gni for ideas) - BUILD.bazel to see the "skia-core" cc_library rule. See also "gms" and "tests" - modules/canvaskit/BUILD.bazel to see the use of the emscripten "wasm_cc_binary" rule, which depends on the "skia-core", "gms", and "tests" rule. Note that it only builds some of the gms as a proof of concept. - The other BUILD.bazel files. Some of these are not platform or feature dependent (e.g. pathops). Others are (e.g. gpu). - All other files. [1] https://docs.bazel.build/versions/4.2.1/skylark/config.html#user-defined-build-settings [2] https://github.com/emscripten-core/emsdk/pull/920 [3] In this CL, that's just the webgl one. [4] https://docs.bazel.build/versions/main/be/c-cpp.html#cc_library.local_defines Change-Id: Ieecf9c106d5e3a6ae97d13d66be06b4b3c207089 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/458637 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Owners-Override: Kevin Lubick <kjlubick@google.com>
2021-11-08 20:26:09 +00:00
# Use the default Bazel C++ toolchain to build the tools used during the build.
build:clang --host_crosstool_top=@bazel_tools//tools/cpp:toolchain
# =============================================================================
# Alias to build configurations below. This makes configuring things from
# the command line easier.
[canvaskit] Add Freetype/Fonts to Bazel Build This re-works src/ports/BUILD.bazel to work like our other BUILD files, i.e. one rule "srcs" that brings in the necessary private filegroups. To work around an abort with LLVM [1], we have to go back to an earlier version of emscripten (temporarily?). Future work should look at using transitions [2] to allow various executables (e.g. CanvasKit, DM) to set their own set of Bazel flags, w/o the build invokers having to specify them. These transitions might be able to handle more complex cases that we currently use if statements in GN to deal with. The Freetype build rule was created by taking the BUILD.gn rule, adding in all the sources listed there and then playing compile-whack-a-mole to add in all the headers and included .c files. Suggested Review Order: - third_party/BUILD.bazel to see freetype build rules - bazel/common_config_settings/ to see treatment of fontmgr like codecs (many possible) and fontmgr_factory (only one). - src/ports/BUILD.bazel - BUILD.bazel - modules/canvaskit/BUILD.bazel. Take note of the gen_rule that calls tools/embed_resources.py to produce the .cpp file containing the embedded font data. - Everything else. [1] https://github.com/emscripten-core/emscripten/issues/15528 [2] https://github.com/bazelbuild/examples/tree/main/rules/starlark_configurations/cc_binary_selectable_copts Bug: skia:12541 Change-Id: I08dab82a901d80507007b354ca20cbfad2c2388f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/471636 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-16 19:09:44 +00:00
build --flag_alias=fontmgr_factory=//bazel/common_config_settings:fontmgr_factory
[infra] Add initial Bazel rules and files These rules can be used to build our GMs on WASM+WebGL and libskia.a with just the CPU backend (and most other features turned off). This can be done with the following commands: - bazel build //modules/canvaskit:gm-bindings-wasm --gpu_backend=gl_backend --with_gl_standard=webgl_standard - bazel build :skia-core --config clang This pivots slightly from http://review.skia.org/463517 by using config_settings [1] instead of platforms for the optional features that we control. This pivot was suggested in [2] We have BUILD.bazel files in many of the subdirectories that specify filegroups for the appropriate files. In an effort to make //BUILD.bazel more readable, it is the responsibility of these subfolders to deal with conditionally including certain .h or .cpp files. This is done using select statements and config_settings or platform constraints as necessary. For example, src/gpu/BUILD.bazel will different private filegroups for each of the supported gpu backends [3] and a more-visible filegroup called "srcs" that has the right selection of the private files to be used for compilation. An effort has been made to avoid using glob() in our BUILD.bazel files. These file lists were made by using `ls -1` and some regex to add in quotes. We might want to make a helper script to assist with that, if necessary. To specify which options we have, the settings in //bazel/common_config_settings/BUILD.bazel have been redesigned. They make use of a macro `string_flag_with_values` that removes the boilerplate. Patchset 36 shows what the file looks like w/o the macro. The top level BUILD.bazel file will still need to use some logic to handle defines, because local_defines is a list of strings, not a list of labels [4]. Suggested Review Order: - WORKSPACE.bazel to see the new dependencies on the emsdk toolchain and bazel_skylib - bazel/common_config_settings/* to see the few settings defined (we have more to define, see BUILD.gn and //gn/skia.gni for ideas) - BUILD.bazel to see the "skia-core" cc_library rule. See also "gms" and "tests" - modules/canvaskit/BUILD.bazel to see the use of the emscripten "wasm_cc_binary" rule, which depends on the "skia-core", "gms", and "tests" rule. Note that it only builds some of the gms as a proof of concept. - The other BUILD.bazel files. Some of these are not platform or feature dependent (e.g. pathops). Others are (e.g. gpu). - All other files. [1] https://docs.bazel.build/versions/4.2.1/skylark/config.html#user-defined-build-settings [2] https://github.com/emscripten-core/emsdk/pull/920 [3] In this CL, that's just the webgl one. [4] https://docs.bazel.build/versions/main/be/c-cpp.html#cc_library.local_defines Change-Id: Ieecf9c106d5e3a6ae97d13d66be06b4b3c207089 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/458637 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Owners-Override: Kevin Lubick <kjlubick@google.com>
2021-11-08 20:26:09 +00:00
build --flag_alias=gpu_backend=//bazel/common_config_settings:gpu_backend
[canvaskit] Expand Bazel rules to include Canvas2D compat layer Importantly, this adds options for encoding using certain codecs, not just decoding. Change-Id: I4a610ebf985b67d4545c71b3f3eed4c7807e6a26 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/472277 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-17 13:05:57 +00:00
build --flag_alias=include_decoder=//bazel/common_config_settings:include_decoder
build --flag_alias=include_encoder=//bazel/common_config_settings:include_encoder
build --flag_alias=include_fontmgr=//bazel/common_config_settings:include_fontmgr
[infra] Add SkParagraph (harfbuzz, ICU) to Canvaskit Bazel build. As a follow-up to https://skia-review.googlesource.com/c/skia/+/476219, this sketches out how we can maybe use cc_library for the things in //modules to make sure something in //src doesn't depend on anything in //modules, for example. The following succeeds: bazel build //modules/skparagraph:skparagraph --config=clang \ --shaper_backend=harfbuzz_shaper --with_icu As does `make bazel_canvaskit_debug` in //modules/canvaskit Suggested Review Order: - third_party/BUILD.bazel for ICU and harfbuzz rules. Pay special attention to the genrules used to call the python script for turning the icu .dat file into .S or .cpp. - bazelrc and bazel/ for new flags and defines that control use of ICU and harfbuzz. Unlike GN, with the public_defines that get added in automatically if icu or harfbuzz is depended upon, we need to set the defines at the top level. This necessity might go away if we change the atoms to depend on //modules/skshaper, which could define that flag. - Top level BUILD.bazel files in //modules/skparagraph, //modules/skshaper, //modules/skunicode, //modules/canvaskit - All other .bazel file changes are automatic. Bug: skia:12541 Change-Id: I38a9e0a9261d7e142eeb271c2ddb23f362f91473 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/478116 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-30 20:59:32 +00:00
build --flag_alias=shaper_backend=//bazel/common_config_settings:shaper_backend
[infra] Add initial Bazel rules and files These rules can be used to build our GMs on WASM+WebGL and libskia.a with just the CPU backend (and most other features turned off). This can be done with the following commands: - bazel build //modules/canvaskit:gm-bindings-wasm --gpu_backend=gl_backend --with_gl_standard=webgl_standard - bazel build :skia-core --config clang This pivots slightly from http://review.skia.org/463517 by using config_settings [1] instead of platforms for the optional features that we control. This pivot was suggested in [2] We have BUILD.bazel files in many of the subdirectories that specify filegroups for the appropriate files. In an effort to make //BUILD.bazel more readable, it is the responsibility of these subfolders to deal with conditionally including certain .h or .cpp files. This is done using select statements and config_settings or platform constraints as necessary. For example, src/gpu/BUILD.bazel will different private filegroups for each of the supported gpu backends [3] and a more-visible filegroup called "srcs" that has the right selection of the private files to be used for compilation. An effort has been made to avoid using glob() in our BUILD.bazel files. These file lists were made by using `ls -1` and some regex to add in quotes. We might want to make a helper script to assist with that, if necessary. To specify which options we have, the settings in //bazel/common_config_settings/BUILD.bazel have been redesigned. They make use of a macro `string_flag_with_values` that removes the boilerplate. Patchset 36 shows what the file looks like w/o the macro. The top level BUILD.bazel file will still need to use some logic to handle defines, because local_defines is a list of strings, not a list of labels [4]. Suggested Review Order: - WORKSPACE.bazel to see the new dependencies on the emsdk toolchain and bazel_skylib - bazel/common_config_settings/* to see the few settings defined (we have more to define, see BUILD.gn and //gn/skia.gni for ideas) - BUILD.bazel to see the "skia-core" cc_library rule. See also "gms" and "tests" - modules/canvaskit/BUILD.bazel to see the use of the emscripten "wasm_cc_binary" rule, which depends on the "skia-core", "gms", and "tests" rule. Note that it only builds some of the gms as a proof of concept. - The other BUILD.bazel files. Some of these are not platform or feature dependent (e.g. pathops). Others are (e.g. gpu). - All other files. [1] https://docs.bazel.build/versions/4.2.1/skylark/config.html#user-defined-build-settings [2] https://github.com/emscripten-core/emsdk/pull/920 [3] In this CL, that's just the webgl one. [4] https://docs.bazel.build/versions/main/be/c-cpp.html#cc_library.local_defines Change-Id: Ieecf9c106d5e3a6ae97d13d66be06b4b3c207089 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/458637 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Owners-Override: Kevin Lubick <kjlubick@google.com>
2021-11-08 20:26:09 +00:00
build --flag_alias=with_gl_standard=//bazel/common_config_settings:with_gl_standard
[canvaskit] Add Freetype/Fonts to Bazel Build This re-works src/ports/BUILD.bazel to work like our other BUILD files, i.e. one rule "srcs" that brings in the necessary private filegroups. To work around an abort with LLVM [1], we have to go back to an earlier version of emscripten (temporarily?). Future work should look at using transitions [2] to allow various executables (e.g. CanvasKit, DM) to set their own set of Bazel flags, w/o the build invokers having to specify them. These transitions might be able to handle more complex cases that we currently use if statements in GN to deal with. The Freetype build rule was created by taking the BUILD.gn rule, adding in all the sources listed there and then playing compile-whack-a-mole to add in all the headers and included .c files. Suggested Review Order: - third_party/BUILD.bazel to see freetype build rules - bazel/common_config_settings/ to see treatment of fontmgr like codecs (many possible) and fontmgr_factory (only one). - src/ports/BUILD.bazel - BUILD.bazel - modules/canvaskit/BUILD.bazel. Take note of the gen_rule that calls tools/embed_resources.py to produce the .cpp file containing the embedded font data. - Everything else. [1] https://github.com/emscripten-core/emscripten/issues/15528 [2] https://github.com/bazelbuild/examples/tree/main/rules/starlark_configurations/cc_binary_selectable_copts Bug: skia:12541 Change-Id: I08dab82a901d80507007b354ca20cbfad2c2388f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/471636 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-16 19:09:44 +00:00
[infra] Add SkParagraph (harfbuzz, ICU) to Canvaskit Bazel build. As a follow-up to https://skia-review.googlesource.com/c/skia/+/476219, this sketches out how we can maybe use cc_library for the things in //modules to make sure something in //src doesn't depend on anything in //modules, for example. The following succeeds: bazel build //modules/skparagraph:skparagraph --config=clang \ --shaper_backend=harfbuzz_shaper --with_icu As does `make bazel_canvaskit_debug` in //modules/canvaskit Suggested Review Order: - third_party/BUILD.bazel for ICU and harfbuzz rules. Pay special attention to the genrules used to call the python script for turning the icu .dat file into .S or .cpp. - bazelrc and bazel/ for new flags and defines that control use of ICU and harfbuzz. Unlike GN, with the public_defines that get added in automatically if icu or harfbuzz is depended upon, we need to set the defines at the top level. This necessity might go away if we change the atoms to depend on //modules/skshaper, which could define that flag. - Top level BUILD.bazel files in //modules/skparagraph, //modules/skshaper, //modules/skunicode, //modules/canvaskit - All other .bazel file changes are automatic. Bug: skia:12541 Change-Id: I38a9e0a9261d7e142eeb271c2ddb23f362f91473 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/478116 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-30 20:59:32 +00:00
build --flag_alias=with_icu=//bazel/common_config_settings:use_icu
build --flag_alias=with_no_icu=no//bazel/common_config_settings:use_icu
[canvaskit] Add more features to Bazel build PS 1 adds particles to the build PS 2+ ports many of the options from //modules/canvaskit/compile.sh With this CL, all the CanvasKit tests pass with both the debug and release build. Change-Id: Id70f0c16a087109c56949417f940849f2e3b5200 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/504537 Reviewed-by: Brian Osman <brianosman@google.com> Reviewed-by: Ben Wagner <bungeman@google.com>
2022-02-04 20:04:15 +00:00
build --flag_alias=disable_tracing=//bazel/common_config_settings:disable_tracing
build --flag_alias=enable_tracing=no//bazel/common_config_settings:disable_tracing
build --flag_alias=disable_effect_serialization=//bazel/common_config_settings:disable_effect_serialization
build --flag_alias=enable_effect_serialization=no//bazel/common_config_settings:disable_effect_serialization
[canvaskit] Add Freetype/Fonts to Bazel Build This re-works src/ports/BUILD.bazel to work like our other BUILD files, i.e. one rule "srcs" that brings in the necessary private filegroups. To work around an abort with LLVM [1], we have to go back to an earlier version of emscripten (temporarily?). Future work should look at using transitions [2] to allow various executables (e.g. CanvasKit, DM) to set their own set of Bazel flags, w/o the build invokers having to specify them. These transitions might be able to handle more complex cases that we currently use if statements in GN to deal with. The Freetype build rule was created by taking the BUILD.gn rule, adding in all the sources listed there and then playing compile-whack-a-mole to add in all the headers and included .c files. Suggested Review Order: - third_party/BUILD.bazel to see freetype build rules - bazel/common_config_settings/ to see treatment of fontmgr like codecs (many possible) and fontmgr_factory (only one). - src/ports/BUILD.bazel - BUILD.bazel - modules/canvaskit/BUILD.bazel. Take note of the gen_rule that calls tools/embed_resources.py to produce the .cpp file containing the embedded font data. - Everything else. [1] https://github.com/emscripten-core/emscripten/issues/15528 [2] https://github.com/bazelbuild/examples/tree/main/rules/starlark_configurations/cc_binary_selectable_copts Bug: skia:12541 Change-Id: I08dab82a901d80507007b354ca20cbfad2c2388f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/471636 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-16 19:09:44 +00:00
# CanvasKit flags
build --flag_alias=ck_enable_fonts=//modules/canvaskit:enable_fonts
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
build --flag_alias=ck_disable_fonts=no//modules/canvaskit:enable_fonts
# =============================================================================
# REMOTE BUILD EXECUTION
# =============================================================================
# =====
# The following was copied from https://github.com/bazelbuild/bazel-toolchains/blob/ea243d43269df23de03a797cff2347e1fc3d02bb/bazelrc/bazel-4.1.0.bazelrc
# We should be free to modify this as we see fit.
[bazel] Add CI job that uses RBE to enforce includes The toolchain now uses extract_ar from https://skia-review.googlesource.com/c/buildbot/+/524764 which is a static executable to extract the .deb files. This was necessary because the llvm-ar that had previously been used requires glibc 2.31+ to run, but our Debian10 machines on Swarming have an older version (2.28). A longer-term fix is to have Bazel support .ar files, which I plan to attempt to contribute this week. The RBE task will be added as an experimental CQ job, to see how it handles the load of running often. With the remote execution cache, I hope it performs well, once the toolchains are cached on both the Swarming machines and in the RBE workers. Note: We had to add several files to the CAS spec (see compile_cas.go) which are required for Bazel to work. Change-Id: Ie70c70d5f33768c957760f9eeb7835025109b487 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524759 Reviewed-by: Joe Gregorio <jcgregorio@google.com>
2022-03-28 13:45:26 +00:00
#
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
# Depending on how many machines are in the remote execution instance, setting
# this higher can make builds faster by allowing more jobs to run in parallel.
# Setting it too high can result in jobs that timeout, however, while waiting
# for a remote machine to execute them.
build:remote --jobs=50
# Set several flags related to specifying the platform, toolchain and java
# properties.
build:remote --java_runtime_version=rbe_jdk
build:remote --tool_java_runtime_version=rbe_jdk
build:remote --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
# Platform flags:
# The toolchain container used for execution is defined in the target indicated
# by "extra_execution_platforms", "host_platform" and "platforms".
# More about platforms: https://docs.bazel.build/versions/master/platforms.html
# Starting with Bazel 0.27.0 strategies do not need to be explicitly
# defined. See https://github.com/bazelbuild/bazel/issues/7480
build:remote --define=EXECUTOR=remote
# Enable remote execution so actions are performed on the remote systems.
build:remote --remote_executor=grpcs://remotebuildexecution.googleapis.com
# Enforce stricter environment rules, which eliminates some non-hermetic
# behavior and therefore improves both the remote cache hit rate and the
# correctness and repeatability of the build.
build:remote --incompatible_strict_action_env=true
# Set a higher timeout value, just in case.
build:remote --remote_timeout=3600
# Enable authentication. This will pick up application default credentials by
# default. You can use --google_credentials=some_file.json to use a service
# account credential instead.
# See https://developers.google.com/remote-build-execution/docs/authentication
build:remote --google_default_credentials=true
# End of the copied RBE settings
#=====
# Use the RBE instance on the skia-rbe GCP project.
build:remote --remote_instance_name projects/skia-rbe/instances/default_instance
# These settings are specific to compiling on our Linux RBE workers. For example,
# Use the worker pool as specified by the gce_linux_platform platform in
# //bazel/rbe/BUILD.bazel.
build:linux-rbe --config=remote
# Use our hermetic toolchain instead of the clang in the toolchain.
build:linux-rbe --crosstool_top=//toolchain:clang_suite
[bazel] Use RBE when building task drivers Local measurements show that using RBE (with a warm cache) vs local can result in a 2x faster build. No-Try: true Change-Id: Ib900a90564f105de848c9aeb0b745e5fec77da53 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/525637 Reviewed-by: Joe Gregorio <jcgregorio@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com>
2022-03-29 15:49:27 +00:00
# We want to run on this RBE platform
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
build:linux-rbe --extra_execution_platforms=//bazel/rbe:gce_linux_platform
[bazel] Use RBE when building task drivers Local measurements show that using RBE (with a warm cache) vs local can result in a 2x faster build. No-Try: true Change-Id: Ib900a90564f105de848c9aeb0b745e5fec77da53 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/525637 Reviewed-by: Joe Gregorio <jcgregorio@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com>
2022-03-29 15:49:27 +00:00
# On the RBE instances, this Java and C++ toolchain are available
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
build:linux-rbe --extra_toolchains=//bazel/rbe/gce_linux/java:all
build:linux-rbe --extra_toolchains=//bazel/rbe/gce_linux/config:cc-toolchain
Reference in New Issue Copy Permalink