AuroraMiddleware/skia2
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
cd692a6bb0
skia2/.bazelrc

102 lines
5.1 KiB
Plaintext
Raw Normal View History

[bazel] Sketch out changes for Mac toolchain Change-Id: Idae84d8d9538012e5cfb75a1a477dbc72a4da5bc Bug: skia:13125 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/526264 Reviewed-by: Jorge Betancourt <jmbetancourt@google.com>
2022-04-01 12:29:11 +00:00
# All of our clang builds should use the hermetic toolchain
[infra] Add hermetic toolchain for C/C++ using Clang+Musl This can successfully build a C library: bazel build --config=clang //third_party:libpng This can build and run a statically-linked executable: bazel test --config=clang //:bazel_test For more verbose compile and linking output, add the `--features diagnostic` flag to a Bazel command (see _make_diagnostic_flags() in toolchain/clang_toolchain_config.bzl. Similarly, a `--features print_search_dirs` can be used to show where clang is looking for libraries etc to link against. These features are made available for easier debugging. Suggested review order: - Read https://docs.bazel.build/versions/4.2.1/tutorial/cc-toolchain-config.html if unfamiliar with setting up C++ toolchains in Bazel - .bazelrc and WORKSPACE.bazel that configure use and download of the toolchain (Clang 13, musl 1.2.2) - toolchain/build_toolchain.bzl which downloads and assembles the toolchain (w/o installing anything on the host machine) - toolchain/BUILD.bazel and toolchain/*trampoline.sh to see the setup of the toolchain rules. - toolchain/clang_toolchain_config.bzl to see the configuration of the toolchain. Pay special attention to the various command line flags that are set. - See that tools/bazel_test.cc has made a new home in experimental/bazel_test/bazel_test.cpp, with a companion BUILD.bazel. Note the addition of some function calls that test use of the C++ standard library. The number being used to test the PNG library is the latest and greatest that verifies we are compiling the one brought in via DEPS (and not a local one). - third_party/* to see how png (and its dependent zlib) have been built. Pay special attention to the musl_compat hack to fix static linking (any idea what the real cause is?) - //BUILD.bazel to see definition of the bazel_test executable. Change-Id: I7b0922d0d45cb9be8df2fd5fa5a1f48492654d5f Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/461178 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-10-20 20:20:42 +00:00
build:clang --crosstool_top=//toolchain:clang_suite
[bazel] Sketch out changes for Mac toolchain Change-Id: Idae84d8d9538012e5cfb75a1a477dbc72a4da5bc Bug: skia:13125 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/526264 Reviewed-by: Jorge Betancourt <jmbetancourt@google.com>
2022-04-01 12:29:11 +00:00
# We don't have a great way to test the host system in order to use the right
# version of the toolchain. So we need to have the user specify what their host is.
build:clang_linux --config=clang
build:clang_linux --compiler=host_is_linux_amd64
build:clang_mac --config=clang
build:clang_mac --compiler=host_is_mac_M1
[infra] Add initial Bazel rules and files These rules can be used to build our GMs on WASM+WebGL and libskia.a with just the CPU backend (and most other features turned off). This can be done with the following commands: - bazel build //modules/canvaskit:gm-bindings-wasm --gpu_backend=gl_backend --with_gl_standard=webgl_standard - bazel build :skia-core --config clang This pivots slightly from http://review.skia.org/463517 by using config_settings [1] instead of platforms for the optional features that we control. This pivot was suggested in [2] We have BUILD.bazel files in many of the subdirectories that specify filegroups for the appropriate files. In an effort to make //BUILD.bazel more readable, it is the responsibility of these subfolders to deal with conditionally including certain .h or .cpp files. This is done using select statements and config_settings or platform constraints as necessary. For example, src/gpu/BUILD.bazel will different private filegroups for each of the supported gpu backends [3] and a more-visible filegroup called "srcs" that has the right selection of the private files to be used for compilation. An effort has been made to avoid using glob() in our BUILD.bazel files. These file lists were made by using `ls -1` and some regex to add in quotes. We might want to make a helper script to assist with that, if necessary. To specify which options we have, the settings in //bazel/common_config_settings/BUILD.bazel have been redesigned. They make use of a macro `string_flag_with_values` that removes the boilerplate. Patchset 36 shows what the file looks like w/o the macro. The top level BUILD.bazel file will still need to use some logic to handle defines, because local_defines is a list of strings, not a list of labels [4]. Suggested Review Order: - WORKSPACE.bazel to see the new dependencies on the emsdk toolchain and bazel_skylib - bazel/common_config_settings/* to see the few settings defined (we have more to define, see BUILD.gn and //gn/skia.gni for ideas) - BUILD.bazel to see the "skia-core" cc_library rule. See also "gms" and "tests" - modules/canvaskit/BUILD.bazel to see the use of the emscripten "wasm_cc_binary" rule, which depends on the "skia-core", "gms", and "tests" rule. Note that it only builds some of the gms as a proof of concept. - The other BUILD.bazel files. Some of these are not platform or feature dependent (e.g. pathops). Others are (e.g. gpu). - All other files. [1] https://docs.bazel.build/versions/4.2.1/skylark/config.html#user-defined-build-settings [2] https://github.com/emscripten-core/emsdk/pull/920 [3] In this CL, that's just the webgl one. [4] https://docs.bazel.build/versions/main/be/c-cpp.html#cc_library.local_defines Change-Id: Ieecf9c106d5e3a6ae97d13d66be06b4b3c207089 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/458637 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Owners-Override: Kevin Lubick <kjlubick@google.com>
2021-11-08 20:26:09 +00:00
# =============================================================================
# Alias to build configurations below. This makes configuring things from
# the command line easier.
[canvaskit] Add Freetype/Fonts to Bazel Build This re-works src/ports/BUILD.bazel to work like our other BUILD files, i.e. one rule "srcs" that brings in the necessary private filegroups. To work around an abort with LLVM [1], we have to go back to an earlier version of emscripten (temporarily?). Future work should look at using transitions [2] to allow various executables (e.g. CanvasKit, DM) to set their own set of Bazel flags, w/o the build invokers having to specify them. These transitions might be able to handle more complex cases that we currently use if statements in GN to deal with. The Freetype build rule was created by taking the BUILD.gn rule, adding in all the sources listed there and then playing compile-whack-a-mole to add in all the headers and included .c files. Suggested Review Order: - third_party/BUILD.bazel to see freetype build rules - bazel/common_config_settings/ to see treatment of fontmgr like codecs (many possible) and fontmgr_factory (only one). - src/ports/BUILD.bazel - BUILD.bazel - modules/canvaskit/BUILD.bazel. Take note of the gen_rule that calls tools/embed_resources.py to produce the .cpp file containing the embedded font data. - Everything else. [1] https://github.com/emscripten-core/emscripten/issues/15528 [2] https://github.com/bazelbuild/examples/tree/main/rules/starlark_configurations/cc_binary_selectable_copts Bug: skia:12541 Change-Id: I08dab82a901d80507007b354ca20cbfad2c2388f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/471636 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-16 19:09:44 +00:00
build --flag_alias=fontmgr_factory=//bazel/common_config_settings:fontmgr_factory
[infra] Add initial Bazel rules and files These rules can be used to build our GMs on WASM+WebGL and libskia.a with just the CPU backend (and most other features turned off). This can be done with the following commands: - bazel build //modules/canvaskit:gm-bindings-wasm --gpu_backend=gl_backend --with_gl_standard=webgl_standard - bazel build :skia-core --config clang This pivots slightly from http://review.skia.org/463517 by using config_settings [1] instead of platforms for the optional features that we control. This pivot was suggested in [2] We have BUILD.bazel files in many of the subdirectories that specify filegroups for the appropriate files. In an effort to make //BUILD.bazel more readable, it is the responsibility of these subfolders to deal with conditionally including certain .h or .cpp files. This is done using select statements and config_settings or platform constraints as necessary. For example, src/gpu/BUILD.bazel will different private filegroups for each of the supported gpu backends [3] and a more-visible filegroup called "srcs" that has the right selection of the private files to be used for compilation. An effort has been made to avoid using glob() in our BUILD.bazel files. These file lists were made by using `ls -1` and some regex to add in quotes. We might want to make a helper script to assist with that, if necessary. To specify which options we have, the settings in //bazel/common_config_settings/BUILD.bazel have been redesigned. They make use of a macro `string_flag_with_values` that removes the boilerplate. Patchset 36 shows what the file looks like w/o the macro. The top level BUILD.bazel file will still need to use some logic to handle defines, because local_defines is a list of strings, not a list of labels [4]. Suggested Review Order: - WORKSPACE.bazel to see the new dependencies on the emsdk toolchain and bazel_skylib - bazel/common_config_settings/* to see the few settings defined (we have more to define, see BUILD.gn and //gn/skia.gni for ideas) - BUILD.bazel to see the "skia-core" cc_library rule. See also "gms" and "tests" - modules/canvaskit/BUILD.bazel to see the use of the emscripten "wasm_cc_binary" rule, which depends on the "skia-core", "gms", and "tests" rule. Note that it only builds some of the gms as a proof of concept. - The other BUILD.bazel files. Some of these are not platform or feature dependent (e.g. pathops). Others are (e.g. gpu). - All other files. [1] https://docs.bazel.build/versions/4.2.1/skylark/config.html#user-defined-build-settings [2] https://github.com/emscripten-core/emsdk/pull/920 [3] In this CL, that's just the webgl one. [4] https://docs.bazel.build/versions/main/be/c-cpp.html#cc_library.local_defines Change-Id: Ieecf9c106d5e3a6ae97d13d66be06b4b3c207089 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/458637 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Owners-Override: Kevin Lubick <kjlubick@google.com>
2021-11-08 20:26:09 +00:00
build --flag_alias=gpu_backend=//bazel/common_config_settings:gpu_backend
[canvaskit] Expand Bazel rules to include Canvas2D compat layer Importantly, this adds options for encoding using certain codecs, not just decoding. Change-Id: I4a610ebf985b67d4545c71b3f3eed4c7807e6a26 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/472277 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-17 13:05:57 +00:00
build --flag_alias=include_decoder=//bazel/common_config_settings:include_decoder
build --flag_alias=include_encoder=//bazel/common_config_settings:include_encoder
build --flag_alias=include_fontmgr=//bazel/common_config_settings:include_fontmgr
[infra] Add SkParagraph (harfbuzz, ICU) to Canvaskit Bazel build. As a follow-up to https://skia-review.googlesource.com/c/skia/+/476219, this sketches out how we can maybe use cc_library for the things in //modules to make sure something in //src doesn't depend on anything in //modules, for example. The following succeeds: bazel build //modules/skparagraph:skparagraph --config=clang \ --shaper_backend=harfbuzz_shaper --with_icu As does `make bazel_canvaskit_debug` in //modules/canvaskit Suggested Review Order: - third_party/BUILD.bazel for ICU and harfbuzz rules. Pay special attention to the genrules used to call the python script for turning the icu .dat file into .S or .cpp. - bazelrc and bazel/ for new flags and defines that control use of ICU and harfbuzz. Unlike GN, with the public_defines that get added in automatically if icu or harfbuzz is depended upon, we need to set the defines at the top level. This necessity might go away if we change the atoms to depend on //modules/skshaper, which could define that flag. - Top level BUILD.bazel files in //modules/skparagraph, //modules/skshaper, //modules/skunicode, //modules/canvaskit - All other .bazel file changes are automatic. Bug: skia:12541 Change-Id: I38a9e0a9261d7e142eeb271c2ddb23f362f91473 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/478116 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-30 20:59:32 +00:00
build --flag_alias=shaper_backend=//bazel/common_config_settings:shaper_backend
[infra] Add initial Bazel rules and files These rules can be used to build our GMs on WASM+WebGL and libskia.a with just the CPU backend (and most other features turned off). This can be done with the following commands: - bazel build //modules/canvaskit:gm-bindings-wasm --gpu_backend=gl_backend --with_gl_standard=webgl_standard - bazel build :skia-core --config clang This pivots slightly from http://review.skia.org/463517 by using config_settings [1] instead of platforms for the optional features that we control. This pivot was suggested in [2] We have BUILD.bazel files in many of the subdirectories that specify filegroups for the appropriate files. In an effort to make //BUILD.bazel more readable, it is the responsibility of these subfolders to deal with conditionally including certain .h or .cpp files. This is done using select statements and config_settings or platform constraints as necessary. For example, src/gpu/BUILD.bazel will different private filegroups for each of the supported gpu backends [3] and a more-visible filegroup called "srcs" that has the right selection of the private files to be used for compilation. An effort has been made to avoid using glob() in our BUILD.bazel files. These file lists were made by using `ls -1` and some regex to add in quotes. We might want to make a helper script to assist with that, if necessary. To specify which options we have, the settings in //bazel/common_config_settings/BUILD.bazel have been redesigned. They make use of a macro `string_flag_with_values` that removes the boilerplate. Patchset 36 shows what the file looks like w/o the macro. The top level BUILD.bazel file will still need to use some logic to handle defines, because local_defines is a list of strings, not a list of labels [4]. Suggested Review Order: - WORKSPACE.bazel to see the new dependencies on the emsdk toolchain and bazel_skylib - bazel/common_config_settings/* to see the few settings defined (we have more to define, see BUILD.gn and //gn/skia.gni for ideas) - BUILD.bazel to see the "skia-core" cc_library rule. See also "gms" and "tests" - modules/canvaskit/BUILD.bazel to see the use of the emscripten "wasm_cc_binary" rule, which depends on the "skia-core", "gms", and "tests" rule. Note that it only builds some of the gms as a proof of concept. - The other BUILD.bazel files. Some of these are not platform or feature dependent (e.g. pathops). Others are (e.g. gpu). - All other files. [1] https://docs.bazel.build/versions/4.2.1/skylark/config.html#user-defined-build-settings [2] https://github.com/emscripten-core/emsdk/pull/920 [3] In this CL, that's just the webgl one. [4] https://docs.bazel.build/versions/main/be/c-cpp.html#cc_library.local_defines Change-Id: Ieecf9c106d5e3a6ae97d13d66be06b4b3c207089 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/458637 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Owners-Override: Kevin Lubick <kjlubick@google.com>
2021-11-08 20:26:09 +00:00
build --flag_alias=with_gl_standard=//bazel/common_config_settings:with_gl_standard
[canvaskit] Add Freetype/Fonts to Bazel Build This re-works src/ports/BUILD.bazel to work like our other BUILD files, i.e. one rule "srcs" that brings in the necessary private filegroups. To work around an abort with LLVM [1], we have to go back to an earlier version of emscripten (temporarily?). Future work should look at using transitions [2] to allow various executables (e.g. CanvasKit, DM) to set their own set of Bazel flags, w/o the build invokers having to specify them. These transitions might be able to handle more complex cases that we currently use if statements in GN to deal with. The Freetype build rule was created by taking the BUILD.gn rule, adding in all the sources listed there and then playing compile-whack-a-mole to add in all the headers and included .c files. Suggested Review Order: - third_party/BUILD.bazel to see freetype build rules - bazel/common_config_settings/ to see treatment of fontmgr like codecs (many possible) and fontmgr_factory (only one). - src/ports/BUILD.bazel - BUILD.bazel - modules/canvaskit/BUILD.bazel. Take note of the gen_rule that calls tools/embed_resources.py to produce the .cpp file containing the embedded font data. - Everything else. [1] https://github.com/emscripten-core/emscripten/issues/15528 [2] https://github.com/bazelbuild/examples/tree/main/rules/starlark_configurations/cc_binary_selectable_copts Bug: skia:12541 Change-Id: I08dab82a901d80507007b354ca20cbfad2c2388f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/471636 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-16 19:09:44 +00:00
[bazel] Unify boolean flags to be enable* I think they read better when it is enable_foo. I have to do less double-negation in my head if they are set to false then. Change-Id: I53bc5575c11d489029904965d991abc5a9d0fd89 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/528838 Reviewed-by: Ben Wagner <bungeman@google.com>
2022-04-08 18:23:38 +00:00
build --flag_alias=with_icu=//bazel/common_config_settings:use_icu
build --flag_alias=with_no_icu=no//bazel/common_config_settings:use_icu
Revert "[bazel] Unify boolean flags to be enable*" This reverts commit 02986aed2a4e9ddff37897ba82c1d715a66d8de7. Reason for revert: Android roll Original change's description: > [bazel] Unify boolean flags to be enable* > > I think they read better when it is enable_foo. I have to > do less double-negation in my head if they are set to false > then. > > Change-Id: I53bc5575c11d489029904965d991abc5a9d0fd89 > Bug: skia:12541 > Reviewed-on: https://skia-review.googlesource.com/c/skia/+/528838 > Reviewed-by: Ben Wagner <bungeman@google.com> Bug: skia:12541 Change-Id: I63f827c41ddc8a149cd190179ad099ff8671579a No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://skia-review.googlesource.com/c/skia/+/529128 Auto-Submit: John Stiles <johnstiles@google.com> Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2022-04-11 17:40:47 +00:00
build --flag_alias=disable_tracing=//bazel/common_config_settings:disable_tracing
build --flag_alias=enable_tracing=no//bazel/common_config_settings:disable_tracing
build --flag_alias=disable_effect_serialization=//bazel/common_config_settings:disable_effect_serialization
build --flag_alias=enable_effect_serialization=no//bazel/common_config_settings:disable_effect_serialization
build --flag_alias=enable_vma=//bazel/common_config_settings:use_vulkan_memory_allocator
build --flag_alias=disable_vma=no//bazel/common_config_settings:use_vulkan_memory_allocator
build --flag_alias=enable_skslc=//bazel/common_config_settings:enable_skslc
build --flag_alias=disable_skslc=no//bazel/common_config_settings:enable_skslc
Revert "[sksl] Make sksl tracing optional" This reverts commit 6bc4bdf6459d1c232744591b8b69896c7abda1ed. Reason for revert: Android roll Original change's description: > [sksl] Make sksl tracing optional > > This removes the required dependency on our JSON code. In the Bazel > rules, this dependency is pushed down into sksl instead of required > by the cc_binary rules. > > It adds a stub version of SkVMDebugTrace.cpp and removes > SkVMDebugTracePlayer unless the appropriate GN or Bazel flag > is set (skia_enable_sksl_tracing and enable_sksl_tracing, > respectively). There was an existing #define that CanvasKit > used (CK_INCLUDE_SKSL_TRACE) and this was changed to > SKSL_ENABLE_TRACING. > > Users of //:skia_core no longer need to specify a JSON dep, > if sksl needs it (e.g. for tracing), then it will specify > the dependency. > > Change-Id: I2fcd29cde118fc391c269ba2d8f8a40a6f164c99 > Bug: skia:12541 > Reviewed-on: https://skia-review.googlesource.com/c/skia/+/528837 > Reviewed-by: John Stiles <johnstiles@google.com> > Commit-Queue: Kevin Lubick <kjlubick@google.com> Bug: skia:12541 Change-Id: Icf75495f19e409d96925ca4dca9e839eca4057ec No-Presubmit: true No-Tree-Checks: true No-Try: true Reviewed-on: https://skia-review.googlesource.com/c/skia/+/529129 Auto-Submit: John Stiles <johnstiles@google.com> Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2022-04-11 17:41:47 +00:00
[canvaskit] Add Freetype/Fonts to Bazel Build This re-works src/ports/BUILD.bazel to work like our other BUILD files, i.e. one rule "srcs" that brings in the necessary private filegroups. To work around an abort with LLVM [1], we have to go back to an earlier version of emscripten (temporarily?). Future work should look at using transitions [2] to allow various executables (e.g. CanvasKit, DM) to set their own set of Bazel flags, w/o the build invokers having to specify them. These transitions might be able to handle more complex cases that we currently use if statements in GN to deal with. The Freetype build rule was created by taking the BUILD.gn rule, adding in all the sources listed there and then playing compile-whack-a-mole to add in all the headers and included .c files. Suggested Review Order: - third_party/BUILD.bazel to see freetype build rules - bazel/common_config_settings/ to see treatment of fontmgr like codecs (many possible) and fontmgr_factory (only one). - src/ports/BUILD.bazel - BUILD.bazel - modules/canvaskit/BUILD.bazel. Take note of the gen_rule that calls tools/embed_resources.py to produce the .cpp file containing the embedded font data. - Everything else. [1] https://github.com/emscripten-core/emscripten/issues/15528 [2] https://github.com/bazelbuild/examples/tree/main/rules/starlark_configurations/cc_binary_selectable_copts Bug: skia:12541 Change-Id: I08dab82a901d80507007b354ca20cbfad2c2388f Reviewed-on: https://skia-review.googlesource.com/c/skia/+/471636 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2021-11-16 19:09:44 +00:00
# CanvasKit flags
build --flag_alias=ck_enable_fonts=//modules/canvaskit:enable_fonts
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
build --flag_alias=ck_disable_fonts=no//modules/canvaskit:enable_fonts
# =============================================================================
# REMOTE BUILD EXECUTION
# =============================================================================
# =====
# The following was copied from https://github.com/bazelbuild/bazel-toolchains/blob/ea243d43269df23de03a797cff2347e1fc3d02bb/bazelrc/bazel-4.1.0.bazelrc
# We should be free to modify this as we see fit.
[bazel] Add CI job that uses RBE to enforce includes The toolchain now uses extract_ar from https://skia-review.googlesource.com/c/buildbot/+/524764 which is a static executable to extract the .deb files. This was necessary because the llvm-ar that had previously been used requires glibc 2.31+ to run, but our Debian10 machines on Swarming have an older version (2.28). A longer-term fix is to have Bazel support .ar files, which I plan to attempt to contribute this week. The RBE task will be added as an experimental CQ job, to see how it handles the load of running often. With the remote execution cache, I hope it performs well, once the toolchains are cached on both the Swarming machines and in the RBE workers. Note: We had to add several files to the CAS spec (see compile_cas.go) which are required for Bazel to work. Change-Id: Ie70c70d5f33768c957760f9eeb7835025109b487 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524759 Reviewed-by: Joe Gregorio <jcgregorio@google.com>
2022-03-28 13:45:26 +00:00
#
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
# Depending on how many machines are in the remote execution instance, setting
# this higher can make builds faster by allowing more jobs to run in parallel.
# Setting it too high can result in jobs that timeout, however, while waiting
# for a remote machine to execute them.
build:remote --jobs=50
# Set several flags related to specifying the platform, toolchain and java
# properties.
build:remote --java_runtime_version=rbe_jdk
build:remote --tool_java_runtime_version=rbe_jdk
build:remote --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
# Platform flags:
# The toolchain container used for execution is defined in the target indicated
# by "extra_execution_platforms", "host_platform" and "platforms".
# More about platforms: https://docs.bazel.build/versions/master/platforms.html
# Starting with Bazel 0.27.0 strategies do not need to be explicitly
# defined. See https://github.com/bazelbuild/bazel/issues/7480
build:remote --define=EXECUTOR=remote
# Enable remote execution so actions are performed on the remote systems.
build:remote --remote_executor=grpcs://remotebuildexecution.googleapis.com
# Enforce stricter environment rules, which eliminates some non-hermetic
# behavior and therefore improves both the remote cache hit rate and the
# correctness and repeatability of the build.
build:remote --incompatible_strict_action_env=true
Update Skia to use the new combined Dawn+Tint repo Includes a manual roll of Dawn, as well as tweaks to the Tint build overrides. Dawn needs git in order to generate a header that has the version of Dawn compiled in. If we use Dawn's GN rules to build it, we need git to generate that file. Our (now updated) Bazel rules to build Dawn do not need to do this because we (Skia) do not use this file [1] [1] https://dawn.googlesource.com/dawn/+/088a600b03679cd20991f145173a573ed9c03480/src/dawn/common/BUILD.gn#177 Change-Id: Id19fdbb9b53fdad2397a0044a4a314b1444faeb0 Cq-Include-Trybots: luci.skia.skia.primary:Build-Debian10-Clang-x86_64-Debug-Dawn,Build-Win-Clang-x86_64-Release-Dawn Reviewed-on: https://skia-review.googlesource.com/c/skia/+/528157 Reviewed-by: Kevin Lubick <kjlubick@google.com> Reviewed-by: Brian Osman <brianosman@google.com> Commit-Queue: Brian Osman <brianosman@google.com>
2022-04-07 14:31:59 +00:00
# No compile task should take more than 180 seconds. Really long running tasks
# are probably a result of spinning up new workers.
build:remote --remote_timeout=180
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
# Enable authentication. This will pick up application default credentials by
# default. You can use --google_credentials=some_file.json to use a service
# account credential instead.
# See https://developers.google.com/remote-build-execution/docs/authentication
build:remote --google_default_credentials=true
# End of the copied RBE settings
#=====
# Use the RBE instance on the skia-rbe GCP project.
build:remote --remote_instance_name projects/skia-rbe/instances/default_instance
# These settings are specific to compiling on our Linux RBE workers. For example,
# Use the worker pool as specified by the gce_linux_platform platform in
# //bazel/rbe/BUILD.bazel.
build:linux-rbe --config=remote
# Use our hermetic toolchain instead of the clang in the toolchain.
build:linux-rbe --crosstool_top=//toolchain:clang_suite
[bazel] Use RBE when building task drivers Local measurements show that using RBE (with a warm cache) vs local can result in a 2x faster build. No-Try: true Change-Id: Ib900a90564f105de848c9aeb0b745e5fec77da53 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/525637 Reviewed-by: Joe Gregorio <jcgregorio@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com>
2022-03-29 15:49:27 +00:00
# We want to run on this RBE platform
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
build:linux-rbe --extra_execution_platforms=//bazel/rbe:gce_linux_platform
[bazel] Use RBE when building task drivers Local measurements show that using RBE (with a warm cache) vs local can result in a 2x faster build. No-Try: true Change-Id: Ib900a90564f105de848c9aeb0b745e5fec77da53 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/525637 Reviewed-by: Joe Gregorio <jcgregorio@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com>
2022-03-29 15:49:27 +00:00
# On the RBE instances, this Java and C++ toolchain are available
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
build:linux-rbe --extra_toolchains=//bazel/rbe/gce_linux/java:all
build:linux-rbe --extra_toolchains=//bazel/rbe/gce_linux/config:cc-toolchain
Reference in New Issue Copy Permalink