skia2/bazel/Makefile

64 lines
3.4 KiB
Makefile
Raw Normal View History

[bazel] Sketching out HelloWorld sk_app using GL bazel run //example:hello_world --config=clang causes a window to open and draws a circle and a square. Text to follow in a future CL. To make this work, I had to get rid of musl and use glibc. All the shared libraries (.so files) that were pre-built and available for download (e.g. from https://packages.debian.org/bullseye/amd64/libgl1/download) were compiled against glibc. When I tried to run a program statically linked with musl and dynamically linked against things using glibc, I got a segmentation fault on things like calloc(). Initial attempts to use glibc had failed because it was thought that the libc.so.6 file could only be referred to by absolute path (and thus Bazel would not be happy about it). As it turns out, that was simply a misconfiguration of the builtin_sysroot parameter to cc_common.create_cc_toolchain_config_info (see //toolchain/clang_toolchain_config.bzl). By setting that to `external/clang_linux_amd64` and not `external/clang_linux_amd64/usr`, the libc binary which had been extracted to `external/clang_linux_amd64/lib/x86_64-linux-gnu` was perfectly reachable from `external/clang_linux_amd64/usr/usr/lib/x86_64-linux-gnu/libc.so` To bring in the shared libraries to link against (e.g. X11, GL) I made build_toolchain.bzl easier to modify in that we simply need to add a debian download url and sha256 hash to a list (rather than having to plumb this through via arguments). Recommended Review Order: - example/BUILD.bazel (not sure if we always want to set bare link arguments like that or if we want to use "features" to pass those along to the toolchain). - tools/sk_app/BUILD.bazel to see initial cc_library for wrapping sk_app code. - toolchain/build_toolchain.bzl to see removal of musl and new list of debs. - toolchain/clang_toolchain_config.bzl (where use of the no-canonical-prefixes was key to compilation success). Notice also that we statically linked libc++ (I did not have any shared libraries for it locally, so I guessed a typical developer might not either). - Rest of toolchain/ for trivial renames. - bazel/Makefile to see extra docs on those targets and a new target that compiles all the exes so far for a quick way to test the build. - third_party/BUILD.bazel and src/gpu/BUILD.bazel which have non-generated changes. (all other BUILD.bazel files do). - go.mod, which needed to update the infra repo version in order to pick up http://review.skia.org/491736). Change-Id: I8687bd227353040eca2dffa9465798d8bd395027 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/492117 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com>
2022-01-11 12:35:26 +00:00
# This target should be run when new files are added or includes changed.
generate:
bazelisk build @org_skia_go_infra//bazel/gazelle:gazelle_cpp
cd .. && bazel-bin/external/org_skia_go_infra/bazel/gazelle/gazelle_cpp_/gazelle_cpp update \
--third_party_file_map="third_party/file_map_for_bazel.json" \
[bazel] Compile gms for wasm and WebGL PS 1 is re-generating existing BUILD.bazel files PS 2 is generating BUILD.bazel files for tests/gms PS 3+ makes modifications to build all of the gms and tests. It is recommended to view this CL with just a diff between PS 2 and the end, due to the large amount of generated changes in PS 1 and 2. We make a filegroup for the gms and tests because they need to be compiled as one large blob in order for the registries to work. Maybe in the future we will break these up, but at least for WASM/JS, the overhead of starting a browser for each new test would likely grind things to a halt, so we just group them all together for now. It's also the most similar to what we currently do. In gm/BUILD.bazel and tests/BUILD.bazel, we add a cc_library that encapsulates all of the deps of the tests, so we can easily include that the build. These were discovered via trial and error, not anything automatic or systematic. The is_skia_dev_build config_setting is very similar to the GN equivalent from which it was based. The list of gms and tests to skip (e.g. which are incompatible with WASM) was determined by building the wasm bundle: modules/canvaskit$ make bazel_gms_release tools/run-wasm-gm-tests$ make run_local_debug # Don't forget to click the button on the screen after the # browser loads This way of invoking the tests will be replace soon with `bazel test <something>`. As such, I didn't bother fully documenting the current way. Suggested review order: - modules/canvaskit/BUILD.bazel taking note that we always use profiling-funcs to make the stacktraces human readable. - gm/BUILD.bazel and tests/BUILD.bazel to see the lists of gms/tests. Notice the tests are roughly partitioned because we don't support things like vulkan/PDF in the wasm build and we will want a way to not build certain tests for certain configurations - tools/* noting some of the cc_libraries added to make dependencies easier to add when needed. - All other files. Change-Id: I43059cd93c28af1c4c12b93d6ebd9c46a12d381f Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/506256 Reviewed-by: Ben Wagner <bungeman@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com>
2022-02-09 18:14:25 +00:00
include src tools gm tests \
modules/skshaper modules/svg modules/skresources modules/skparagraph modules/skunicode \
modules/skottie modules/skresources modules/sksg experimental/ffmpeg \
modules/particles \
experimental/bazel_test example \
modules/canvaskit/go \
infra/bots/task_drivers
[bazel] Sketching out HelloWorld sk_app using GL bazel run //example:hello_world --config=clang causes a window to open and draws a circle and a square. Text to follow in a future CL. To make this work, I had to get rid of musl and use glibc. All the shared libraries (.so files) that were pre-built and available for download (e.g. from https://packages.debian.org/bullseye/amd64/libgl1/download) were compiled against glibc. When I tried to run a program statically linked with musl and dynamically linked against things using glibc, I got a segmentation fault on things like calloc(). Initial attempts to use glibc had failed because it was thought that the libc.so.6 file could only be referred to by absolute path (and thus Bazel would not be happy about it). As it turns out, that was simply a misconfiguration of the builtin_sysroot parameter to cc_common.create_cc_toolchain_config_info (see //toolchain/clang_toolchain_config.bzl). By setting that to `external/clang_linux_amd64` and not `external/clang_linux_amd64/usr`, the libc binary which had been extracted to `external/clang_linux_amd64/lib/x86_64-linux-gnu` was perfectly reachable from `external/clang_linux_amd64/usr/usr/lib/x86_64-linux-gnu/libc.so` To bring in the shared libraries to link against (e.g. X11, GL) I made build_toolchain.bzl easier to modify in that we simply need to add a debian download url and sha256 hash to a list (rather than having to plumb this through via arguments). Recommended Review Order: - example/BUILD.bazel (not sure if we always want to set bare link arguments like that or if we want to use "features" to pass those along to the toolchain). - tools/sk_app/BUILD.bazel to see initial cc_library for wrapping sk_app code. - toolchain/build_toolchain.bzl to see removal of musl and new list of debs. - toolchain/clang_toolchain_config.bzl (where use of the no-canonical-prefixes was key to compilation success). Notice also that we statically linked libc++ (I did not have any shared libraries for it locally, so I guessed a typical developer might not either). - Rest of toolchain/ for trivial renames. - bazel/Makefile to see extra docs on those targets and a new target that compiles all the exes so far for a quick way to test the build. - third_party/BUILD.bazel and src/gpu/BUILD.bazel which have non-generated changes. (all other BUILD.bazel files do). - go.mod, which needed to update the infra repo version in order to pick up http://review.skia.org/491736). Change-Id: I8687bd227353040eca2dffa9465798d8bd395027 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/492117 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com>
2022-01-11 12:35:26 +00:00
# This target should be run after the go.mod file is updated (e.g. version rolls or new updates)
gazelle_update_repo:
cd .. && bazelisk run //:gazelle -- update-repos -from_file=go.mod -to_macro=go_repositories.bzl%go_repositories
# Run this target to test all known working Bazel builds
known_good_builds:
bazelisk build //experimental/bazel_test/... --config=clang
bazelisk run //experimental/bazel_test:bazel_test_exe --config=clang
bazelisk build //:skia_core --config=clang
bazelisk build //src/sksl/lex:sksllex --config=clang
bazelisk build //tools/skdiff --config=clang
bazelisk build //tools/skslc --config=clang
bazelisk build //modules/canvaskit:canvaskit_wasm --compilation_mode opt --sandbox_base=/dev/shm
# Test the enforcement of include what you use
bazelisk build //example:hello_world_gl --config=clang --features skia_enforce_iwyu
# Both with and without a GPU backend should be error free (i.e. IWYU should let us
# conditionally import things.
bazelisk build //src/svg/... --config=clang --features skia_enforce_iwyu \
--gpu_backend=gl_backend --include_decoder=jpeg_decode_codec
bazelisk build //src/svg/... --config=clang --features skia_enforce_iwyu
bazelisk build //tools/debugger --config=clang --gpu_backend=gl_backend \
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
--features skia_enforce_iwyu
bazelisk build //:skia_core --config=clang --features skia_enforce_iwyu
bazelisk build //src/utils/... --config=clang --features skia_enforce_iwyu
bazelisk build //src/utils/... --config=clang --gpu_backend=gl_backend \
--features skia_enforce_iwyu
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
rbe_known_good_builds:
bazelisk build //experimental/bazel_test/... --config=linux-rbe
bazelisk run //experimental/bazel_test:bazel_test_exe --config=linux-rbe
bazelisk build //:skia_core --config=linux-rbe
bazelisk build //src/sksl/lex:sksllex --config=linux-rbe
bazelisk build //tools/skdiff --config=linux-rbe
bazelisk build //tools/skslc --config=linux-rbe
# TODO(kjlubick) CanvasKit in release mode (i.e. with Closure) requires
# https://github.com/emscripten-core/emscripten/pull/16640 to land
bazelisk build //modules/canvaskit:canvaskit_wasm --compilation_mode dbg --config=linux-rbe \
--jobs 100
[bazel] Add RBE support using hermetic Linux Clang toolchain A new RBE worker-pool called gce_linux was created in conjunction with this CL. See https://docs.google.com/document/d/14xMZCKews69SSTfULhE8HDUzT5XvPwZ4CvRufEvcZ74/edit# for some details on that. Note: everything under bazel/rbe/gce_linux was autogenerated and can be ignored from manual review. It basically specifies what files are on the RBE image that are necessary for running Bazel. Testing it out can be done by authenticating for RBE gcloud auth application-default login --no-browser Then, run make -C bazel rbe_known_good_builds to test it out. On my 4 core laptop with an empty local cache, but a warm remote cache, the build took <2 min instead of the 10+ minutes it would have [1]. The folder structure in //bazel/rbe is meant to let us have multiple remote configurations there, e.g. //bazel/rbe/gce_windows. Suggested Review Order: - bazel/rbe/README.md - bazel/rbe/gce_linux_container/Dockerfile to see the bare-bones RBE image. - bazel/rbe/BUILD.bazel to see a custom platform defined. It is nearly identical to the autogenerated one in bazel/rbe/gce_linux/config/BUILD, with one extra field to force the gce_linux pool to be used. - .bazelrc to see the settings needed to make --config=linux-rbe work. The naming convention was inspired by SkCMS's setup [2], and allows us to have some common RBE settings (i.e. config:remote) and some specialized ones for the given host machine (e.g. config:linux-rbe) A very important, but subtle configuration, is on line 86 of .bazelrc where we say to use our hermetic toolchain and not whatever C++ compiler and headers are on the host machine (aka the RBE container). - toolchain/build_toolchain.bzl to see some additional dependencies needed in the toolchain (to run IWYU) which I had installed locally but didn't realize were important. - third_party/BUILD.bazel to see an example of how failing to specify all files can result in something that works locally, but fails remotely. --execution_log_json_file=/tmp/execlog.json helped debug these issues. - All other files. [1] http://go/scrcast/NjM1ODE4MDI0NzM3MTc3Nnw3ODViZmFkMi1iOA [2] https://skia.googlesource.com/skcms/+/30c8e303800c256febb03a09fdcda7f75d119b1b/.bazelrc#20 Change-Id: Ia0a9e6a06c1a13071949ab402dc5d897df6b12e1 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/524359 Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
2022-03-25 18:59:33 +00:00
# Test the enforcement of include what you use
bazelisk build //example:hello_world_gl --config=linux-rbe --features skia_enforce_iwyu
# Both with and without a GPU backend should be error free (i.e. IWYU should let us
# conditionally import things.
bazelisk build //src/svg/... --config=linux-rbe --features skia_enforce_iwyu \
--gpu_backend=gl_backend --include_decoder=jpeg_decode_codec
bazelisk build //src/svg/... --config=linux-rbe --features skia_enforce_iwyu
bazelisk build //tools/debugger --config=linux-rbe --gpu_backend=gl_backend \
--features skia_enforce_iwyu
bazelisk build //:skia_core --config=linux-rbe --features skia_enforce_iwyu