08ece0c9a0
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Kevin Lubick
|
08ece0c9a0 |
[includes] Enforce IWYU on sksl code
PS1 regenerates the Bazel files. Use it as the base change when
comparing patchsets.
IWYU seems to do a good job of working with MyFile.cpp and
MyFile.h, but if there is just a MyHeader.h, it doesn't always
seem to throw errors if the includes aren't correct. This was
observed with include/sksl/DSL.h This might be due to the fact
that headers are not compiled on their own, so they are never
sent directly to the IWYU binary.
This change sets enforce_iwyu_on_package() on the all sksl
packages and then fixes the includes until all those checks
are happy. There were a few files that needed fixes outside
of the sksl folder. Examples include:
- src/gpu/effects/GrConvexPolyEffect.cpp
- tests/SkSLDSLTest.cpp
To really enforce this, we need to add a CI/CQ job that runs
bazel build //example:hello_world_gl --config=clang \
--sandbox_base=/dev/shm --features skia_enforce_iwyu
If that failed, a dev could make the changes described in
the logs and/or run the command locally to see those
prescribed fixes.
I had to add several entries to toolchain/IWYU_mapping.imp
in order to fix some private includes and other atypical
choices. I tried adding a rule there to allow inclusion of
SkTypes.h to make sure defines like SK_SUPPORT_GPU, but
could not get it to work for all cases, so I deferred to
using the IWYU pragma: keep (e.g. SkSLPipelineStageCodeGenerator.h)
Change-Id: I4c3e536d8e69ff7ff2d26fe61a525a6c2e80db06
Bug: skia:13052
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/522256
Reviewed-by: John Stiles <johnstiles@google.com>
Reviewed-by: Greg Daniel <egdaniel@google.com>
|
||
|
Kevin Lubick
|
e253cc3e55 |
[bazel] Use toolchain features to opt files into being IWYU compliant.
PS1 regenerates the Bazel files.
It is recommended to review this CL with a diff from PS1.
Example output when a file does not pass the test:
tools/sk_app/CommandSet.h should add these lines:
#include "include/core/SkTypes.h"
#include "include/private/SkTArray.h"
#include "tools/skui/InputState.h"
#include "tools/skui/Key.h"
#include "tools/skui/ModifierKey.h"
namespace sk_app { class Window; }
tools/sk_app/CommandSet.h should remove these lines:
- #include "tools/sk_app/Window.h"
The full include-list for tools/sk_app/CommandSet.h:
#include "include/core/SkString.h"
#include "include/core/SkTypes.h"
#include "include/private/SkTArray.h"
#include "tools/skui/InputState.h"
#include "tools/skui/Key.h"
#include "tools/skui/ModifierKey.h"
#include <functional>
#include <vector>
class SkCanvas;
namespace sk_app { class Window; }
---
This makes use of Bazel's toolchain features
https://bazel.build/docs/cc-toolchain-config-reference#features
to allow us to configure compiler flags when compiling
individual files. This analysis is off by default, and can
be turned on with --features skia_enforce_iwyu. When enabled,
it will only be run for files that have opted in.
Example:
bazelisk build //example:hello_world_gl --config=clang \
--sandbox_base=/dev/shm --features skia_enforce_iwyu
There are two ways to opt files in:
- Add enforce_iwyu = True to a generated_cc_atom rule
- Add enforce_iwyu_on_package() to a BUILD.bazel file
(which enforces IWYU for all rules in that file)
Note that Bazel does not propagate features to dependencies
or dependents, so trying to enable the feature on cc_library
or cc_executable targets will only impact any files listed in
srcs or hdrs, not deps. This may be counter-intuitive when
compared to things like defines.
IWYU supports a mapping file, which we supply to help properly
handle things system headers (//toolchain/IWYU_mapping.imp)
Suggested Review Order:
- toolchain/build_toolchain.bzl to see how we get the IWYU
binaries into the toolchain
- toolchain/BUILD.bazel and toolchain/IWYU_mapping.imp
to see how the mapping file is made available for
all compile steps
- toolchain/clang_toolchain_config.bzl, where we define the
skia_enforce_iwyu feature to turn on any verification at
all and skia_opt_file_into_iwyu to enable the check for
specific files using a define.
- toolchain/clang_trampoline.sh, which is the toolchain is
configured to call instead of clang directly (see line 83
of clang_toolchain_config.bzl). This bash script used to
just forward all arguments directly onto clang. Now it
inspects them and either calls clang directly (if
it does not find the define in the arguments or we are
linking [bazel sometimes links with clang instead of ld])
or calls clang and then include-what-you-use. In all cases,
the trampoline sends the arguments to clang and IWYU
unchanged).
- //tools/sk_app/... to see enforcement enabled (and fixed)
for select files, as an example of that method.
- //experimental/bazel_test/... to see enforcement enabled
for all rules in a BUILD.bazel file.
- all other files.
Change-Id: I60a2ea9d5dc9955b6a8f166bd449de9e2b81a233
Bug: skia:13052
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/519776
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
|
||
|
Kevin Lubick
|
7ac7413f08 |
[bazel] Support Vulkan
PS1 regenerates BUILD.bazel files I suggest reviewing the deltas between PS1 and the latest PS to focus on the interesting bits. The changes here allow for a Vulkan-only build of HelloWorld based on sk_app. The toughest change was properly fetching the VisualID after removing the gl calls that used to fill that in. There are a few changes that fix resolution of Dawn header files, but those won't actually be built until a follow-on CL. Change-Id: I54fb58b5dd7ecd4313562aed401759b3eaed53c0 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/516999 Reviewed-by: Greg Daniel <egdaniel@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com> |
||
|
Kevin Lubick
|
acab911351 |
[bazel] Make use of test_on_env to spin up server for gms
In order to extract the PNG files produced by our CanvasKit gms, we need our JS tests to POST them to a server which can write to disk. The easiest way to do this is to use the test_on_env rule defined in the Skia Infra repo for exactly this purpose. This required https://skia-review.googlesource.com/c/buildbot/+/510717 to be able to configure the binary correctly and https://skia-review.googlesource.com/c/buildbot/+/511862, for nicer debugging so the skia-infra dep was updated via the following commands: $ go get go.skia.org/infra@d8a552a29e $ go mod download $ make -C infra/bots train $ make -C bazel gazelle_update_repo This caused many automated changes to infra/bots/tasks.json The flow is: 1. User types bazelisk test :hello_world_test_with_env 2. The test_on_env rule starts gold_test_env and waits for the file defined in $ENV_READY_FILE to be created. 3. gold_test_env starts a web server on a random port. It writes this port number to $ENV_DIR/port. Then, it creates $ENV_READY_FILE to signal ready. 4. test_on_env sees the ready file and then starts the karma_test rule. (Reminder: this is a bash script which starts karma using the Bazel-bundled chromium). 5. The karma_test rule runs the karma.bazel.js file (which has been injected with some JS code to fill in Bazel paths and settings) using Bazel-bundled node. This reads in the port file and sets up a Karma proxy to redirect /gold_rpc/report to http://localhost:PORT/report 6. The JS tests run via Karma (and do assertions via Jasmine). Some tests, the gms, make POST requests to the proxy. 7. gold_test_env gets these POST requests writes the images to a special Bazel folder on disk as defined by $TEST_UNDECLARED_OUTPUTS_DIR. 8. test_on_env identifies that the tests finish (because the karma_test script returns 0). It sends SIGINT to gold_test_env. 9. gold_test_env stops the webserver. The special Bazel folder will zip up anything inside it and make it available for future rules (e.g. a rule that will upload to Gold via goldctl). Suggested Review Order: - bazel/karma_test.bzl to see the test_on_env rule bundled into the karma_test macro. I chose to put it there because it might be confusing to have to define both a karma_test and test_on_env rule in the same package but not be able to call one because it will fail to talk to the server. - gold_test_env.go to see how the appropriate files are written to signal the environment is ready and the handlers are set up. - karma.bazel.js to see how we make our own proxy given the port from the env binary. The fact that we could not create our own proxy with the existing karma_test rule was why the chain ending in https://skia-review.googlesource.com/c/skia/+/508797 had to be abandoned. - tests/*.js to see how the environment is probed via /healthz and then used to make POST requests with data. - Everything else. Change-Id: I32a90def41796ca94cf187d640cfff8e262f85f6 BUG: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/510737 Reviewed-by: Leandro Lovisolo <lovisolo@google.com> |
||
|
Kevin Lubick
|
b902658a1a |
[bazel] Fix Bazel build by guarding addToKey
PS1 is the automatic regenerated changes. PS2-3 adds an #ifdef guard to the addToKey method on shaders. The SkShaderCodeDictionary class helps generate SkSL and is only necessary when we are building with SkSL (gpu builds and cpu builds with SkVM). Suggested Review order: - Use Gerrit to diff PS 1 and the last PS - src/core/BUILD.bazel adds some sources to the "only necessary if sksl is enabled" bucket - All the .cpp and .h files to see the #ifdef is added correctly. Change-Id: I4d4ce61a4957ef1e0840204acff08ce7e616f9cb Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/512157 Reviewed-by: Michael Ludwig <michaelludwig@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com> |
||
|
Kevin Lubick
|
e94b52c442 |
[canvaskit] Add skottie to Bazel build
PS 1 regenerates existing Bazel files PS 2 adds generated Bazel files to skottie and its dependencies, as well as incorporating it into CanvasKit. This changes the version of Bazel we use to 5.0.0 (recently released).We had been using a pre-release of 6.0 because we wanted the new features in one of the 5.0 release candidates, but not the regression that was there (and reverted before the full 5.0 release). I'd like to stick to the latest stable Bazel release where possible. Suggested Review Order: - //modules/skottie/BUILD.bazel (this was hand written to encapsulate the skottie library). The files in the deps are based on skottie.gni. - //modules/skresources/BUILD.bazel and //modules/sksg/BUILD.bazel which expose all sources - //third_party/file_map_for_bazel.json which ignores the ffmpeg libraries (we won't actually build the SkVideoDecoder stuff because HAVE_VIDEO_DECODER is not set). - //modules/canvaskit/BUILD.bazel which makes use of the skottie library and includes the interface skottie.js file. - .bazelversion which changes the Bazel version used (e.g. by Bazelisk). - All other changes should be auto-generated or related to deleted files. Change-Id: Ic26f9a9dea5310f2cbd9cda7d701847924a39a22 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503828 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Jorge Betancourt <jmbetancourt@google.com> |
||
|
Kevin Lubick
|
7a14f783bd |
[bazel] Sketching out HelloWorld sk_app using GL
bazel run //example:hello_world --config=clang causes a window to open and draws a circle and a square. Text to follow in a future CL. To make this work, I had to get rid of musl and use glibc. All the shared libraries (.so files) that were pre-built and available for download (e.g. from https://packages.debian.org/bullseye/amd64/libgl1/download) were compiled against glibc. When I tried to run a program statically linked with musl and dynamically linked against things using glibc, I got a segmentation fault on things like calloc(). Initial attempts to use glibc had failed because it was thought that the libc.so.6 file could only be referred to by absolute path (and thus Bazel would not be happy about it). As it turns out, that was simply a misconfiguration of the builtin_sysroot parameter to cc_common.create_cc_toolchain_config_info (see //toolchain/clang_toolchain_config.bzl). By setting that to `external/clang_linux_amd64` and not `external/clang_linux_amd64/usr`, the libc binary which had been extracted to `external/clang_linux_amd64/lib/x86_64-linux-gnu` was perfectly reachable from `external/clang_linux_amd64/usr/usr/lib/x86_64-linux-gnu/libc.so` To bring in the shared libraries to link against (e.g. X11, GL) I made build_toolchain.bzl easier to modify in that we simply need to add a debian download url and sha256 hash to a list (rather than having to plumb this through via arguments). Recommended Review Order: - example/BUILD.bazel (not sure if we always want to set bare link arguments like that or if we want to use "features" to pass those along to the toolchain). - tools/sk_app/BUILD.bazel to see initial cc_library for wrapping sk_app code. - toolchain/build_toolchain.bzl to see removal of musl and new list of debs. - toolchain/clang_toolchain_config.bzl (where use of the no-canonical-prefixes was key to compilation success). Notice also that we statically linked libc++ (I did not have any shared libraries for it locally, so I guessed a typical developer might not either). - Rest of toolchain/ for trivial renames. - bazel/Makefile to see extra docs on those targets and a new target that compiles all the exes so far for a quick way to test the build. - third_party/BUILD.bazel and src/gpu/BUILD.bazel which have non-generated changes. (all other BUILD.bazel files do). - go.mod, which needed to update the infra repo version in order to pick up http://review.skia.org/491736). Change-Id: I8687bd227353040eca2dffa9465798d8bd395027 Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/492117 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Commit-Queue: Kevin Lubick <kjlubick@google.com> |
||
|
Kevin Lubick
|
96e4053be7 |
Update Bazel files
- Use latest emscripten toolchain (3.1.0) - Autogenerate the atoms and manually fix some of the file lists. - Add a known_good_builds target to bazel/Makefile to help check the things we expect to work with Bazel. Change-Id: Ia5f51e7b9eb5c108386820ad59180c8f862f5a70 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/491438 Reviewed-by: Ben Wagner <bungeman@google.com> |
||
|
Kevin Lubick
|
9488e0237d |
[infra] Experiment generating BUILD.bazel files
This uses the gazelle extension from https://skia-review.googlesource.com/c/buildbot/+/473357 Review Tips: - Ignore any changes to .h or .cpp files. Those have been pulled out into their own CLs. - Start with bazel/macros.bzl. - Read the CL with the generation code, if you haven't already. - Look at third_party/file_map_for_bazel.json. - See experimental/bazel_test for an idea of how a cc_binary would be made. - Spot check one or two of the BUILD.bazel files. This CL generates the "atomic" rules for src/, include/ and modules/skshaper, as a starting point. `bazel build --config clang //include/...` works `bazel build --config clang //src/...` starts compiling, (which verifies that the BUILD.bazel files are all valid), but runs into errors because not all third_party deps have been resolved, and there are some files missing from the toolchain still (e.g. EGL headers). Change-Id: Ib7e0fb0efdb9f08655f06cbc56e9bb4cf416294b Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/474240 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> |
||
|
Kevin Lubick
|
1f8c31b101 |
[infra] Add initial Bazel rules and files
These rules can be used to build our GMs on WASM+WebGL and libskia.a with just the CPU backend (and most other features turned off). This can be done with the following commands: - bazel build //modules/canvaskit:gm-bindings-wasm --gpu_backend=gl_backend --with_gl_standard=webgl_standard - bazel build :skia-core --config clang This pivots slightly from http://review.skia.org/463517 by using config_settings [1] instead of platforms for the optional features that we control. This pivot was suggested in [2] We have BUILD.bazel files in many of the subdirectories that specify filegroups for the appropriate files. In an effort to make //BUILD.bazel more readable, it is the responsibility of these subfolders to deal with conditionally including certain .h or .cpp files. This is done using select statements and config_settings or platform constraints as necessary. For example, src/gpu/BUILD.bazel will different private filegroups for each of the supported gpu backends [3] and a more-visible filegroup called "srcs" that has the right selection of the private files to be used for compilation. An effort has been made to avoid using glob() in our BUILD.bazel files. These file lists were made by using `ls -1` and some regex to add in quotes. We might want to make a helper script to assist with that, if necessary. To specify which options we have, the settings in //bazel/common_config_settings/BUILD.bazel have been redesigned. They make use of a macro `string_flag_with_values` that removes the boilerplate. Patchset 36 shows what the file looks like w/o the macro. The top level BUILD.bazel file will still need to use some logic to handle defines, because local_defines is a list of strings, not a list of labels [4]. Suggested Review Order: - WORKSPACE.bazel to see the new dependencies on the emsdk toolchain and bazel_skylib - bazel/common_config_settings/* to see the few settings defined (we have more to define, see BUILD.gn and //gn/skia.gni for ideas) - BUILD.bazel to see the "skia-core" cc_library rule. See also "gms" and "tests" - modules/canvaskit/BUILD.bazel to see the use of the emscripten "wasm_cc_binary" rule, which depends on the "skia-core", "gms", and "tests" rule. Note that it only builds some of the gms as a proof of concept. - The other BUILD.bazel files. Some of these are not platform or feature dependent (e.g. pathops). Others are (e.g. gpu). - All other files. [1] https://docs.bazel.build/versions/4.2.1/skylark/config.html#user-defined-build-settings [2] https://github.com/emscripten-core/emsdk/pull/920 [3] In this CL, that's just the webgl one. [4] https://docs.bazel.build/versions/main/be/c-cpp.html#cc_library.local_defines Change-Id: Ieecf9c106d5e3a6ae97d13d66be06b4b3c207089 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/458637 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Owners-Override: Kevin Lubick <kjlubick@google.com> |
||
|
Kevin Lubick
|
97284f255b |
[infra] Add initial platforms and constraint values
This makes use of Bazel's pre-defined platforms https://github.com/bazelbuild/platforms and some of our own defined values (see //bazel/common_config_settings/BUILD.bazel) to customize the build rules. I verified this by building bazel_test locally for linux x64 as well as using the third_party deps for a WASM build (using build files not seen in this CL). Suggested Review Order: - https://docs.bazel.build/versions/main/platforms.html if not already familiar with Bazel Platforms - third_party/BUILD.bazel to see that 1) all globs have been removed and 2) select() targets various platform constants or groups of constants to control sources, headers, and local_defines. - common_config_settings/ to see the groups of constraints created, as well as new constraint_settings defined (skdebug_impl) - supported_combinations/ to see how we can define supported sets of the constraint values (aka Bazel platforms). I imagine expanding this more, so we might have platforms named "linux_x64_emptyfontmgr_vulkan" or such. - //BUILD.bazel and bazel_test.cpp to see use of SkDebugf. - Everything else. Change-Id: I49e4abdbcf7b76f0674efdbb1f53dc8823d110ee Bug: skia:12541 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/463517 Reviewed-by: Ben Wagner <bungeman@google.com> Reviewed-by: Leandro Lovisolo <lovisolo@google.com> Owners-Override: Kevin Lubick <kjlubick@google.com> |