AuroraMiddleware/skia2
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
59,416 Commits 1 Branch 0 Tags 544 MiB
953c92880b
Commit Graph

8 Commits

Author SHA1 Message Date
Kevin Lubick
acab911351 [bazel] Make use of test_on_env to spin up server for gms 
In order to extract the PNG files produced by our CanvasKit gms,
we need our JS tests to POST them to a server which can write to
disk. The easiest way to do this is to use the test_on_env
rule defined in the Skia Infra repo for exactly this purpose.

This required https://skia-review.googlesource.com/c/buildbot/+/510717
to be able to configure the binary correctly and
https://skia-review.googlesource.com/c/buildbot/+/511862, for nicer
debugging so the skia-infra dep was updated via the following commands:
$ go get go.skia.org/infra@d8a552a29e
$ go mod download
$ make -C infra/bots train
$ make -C bazel gazelle_update_repo
This caused many automated changes to infra/bots/tasks.json

The flow is:
1. User types bazelisk test :hello_world_test_with_env
2. The test_on_env rule starts gold_test_env and waits
   for the file defined in $ENV_READY_FILE to be created.
3. gold_test_env starts a web server on a random port. It
   writes this port number to $ENV_DIR/port. Then, it
   creates $ENV_READY_FILE to signal ready.
4. test_on_env sees the ready file and then starts the
   karma_test rule. (Reminder: this is a bash script
   which starts karma using the Bazel-bundled chromium).
5. The karma_test rule runs the karma.bazel.js file (which
   has been injected with some JS code to fill in Bazel
   paths and settings) using Bazel-bundled node. This reads
   in the port file and sets up a Karma proxy to redirect
   /gold_rpc/report to http://localhost:PORT/report
6. The JS tests run via Karma (and do assertions via Jasmine).
   Some tests, the gms, make POST requests to the proxy.
7. gold_test_env gets these POST requests writes the images
   to a special Bazel folder on disk as defined by
   $TEST_UNDECLARED_OUTPUTS_DIR.
8. test_on_env identifies that the tests finish (because the
   karma_test script returns 0). It sends SIGINT to gold_test_env.
9. gold_test_env stops the webserver. The special Bazel folder
   will zip up anything inside it and make it available for
   future rules (e.g. a rule that will upload to Gold via goldctl).

Suggested Review Order:
 - bazel/karma_test.bzl to see the test_on_env rule bundled into
   the karma_test macro. I chose to put it there because it might
   be confusing to have to define both a karma_test and test_on_env
   rule in the same package but not be able to call one because it
   will fail to talk to the server.
 - gold_test_env.go to see how the appropriate files are written
   to signal the environment is ready and the handlers are set up.
 - karma.bazel.js to see how we make our own proxy given the
   port from the env binary. The fact that we could not create
   our own proxy with the existing karma_test rule was why the
   chain ending in https://skia-review.googlesource.com/c/skia/+/508797
   had to be abandoned.
 - tests/*.js to see how the environment is probed via /healthz
   and then used to make POST requests with data.
 - Everything else.

Change-Id: I32a90def41796ca94cf187d640cfff8e262f85f6
BUG: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/510737
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
 2022-02-28 14:05:54 +00:00
Kevin Lubick
b902658a1a [bazel] Fix Bazel build by guarding addToKey 
PS1 is the automatic regenerated changes.
PS2-3 adds an #ifdef guard to the addToKey method on shaders.
The SkShaderCodeDictionary class helps generate SkSL and is
only necessary when we are building with SkSL (gpu builds and
cpu builds with SkVM).

Suggested Review order:
 - Use Gerrit to diff PS 1 and the last PS
 - src/core/BUILD.bazel adds some sources to the "only
   necessary if sksl is enabled" bucket
 - All the .cpp and .h files to see the #ifdef is added
   correctly.

Change-Id: I4d4ce61a4957ef1e0840204acff08ce7e616f9cb
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/512157
Reviewed-by: Michael Ludwig <michaelludwig@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
 2022-02-24 18:10:40 +00:00
Kevin Lubick
e94b52c442 [canvaskit] Add skottie to Bazel build 
PS 1 regenerates existing Bazel files
PS 2 adds generated Bazel files to skottie and its dependencies,
as well as incorporating it into CanvasKit.

This changes the version of Bazel we use to 5.0.0 (recently
released).We had been using a pre-release of 6.0 because we
wanted the new features in one of the 5.0 release candidates,
but not the regression that was there (and reverted before the
full 5.0 release). I'd like to stick to the latest stable Bazel
release where possible.

Suggested Review Order:
 - //modules/skottie/BUILD.bazel (this was hand written
   to encapsulate the skottie library). The files in the
   deps are based on skottie.gni.
 - //modules/skresources/BUILD.bazel and //modules/sksg/BUILD.bazel
   which expose all sources
 - //third_party/file_map_for_bazel.json which ignores the
   ffmpeg libraries (we won't actually build the SkVideoDecoder
   stuff because HAVE_VIDEO_DECODER is not set).
 - //modules/canvaskit/BUILD.bazel which makes use of the skottie
   library and includes the interface skottie.js file.
 - .bazelversion which changes the Bazel version used (e.g. by
   Bazelisk).
 - All other changes should be auto-generated or related to
   deleted files.

Change-Id: Ic26f9a9dea5310f2cbd9cda7d701847924a39a22
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503828
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Jorge Betancourt <jmbetancourt@google.com>
 2022-02-04 19:18:27 +00:00
Kevin Lubick
7a14f783bd [bazel] Sketching out HelloWorld sk_app using GL 
bazel run //example:hello_world --config=clang
causes a window to open and draws a circle and a square.
Text to follow in a future CL.

To make this work, I had to get rid of musl and use glibc.
All the shared libraries (.so files) that were pre-built
and available for download (e.g. from https://packages.debian.org/bullseye/amd64/libgl1/download)
were compiled against glibc. When I tried to run a
program statically linked with musl and dynamically linked
against things using glibc, I got a segmentation fault
on things like calloc().

Initial attempts to use glibc had failed because it was thought
that the libc.so.6 file could only be referred to by absolute
path (and thus Bazel would not be happy about it). As it turns out,
that was simply a misconfiguration of the builtin_sysroot
parameter to cc_common.create_cc_toolchain_config_info
(see //toolchain/clang_toolchain_config.bzl). By setting that
to `external/clang_linux_amd64` and not
`external/clang_linux_amd64/usr`, the libc binary which had
been extracted to `external/clang_linux_amd64/lib/x86_64-linux-gnu`
was perfectly reachable from
`external/clang_linux_amd64/usr/usr/lib/x86_64-linux-gnu/libc.so`

To bring in the shared libraries to link against (e.g. X11, GL)
I made build_toolchain.bzl easier to modify in that we simply need
to add a debian download url and sha256 hash to a list (rather than
having to plumb this through via arguments).

Recommended Review Order:
 - example/BUILD.bazel (not sure if we always want to set bare
   link arguments like that or if we want to use "features" to
   pass those along to the toolchain).
 - tools/sk_app/BUILD.bazel to see initial cc_library for
   wrapping sk_app code.
 - toolchain/build_toolchain.bzl to see removal of musl and
   new list of debs.
 - toolchain/clang_toolchain_config.bzl (where use of the
   no-canonical-prefixes was key to compilation success).
   Notice also that we statically linked libc++ (I did not
   have any shared libraries for it locally, so I guessed
   a typical developer might not either).
 - Rest of toolchain/ for trivial renames.
 - bazel/Makefile to see extra docs on those targets and
   a new target that compiles all the exes so far for a
   quick way to test the build.
 - third_party/BUILD.bazel and src/gpu/BUILD.bazel which have
   non-generated changes. (all other BUILD.bazel files do).
 - go.mod, which needed to update the infra repo version in
   order to pick up http://review.skia.org/491736).

Change-Id: I8687bd227353040eca2dffa9465798d8bd395027
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/492117
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
 2022-01-11 13:06:19 +00:00
Kevin Lubick
96e4053be7 Update Bazel files 
- Use latest emscripten toolchain (3.1.0)
 - Autogenerate the atoms and manually fix some of the file lists.
 - Add a known_good_builds target to bazel/Makefile to help
   check the things we expect to work with Bazel.

Change-Id: Ia5f51e7b9eb5c108386820ad59180c8f862f5a70
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/491438
Reviewed-by: Ben Wagner <bungeman@google.com>
 2022-01-06 18:10:57 +00:00
Kevin Lubick
9488e0237d [infra] Experiment generating BUILD.bazel files 
This uses the gazelle extension from
https://skia-review.googlesource.com/c/buildbot/+/473357

Review Tips:
 - Ignore any changes to .h or .cpp files. Those have been
   pulled out into their own CLs.
 - Start with bazel/macros.bzl.
 - Read the CL with the generation code, if you haven't already.
 - Look at third_party/file_map_for_bazel.json.
 - See experimental/bazel_test for an idea of how a cc_binary
   would be made.
 - Spot check one or two of the BUILD.bazel files.

This CL generates the "atomic" rules for src/, include/ and
modules/skshaper, as a starting point.

`bazel build --config clang //include/...` works

`bazel build --config clang //src/...` starts compiling,
(which verifies that the BUILD.bazel files are all valid),
but runs into errors because not all third_party deps have
been resolved, and there are some files missing from the
toolchain still (e.g. EGL headers).

Change-Id: Ib7e0fb0efdb9f08655f06cbc56e9bb4cf416294b
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/474240
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
 2021-11-23 18:53:16 +00:00
Kevin Lubick
1f8c31b101 [infra] Add initial Bazel rules and files 
These rules can be used to build our GMs on WASM+WebGL and
libskia.a with just the CPU backend (and most other features
turned off).

This can be done with the following commands:
  - bazel build //modules/canvaskit:gm-bindings-wasm --gpu_backend=gl_backend --with_gl_standard=webgl_standard
  - bazel build :skia-core --config clang

This pivots slightly from http://review.skia.org/463517
by using config_settings [1] instead of platforms for
the optional features that we control. This pivot was
suggested in [2]

We have BUILD.bazel files in many of the subdirectories
that specify filegroups for the appropriate files. In
an effort to make //BUILD.bazel more readable, it is
the responsibility of these subfolders to deal with
conditionally including certain .h or .cpp files.
This is done using select statements and config_settings
or platform constraints as necessary.

For example, src/gpu/BUILD.bazel will different private
filegroups for each of the supported gpu backends [3]
and a more-visible filegroup called "srcs" that has
the right selection of the private files to be used
for compilation.

An effort has been made to avoid using glob() in our
BUILD.bazel files. These file lists were made by using
`ls -1` and some regex to add in quotes. We might want
to make a helper script to assist with that, if necessary.

To specify which options we have, the settings in
//bazel/common_config_settings/BUILD.bazel have been
redesigned. They make use of a macro `string_flag_with_values`
that removes the boilerplate. Patchset 36 shows what the
file looks like w/o the macro.

The top level BUILD.bazel file will still need to use
some logic to handle defines, because local_defines is
a list of strings, not a list of labels [4].

Suggested Review Order:
  - WORKSPACE.bazel to see the new dependencies on the
    emsdk toolchain and bazel_skylib
  - bazel/common_config_settings/* to see the few settings
    defined (we have more to define, see BUILD.gn and
    //gn/skia.gni for ideas)
  - BUILD.bazel to see the "skia-core" cc_library rule.
    See also "gms" and "tests"
  - modules/canvaskit/BUILD.bazel to see the use of
    the emscripten "wasm_cc_binary" rule, which depends
    on the "skia-core", "gms", and "tests" rule. Note that
    it only builds some of the gms as a proof of concept.
  - The other BUILD.bazel files. Some of these are not
    platform or feature dependent (e.g. pathops). Others
    are (e.g. gpu).
  - All other files.

[1] https://docs.bazel.build/versions/4.2.1/skylark/config.html#user-defined-build-settings
[2] https://github.com/emscripten-core/emsdk/pull/920
[3] In this CL, that's just the webgl one.
[4] https://docs.bazel.build/versions/main/be/c-cpp.html#cc_library.local_defines

Change-Id: Ieecf9c106d5e3a6ae97d13d66be06b4b3c207089
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/458637
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Owners-Override: Kevin Lubick <kjlubick@google.com>
 2021-11-09 12:32:25 +00:00
Kevin Lubick
97284f255b [infra] Add initial platforms and constraint values 
This makes use of Bazel's pre-defined platforms
https://github.com/bazelbuild/platforms
and some of our own defined values (see
//bazel/common_config_settings/BUILD.bazel) to customize
the build rules.

I verified this by building bazel_test locally for
linux x64 as well as using the third_party deps for
a WASM build (using build files not seen in this CL).

Suggested Review Order:
 - https://docs.bazel.build/versions/main/platforms.html if not
   already familiar with Bazel Platforms
 - third_party/BUILD.bazel to see that 1) all globs have
   been removed and 2) select() targets various
   platform constants or groups of constants to control
   sources, headers, and local_defines.
 - common_config_settings/ to see the groups of constraints
   created, as well as new constraint_settings defined
   (skdebug_impl)
 - supported_combinations/ to see how we can define supported
   sets of the constraint values (aka Bazel platforms).
   I imagine expanding this more, so we might have platforms
   named "linux_x64_emptyfontmgr_vulkan" or such.
 - //BUILD.bazel and bazel_test.cpp to see use of SkDebugf.
 - Everything else.

Change-Id: I49e4abdbcf7b76f0674efdbb1f53dc8823d110ee
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/463517
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Owners-Override: Kevin Lubick <kjlubick@google.com>
 2021-10-26 18:27:13 +00:00