In order to extract the PNG files produced by our CanvasKit gms,
we need our JS tests to POST them to a server which can write to
disk. The easiest way to do this is to use the test_on_env
rule defined in the Skia Infra repo for exactly this purpose.
This required https://skia-review.googlesource.com/c/buildbot/+/510717
to be able to configure the binary correctly and
https://skia-review.googlesource.com/c/buildbot/+/511862, for nicer
debugging so the skia-infra dep was updated via the following commands:
$ go get go.skia.org/infra@d8a552a29e
$ go mod download
$ make -C infra/bots train
$ make -C bazel gazelle_update_repo
This caused many automated changes to infra/bots/tasks.json
The flow is:
1. User types bazelisk test :hello_world_test_with_env
2. The test_on_env rule starts gold_test_env and waits
for the file defined in $ENV_READY_FILE to be created.
3. gold_test_env starts a web server on a random port. It
writes this port number to $ENV_DIR/port. Then, it
creates $ENV_READY_FILE to signal ready.
4. test_on_env sees the ready file and then starts the
karma_test rule. (Reminder: this is a bash script
which starts karma using the Bazel-bundled chromium).
5. The karma_test rule runs the karma.bazel.js file (which
has been injected with some JS code to fill in Bazel
paths and settings) using Bazel-bundled node. This reads
in the port file and sets up a Karma proxy to redirect
/gold_rpc/report to http://localhost:PORT/report
6. The JS tests run via Karma (and do assertions via Jasmine).
Some tests, the gms, make POST requests to the proxy.
7. gold_test_env gets these POST requests writes the images
to a special Bazel folder on disk as defined by
$TEST_UNDECLARED_OUTPUTS_DIR.
8. test_on_env identifies that the tests finish (because the
karma_test script returns 0). It sends SIGINT to gold_test_env.
9. gold_test_env stops the webserver. The special Bazel folder
will zip up anything inside it and make it available for
future rules (e.g. a rule that will upload to Gold via goldctl).
Suggested Review Order:
- bazel/karma_test.bzl to see the test_on_env rule bundled into
the karma_test macro. I chose to put it there because it might
be confusing to have to define both a karma_test and test_on_env
rule in the same package but not be able to call one because it
will fail to talk to the server.
- gold_test_env.go to see how the appropriate files are written
to signal the environment is ready and the handlers are set up.
- karma.bazel.js to see how we make our own proxy given the
port from the env binary. The fact that we could not create
our own proxy with the existing karma_test rule was why the
chain ending in https://skia-review.googlesource.com/c/skia/+/508797
had to be abandoned.
- tests/*.js to see how the environment is probed via /healthz
and then used to make POST requests with data.
- Everything else.
Change-Id: I32a90def41796ca94cf187d640cfff8e262f85f6
BUG: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/510737
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
In order to load CanvasKit, we need to add support for statically
served files. On the karma side, this is done by adding an
object [1] to the files list (example: [2]).
Then, we need to include canvaskit.js in with the karma test
files and use the callback to load canvaskit.wasm from the
correct file location.
[1] http://karma-runner.github.io/6.3/config/files.html
[2] 4f7b656012/modules/canvaskit/karma.conf.js (L13)
Change-Id: I7482d6e949a5e8efd0ca882efe5afbe0dc16c0e4
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/510736
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Run the tests in headless mode and output the logs
bazel test :hello_world --test_output=all
Start up a visible web browser with the karma test driver
(need to go to Debug tab to actually run tests)
bazel run :hello_world
Suggested review order
- package.json to see the karma dependencies to run
jasmine tests on chrome and firefox.
- WORKSPACE.bazel to see how the packages listed in
package.json and package-lock.json are downloaded
into the Bazel sandbox/cache via the npm_install rule.
As mentioned in the package.json comment, the version
of build_bazel_rules_nodejs which emscripten uses [1]
is 4.4.1 and if we tried to install it ourselves, that
installation will be ignored. We also bring in hermetic
browsers via io_bazel_rules_webtesting.
- bazel/karma_test.bzl which defines a new rule _karma_test
and a macro karma_test which joins the new rule with
an existing web_test rule to run it on a hermetic browser
which Bazel downloads. This rule takes heavy inspiration
from @bazel/concatjs [2], but is much simpler and lets us
configure more things (e.g. proxies, so we can work with
test_on_env).
- karma.bazel.js, which is a pretty ordinary looking karma
configuration file [2] with effectively a JS macro
BAZEL_APPLY_SETTINGS. JS doesn't have a preprocessor or
actual macros, but this string will be replaced by the
JS code in karma_test.bzl which will set correct filepaths
for Bazel content.
- All other files.
[1] c33c7be17f/bazel/deps.bzl (L10)
[2] 700b7a3c5f/packages/concatjs/web_test/karma_web_test.bzl (L318)
[3] http://karma-runner.github.io/6.3/config/configuration-file.html
Change-Id: Id64c0a86d6be37d627762cef0beaaf23ad390ac1
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/509717
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
