In order to extract the PNG files produced by our CanvasKit gms,
we need our JS tests to POST them to a server which can write to
disk. The easiest way to do this is to use the test_on_env
rule defined in the Skia Infra repo for exactly this purpose.
This required https://skia-review.googlesource.com/c/buildbot/+/510717
to be able to configure the binary correctly and
https://skia-review.googlesource.com/c/buildbot/+/511862, for nicer
debugging so the skia-infra dep was updated via the following commands:
$ go get go.skia.org/infra@d8a552a29e
$ go mod download
$ make -C infra/bots train
$ make -C bazel gazelle_update_repo
This caused many automated changes to infra/bots/tasks.json
The flow is:
1. User types bazelisk test :hello_world_test_with_env
2. The test_on_env rule starts gold_test_env and waits
for the file defined in $ENV_READY_FILE to be created.
3. gold_test_env starts a web server on a random port. It
writes this port number to $ENV_DIR/port. Then, it
creates $ENV_READY_FILE to signal ready.
4. test_on_env sees the ready file and then starts the
karma_test rule. (Reminder: this is a bash script
which starts karma using the Bazel-bundled chromium).
5. The karma_test rule runs the karma.bazel.js file (which
has been injected with some JS code to fill in Bazel
paths and settings) using Bazel-bundled node. This reads
in the port file and sets up a Karma proxy to redirect
/gold_rpc/report to http://localhost:PORT/report
6. The JS tests run via Karma (and do assertions via Jasmine).
Some tests, the gms, make POST requests to the proxy.
7. gold_test_env gets these POST requests writes the images
to a special Bazel folder on disk as defined by
$TEST_UNDECLARED_OUTPUTS_DIR.
8. test_on_env identifies that the tests finish (because the
karma_test script returns 0). It sends SIGINT to gold_test_env.
9. gold_test_env stops the webserver. The special Bazel folder
will zip up anything inside it and make it available for
future rules (e.g. a rule that will upload to Gold via goldctl).
Suggested Review Order:
- bazel/karma_test.bzl to see the test_on_env rule bundled into
the karma_test macro. I chose to put it there because it might
be confusing to have to define both a karma_test and test_on_env
rule in the same package but not be able to call one because it
will fail to talk to the server.
- gold_test_env.go to see how the appropriate files are written
to signal the environment is ready and the handlers are set up.
- karma.bazel.js to see how we make our own proxy given the
port from the env binary. The fact that we could not create
our own proxy with the existing karma_test rule was why the
chain ending in https://skia-review.googlesource.com/c/skia/+/508797
had to be abandoned.
- tests/*.js to see how the environment is probed via /healthz
and then used to make POST requests with data.
- Everything else.
Change-Id: I32a90def41796ca94cf187d640cfff8e262f85f6
BUG: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/510737
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
bazel run //example:hello_world --config=clang
causes a window to open and draws a circle and a square.
Text to follow in a future CL.
To make this work, I had to get rid of musl and use glibc.
All the shared libraries (.so files) that were pre-built
and available for download (e.g. from https://packages.debian.org/bullseye/amd64/libgl1/download)
were compiled against glibc. When I tried to run a
program statically linked with musl and dynamically linked
against things using glibc, I got a segmentation fault
on things like calloc().
Initial attempts to use glibc had failed because it was thought
that the libc.so.6 file could only be referred to by absolute
path (and thus Bazel would not be happy about it). As it turns out,
that was simply a misconfiguration of the builtin_sysroot
parameter to cc_common.create_cc_toolchain_config_info
(see //toolchain/clang_toolchain_config.bzl). By setting that
to `external/clang_linux_amd64` and not
`external/clang_linux_amd64/usr`, the libc binary which had
been extracted to `external/clang_linux_amd64/lib/x86_64-linux-gnu`
was perfectly reachable from
`external/clang_linux_amd64/usr/usr/lib/x86_64-linux-gnu/libc.so`
To bring in the shared libraries to link against (e.g. X11, GL)
I made build_toolchain.bzl easier to modify in that we simply need
to add a debian download url and sha256 hash to a list (rather than
having to plumb this through via arguments).
Recommended Review Order:
- example/BUILD.bazel (not sure if we always want to set bare
link arguments like that or if we want to use "features" to
pass those along to the toolchain).
- tools/sk_app/BUILD.bazel to see initial cc_library for
wrapping sk_app code.
- toolchain/build_toolchain.bzl to see removal of musl and
new list of debs.
- toolchain/clang_toolchain_config.bzl (where use of the
no-canonical-prefixes was key to compilation success).
Notice also that we statically linked libc++ (I did not
have any shared libraries for it locally, so I guessed
a typical developer might not either).
- Rest of toolchain/ for trivial renames.
- bazel/Makefile to see extra docs on those targets and
a new target that compiles all the exes so far for a
quick way to test the build.
- third_party/BUILD.bazel and src/gpu/BUILD.bazel which have
non-generated changes. (all other BUILD.bazel files do).
- go.mod, which needed to update the infra repo version in
order to pick up http://review.skia.org/491736).
Change-Id: I8687bd227353040eca2dffa9465798d8bd395027
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/492117
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Using prometheus's pushgateway since the metric will be reported from a task driver (a short-lived service-level job).
Updated go.mod/go.sum to get pushgateway libraries with:
$ go get go.skia.org/infra@2dce552b7a
$ go mod download
$ make -C infra/bots train
Bug: skia:12521
Change-Id: Ic63487b08d06163ef92556999c4920e5f178e285
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/487227
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
This way we can avoid breaking the existing flow until
we are sure it is building correctly.
Change-Id: I619f44e3d217eaabbd2f92c319cbd521451ee07e
Bug: skia:10614
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/478616
Reviewed-by: Eric Boren <borenet@google.com>
Updated go.mod/go.sum with:
$ go get go.skia.org/infra@4b39063800
$ go mod download
$ make -C infra/bots train
Bug: skia:12653
Change-Id: Iad62ea481724ee92d96b6d6d805a3f7a14c85917
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/475643
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
This adds a simple go program to test the installed go
toolchain, and a Make rule to codify the arguments to
our gazelle binary, built with extensions.
I could not figure out how to get the .json file to work
with the gazelle() Bazel rule, but this works ok for now.
Bug: skia:12541
Change-Id: I5067b15c7518951aeb69559d3871799d3b5745f4
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/475716
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
To pick up canary.go task driver change in buildbot repo
Updated with:
$ go get go.skia.org/infra@74751ee0f1
$ go mod download
$ make -C infra/bots train
Bug: skia:12657
Change-Id: I3f42dd6a8041d2c84903e736e76029c198c811f7
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/473144
Reviewed-by: Eric Boren <borenet@google.com>
This reverts commit 12e786730f.
Reason for revert: missing CIPD package on armv6l
Original change's description:
> [python3] More Recipes -> Python 3 fixes
>
> - Set environment variables to force usage of Python 3 in more places
> - Fix more compatibility issues
> - Mark recipes as only supporting Python 3
> - Includes a roll of the infra code
>
> Change-Id: I24e3827a6402c454bdc9467d28864d360632f9e6
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/470303
> Reviewed-by: Ravi Mistry <rmistry@google.com>
> Commit-Queue: Eric Boren <borenet@google.com>
Change-Id: If7b6fcb7838ac053af2c5eb45a7a1ac4aed340a5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/472736
Auto-Submit: Eric Boren <borenet@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
- Set environment variables to force usage of Python 3 in more places
- Fix more compatibility issues
- Mark recipes as only supporting Python 3
- Includes a roll of the infra code
Change-Id: I24e3827a6402c454bdc9467d28864d360632f9e6
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/470303
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
To pick up canary commit msg change in buildbot repo
Updated with:
$ go get go.skia.org/infra@553519b41e
$ go mod download
$ make -C infra/bots train
Bug: skia:12630
Change-Id: Iba6dc47dd014d690b54b017a2f9d4d597ecf0d97
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/469857
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
Bug: chromium:1244124
Change-Id: If44330139ddecc37da40ad7ad4ef90dbf45ddf6a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/447586
Reviewed-by: Erik Rose <erikrose@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
This picks up the change to use Bot-Commit+1 for RecreateSKPs
Bug: skia:12124
Change-Id: I4a22536e216a05fef15686503dbbfbb02b5e4808
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/445109
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
Removes --alsologtostderr flag, which was removed along with glog.
Change-Id: I8d2b5ac267feb0f8f88e19d346c7292fc825b1fb
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/442316
Reviewed-by: Erik Rose <erikrose@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
To pick up recent changes in buildbot repo for SkCQ
Updated with:
$ go get go.skia.org/infra@83d167c8de
$ go mod download
$ make -C infra/bots train
Bug: skia:12242
Change-Id: I4cca5c90ca7ba0a3180b8703256fed4983061e46
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/430879
Commit-Queue: Ravi Mistry <rmistry@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Change-Id: I13ae7034cda4291743f433406ee2e8267dd48d3e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/416576
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
This also updates a few other CIPD packages.
Change-Id: I078e7ac8da49ae79392263013158956a6d29c656
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/422096
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
This reverts commit 8cb7c3be75.
Just leave Win7 Golo tasks on python 2.
Change-Id: Idf1d6cc6d303107afd8e74cdec94e7a9ec91399e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/416316
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
This reverts commit c227bbea02.
Reason for revert: Breaking Win7 Golo
Original change's description:
> [infra] Roll Infra dep to update CIPD packages
>
> This is intended to pick up https://skia-review.googlesource.com/c/buildbot/+/414458 which updates the tasks to use cpython3, but it also updates the luci-auth, kitchen, and git packages.
>
> Also includes fixes needed after https://skia-review.googlesource.com/c/buildbot/+/387896
>
> Bug: chromium:1210114
> Change-Id: I2003be762b59a08a6b972b6563e4f3a9920b082d
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/414438
> Reviewed-by: Kevin Lubick <kjlubick@google.com>
> Reviewed-by: Joe Gregorio <jcgregorio@google.com>
> Commit-Queue: Eric Boren <borenet@google.com>
TBR=borenet@google.com,jcgregorio@google.com,kjlubick@google.com
Change-Id: Ie4cf5932ad163d2085ecd7c08b915cbeaea3f14b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:1210114
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/415899
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
To pick up recent changes in buildbot repo for master-> main migration
Updated with:
$ go get go.skia.org/infra@1a8ff8585f
$ make -C infra/bots train
Bug: skia:11987
Change-Id: Iffa108dff585153c5bf47e709e99d3fa805d6d51
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/409277
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
Change-Id: Ibe4e929b7d160b3dbf0e5b6ae093b4bab419817d
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/375198
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
Ran the following commands
from $SKIA_ROOT
go get go.skia.org/infra@e5c4a9cfc4
make -C infra/bots train
Change-Id: Id00ebbe564edfe9024e3291ff0726e92bc05965f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/338217
Commit-Queue: Nathaniel Nifong <nifong@google.com>
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Main impetus is to update Debugger autodeploy to be the lit-html version.
Ran the following commands from $SKIA_ROOT:
$ go get go.skia.org/infra@1e7e10cc95
$ make -C infra/bots train
Change-Id: I75f5d1d135530411e6a3068b18dde24bc61ee10a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/335581
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
Auto-Submit: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
Change-Id: Id97669b7916a6909a39dda79ecb9671f7fe1caf0
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/330818
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
This picks up a new version of the CIPD deps and a new helper for
writing task drivers.
Bug: skia:10812
Change-Id: I8b9b57acd4d8eee9cdea86008da1f3039af0cdc9
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/328496
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
