This loads in the known digests from Gold, starts the test
harness (which runs the GMs using puppeteer) and then uses
goldctl to upload the results to Gold when finished.
This will fail (and should not be landed) until
https://skia-review.googlesource.com/c/buildbot/+/328156
makes it into goldctl and the cipd build.
Bug: skia:10812
Change-Id: I89e5cf188d8f2adeba4ff676525d9bfbdcb46d5a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/328380
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Bug: skia:10477
Change-Id: Ibf9bcb1d03a6003d00b124db8d826c7952842fef
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/300780
Commit-Queue: Ravi Mistry <rmistry@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Some of these use the now-deprecated "command" feature.
Change-Id: I7e0bf0141821e2807146d9b0bb99a69da3a47fd2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/291037
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
Also, create and use new service account for the G3 compile bots.
Interesting failure here refers to when the bot fails only because of
test failures and not:
* merge failures
* unknown failure reasons (eg: some flaky timeouts from the script running in borg)
* when the bot is not owned or approved by a Googler
NoTry: true
Bug: skia:9631
Change-Id: I964fd7874269116cca0e25d1b516283a00d5133f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/254278
Commit-Queue: Ravi Mistry <rmistry@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Also, create and use new service account for the Android compile bots.
NoTry: true
Bug: skia:9568
Change-Id: I974229a87f8c00bf6e4d0d5e49bbfdde6ca84e19
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/250978
Commit-Queue: Ravi Mistry <rmistry@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
We've failed to get these bots to report where issues happen
any more finely than the name of the executable, which makes
them a real pain to fix when they go red.
We don't expect we'll be able to run cleanly in this mode for
long without bots enforcing it, so remove support from GN too.
Change-Id: Ie86f0cbf2f5f859ac2ddb869da7e5b8d31b33fa0
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/237195
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Mike Klein <mtklein@google.com>
This makes dependent repos (eg. lottie-ci) significantly less hacky.
Bug: skia:9259
Change-Id: I8a933a927608d96e59a017dd7ef8e52d1d828a4d
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/227438
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Bug: skia:
Change-Id: I125466990b01b2fb1c9c30b2437a6bac134dda96
Reviewed-on: https://skia-review.googlesource.com/c/189302
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
To get the gold images out of the browser tests, this adds
testReporter.js and pathkit_aggregator.go. testReporter bundles
up the output as a base64 encoded PNG and sends it over the local
network to pathkit_aggregator. pathkit_aggregator will keep
a list of test results reported in this way and write the PNGs
to /OUT of the container (which is the swarming output directory).
Finally, after all the tests are run, the helper script "test_pathkit.sh"
makes a POST request that creates the JSON file that gold expects
(following the schema https://github.com/google/skia-buildbot/blob/master/golden/docs/INGESTION.md)
pathkit_aggregator takes many command line arguments which control
the keys that Gold needs in order to ingest and handle the data.
Of note, this creates a new set (i.e. source_type) of gold images
called "pathkit", which will distinguish it from "gm", "image", etc.
There will be at least 2 sub-sets of "pathkit" images, "canvas" and "svg",
(representing the 2 output types of PathKit). This CL doesn't
quite handle SVG yet, as it needs a way to convert SVG to PNG in the
browser and will be addressed in a follow up CL.
A "standard" gm is sized at 600x600. This was arbitrarily picked.
Note that the functions in testReporter.js return Promises based
on the fetch requests to post the data. This eliminates the race
condition between the /report_gold_data and /dump_json since
running the karma tests won't return until all reports are done.
Other changes of note:
- Adds go to karma-chrome-tests container.
- renames recipe_modules/build/wasm.py -> pathkit.py to be consistent with
the name of test_pathkit.py and make for easier grepping.
- Increases the JS test timeout to 10s (up from 5) to hopefully avoid
the flakes seen in the Debug Test.
Bug: skia:8216
Change-Id: Ic2cad54f3d19cc16601cf2e9a87798db1e6887a2
Reviewed-on: https://skia-review.googlesource.com/147042
Reviewed-by: Stephan Altmueller <stephana@google.com>
This adds a few JS tests (see *.spec.test) and runs them
using Karma and a Docker image containing Karma, node, and
Chrome (currently at 68).
We only add a Debug Test here because the Release version
has some test helpers (e.g. Region) compiled out. If those
end up not mattering for tests, we can add in a Release version.
Bug: skia:8216
Change-Id: Ica6ab3a4f21688cfa175a90d42c2254d38e8fcf1
Reviewed-on: https://skia-review.googlesource.com/145723
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
This is necessary for lottie-ci so we can upload to different
buckets and fetch different hash files.
Bug: skia:
Change-Id: I98b4d551df34473e2b5fc04284b0de04fbc097f3
Reviewed-on: https://skia-review.googlesource.com/144840
Commit-Queue: Stephan Altmueller <stephana@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
In https://skia-review.googlesource.com/c/skia/+/143107, the Metal tasks
were added with a whitelist of one unit test, so there are no Gold
results to upload. Gold treats the empty results files as invalid, which
generates Trooper alerts.
No-Try: true
Change-Id: Ifcdfccefb173c5cb970d60da5b7c3a89d6bc15db
Reviewed-on: https://skia-review.googlesource.com/143500
Commit-Queue: Stephan Altmueller <stephana@google.com>
Reviewed-by: Stephan Altmueller <stephana@google.com>
Reviewed-by: Greg Daniel <egdaniel@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
Add NEO driver to opencl_linux asset.
Bug: skia:8081
Change-Id: Ic36c670f3cabd73119845e94fcb2f54525e83443
Reviewed-on: https://skia-review.googlesource.com/138264
Commit-Queue: Mike Klein <mtklein@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
- adds building the testlab driver (run_testlab) as a separate step
- adds gcloud isolate necessary to run testlab
- adds Testlab support and uploading a verified AKP to GCS (with
meta data attached).
Bug: skia:
Change-Id: I1bf265f46c99360eb3a9eb684886f93de48085fe
Reviewed-on: https://skia-review.googlesource.com/111603
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Commit-Queue: Stephan Altmueller <stephana@google.com>
Bug: skia:
Change-Id: I31c8a625e2b4ed30a8cf0f03035b311ca1c21f88
Reviewed-on: https://skia-review.googlesource.com/63060
Commit-Queue: Yuqian Li <liyuqian@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
This simply uploads the results of an LLVM coverage to GCS
for later ingestion/display.
Bug: skia:7080
Change-Id: I7dcfa2307a239734a614990aca899ea37129126b
Reviewed-on: https://skia-review.googlesource.com/53880
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
This may help debug timeouts for
Test-Ubuntu-GCC-ShuttleA-GPU-GTX550Ti-x86_64-Release-Valgrind_PreAbandonGpuContext
Also stop uploading *AbandonGpuContext results to Gold.
No-Try: true
Change-Id: Ifa10d47cf27f86424d3936f451d190ee34f5637d
Reviewed-on: https://skia-review.googlesource.com/14525
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
BUG=skia:6070
Change-Id: I67c040239ca2cf77a7430aa1a45b1bb69fef3b91
Reviewed-on: https://skia-review.googlesource.com/8069
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
BUG=skia:6070
Change-Id: I02edcb0b1a98967e5d84f89a5ef578b36efadd8f
Reviewed-on: https://skia-review.googlesource.com/7825
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
