To get the gold images out of the browser tests, this adds
testReporter.js and pathkit_aggregator.go. testReporter bundles
up the output as a base64 encoded PNG and sends it over the local
network to pathkit_aggregator. pathkit_aggregator will keep
a list of test results reported in this way and write the PNGs
to /OUT of the container (which is the swarming output directory).
Finally, after all the tests are run, the helper script "test_pathkit.sh"
makes a POST request that creates the JSON file that gold expects
(following the schema https://github.com/google/skia-buildbot/blob/master/golden/docs/INGESTION.md)
pathkit_aggregator takes many command line arguments which control
the keys that Gold needs in order to ingest and handle the data.
Of note, this creates a new set (i.e. source_type) of gold images
called "pathkit", which will distinguish it from "gm", "image", etc.
There will be at least 2 sub-sets of "pathkit" images, "canvas" and "svg",
(representing the 2 output types of PathKit). This CL doesn't
quite handle SVG yet, as it needs a way to convert SVG to PNG in the
browser and will be addressed in a follow up CL.
A "standard" gm is sized at 600x600. This was arbitrarily picked.
Note that the functions in testReporter.js return Promises based
on the fetch requests to post the data. This eliminates the race
condition between the /report_gold_data and /dump_json since
running the karma tests won't return until all reports are done.
Other changes of note:
- Adds go to karma-chrome-tests container.
- renames recipe_modules/build/wasm.py -> pathkit.py to be consistent with
the name of test_pathkit.py and make for easier grepping.
- Increases the JS test timeout to 10s (up from 5) to hopefully avoid
the flakes seen in the Debug Test.
Bug: skia:8216
Change-Id: Ic2cad54f3d19cc16601cf2e9a87798db1e6887a2
Reviewed-on: https://skia-review.googlesource.com/147042
Reviewed-by: Stephan Altmueller <stephana@google.com>
This adds a few JS tests (see *.spec.test) and runs them
using Karma and a Docker image containing Karma, node, and
Chrome (currently at 68).
We only add a Debug Test here because the Release version
has some test helpers (e.g. Region) compiled out. If those
end up not mattering for tests, we can add in a Release version.
Bug: skia:8216
Change-Id: Ica6ab3a4f21688cfa175a90d42c2254d38e8fcf1
Reviewed-on: https://skia-review.googlesource.com/145723
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
They crash, presumably because they use multiple large images.
Change-Id: I277958cd6661820085b4e1a04e0668e3a3e9fddc
Reviewed-on: https://skia-review.googlesource.com/145880
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Brian Salomon <bsalomon@google.com>
This contains the Go code dependencies of our infra repo and will be
used to insulate us from upstream changes which break us due to API
changes, etc.
Bug: skia:
Change-Id: I6ce36a4e41b6d7686fe2598ba980640854cd4f11
Reviewed-on: https://skia-review.googlesource.com/145158
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ravi Mistry <rmistry@google.com>
Change-Id: I88ee8200b82d12ba8cee637e01b360ce2d2bb2b3
Reviewed-on: https://skia-review.googlesource.com/145360
Auto-Submit: Brian Salomon <bsalomon@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Salomon <bsalomon@google.com>
Change-Id: I0eb3dcc4d38eb2aeab22d1df1515e499cf21dc62
Reviewed-on: https://skia-review.googlesource.com/144608
Commit-Queue: Eric Boren <borenet@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
depot_tools:
https://crrev.com/5300616351f523c605de80f5ece8c29321a78847 bot_update: expose patch_refs as an option to ensure_checkout. (jbudorick@chromium.org)
R=borenet@google.com
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I3719ebfe726d6662c0e3b8a9f4e21ca62000a292
Reviewed-on: https://skia-review.googlesource.com/145144
Reviewed-by: John Budorick <jbudorick@chromium.org>
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
This CL prepares for installing Xcode in a Swarming cache dir from a
CIPD package. https://skia-review.googlesource.com/c/skia/+/144608
Change-Id: I123a409f97397cc0e87de0dcd05d7ea08d3c0524
Reviewed-on: https://skia-review.googlesource.com/144903
Commit-Queue: Eric Boren <borenet@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
This is necessary for lottie-ci so we can upload to different
buckets and fetch different hash files.
Bug: skia:
Change-Id: I98b4d551df34473e2b5fc04284b0de04fbc097f3
Reviewed-on: https://skia-review.googlesource.com/144840
Commit-Queue: Stephan Altmueller <stephana@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
In https://skia-review.googlesource.com/c/skia/+/143107, the Metal tasks
were added with a whitelist of one unit test, so there are no Gold
results to upload. Gold treats the empty results files as invalid, which
generates Trooper alerts.
No-Try: true
Change-Id: Ifcdfccefb173c5cb970d60da5b7c3a89d6bc15db
Reviewed-on: https://skia-review.googlesource.com/143500
Commit-Queue: Stephan Altmueller <stephana@google.com>
Reviewed-by: Stephan Altmueller <stephana@google.com>
Reviewed-by: Greg Daniel <egdaniel@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
It is unused, is becoming a maintainence burden and source of bugs,
and takes up a lot of time on the *SAN bots.
Change-Id: If383eb6e4838ca23140f9e16d518b1bfc655fa12
Reviewed-on: https://skia-review.googlesource.com/143307
Auto-Submit: Mike Klein <mtklein@google.com>
Commit-Queue: Herb Derby <herb@google.com>
Reviewed-by: Herb Derby <herb@google.com>
This is mainly to debug TSAN issues.
I'm not exactly sure why we run this config on ASAN and MSAN bots.
This CL will disable them along with TSAN, and depending on how
much time that saves on the bots, I might only turn TSAN back on
in the end.
Change-Id: Id4d92f27a917b4e14b96c25e97e8be70ca2004f5
Reviewed-on: https://skia-review.googlesource.com/143302
Commit-Queue: Mike Klein <mtklein@chromium.org>
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
Auto-Submit: Mike Klein <mtklein@chromium.org>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
This reverts commit e3e68cf14f.
Reason for revert: The test was hanging due to https://skia-review.googlesource.com/c/skia/+/142102, which has been reverted.
Original change's description:
> Blacklist hanging test.
>
> Blacklist gr_mipmap_build_511x512 for nanobench on the Acer Chromebook
> R13 Convertible to avoid timeouts for Release build.
>
> Bug: skia:8187
> No-Try: true
> Change-Id: Ic63fb705a6a6cc6576978c96a8dd5206dc59ba0d
> Reviewed-on: https://skia-review.googlesource.com/143162
> Commit-Queue: Brian Osman <brianosman@google.com>
> Auto-Submit: Ben Wagner <benjaminwagner@google.com>
> Reviewed-by: Brian Osman <brianosman@google.com>
TBR=benjaminwagner@google.com,brianosman@google.com
Change-Id: Id5144f1ada6d2059e473a2891816775ea7fe2663
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: skia:8187
Reviewed-on: https://skia-review.googlesource.com/143320
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
Blacklist gr_mipmap_build_511x512 for nanobench on the Acer Chromebook
R13 Convertible to avoid timeouts for Release build.
Bug: skia:8187
No-Try: true
Change-Id: Ic63fb705a6a6cc6576978c96a8dd5206dc59ba0d
Reviewed-on: https://skia-review.googlesource.com/143162
Commit-Queue: Brian Osman <brianosman@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Currently the test bot is set up with a whitelist that only runs one test.
Bug: skia:
Change-Id: Ief6a978a527cf581929cafe0f4cd002c35b24235
Reviewed-on: https://skia-review.googlesource.com/143107
Commit-Queue: Greg Daniel <egdaniel@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Bug: skia:7376
No-Try: true
Change-Id: I0f79e6a1e07103769f3205f069b84e932c96bd64
Reviewed-on: https://skia-review.googlesource.com/142983
Commit-Queue: Brian Osman <brianosman@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
This bot occasionally produces empty GM images. We believe this is due
to TSAN's thread scheduling interacting poorly with threaded GL drivers.
gltestpersistentcache fails if the cold cache and hot cache images don't
match and this flakiness causes it to fail intermittently.
Change-Id: Ib914c547120e05fac921d7e70de8e36a53bfb61b
Reviewed-on: https://skia-review.googlesource.com/142591
Commit-Queue: Brian Salomon <bsalomon@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Auto-Submit: Brian Salomon <bsalomon@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Bug: skia:8136
Change-Id: I18c4ad549c52346ebfe23d172597d5da205e5c4d
Reviewed-on: https://skia-review.googlesource.com/142105
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
* Output fiddle.json
* Do not send to CQ while we investigate
Bug: skia:8151
Change-Id: I309c732cc6b7db72054fd119b8c68dd32a765d6f
Reviewed-on: https://skia-review.googlesource.com/142301
Commit-Queue: Ravi Mistry <rmistry@google.com>
Reviewed-by: Cary Clark <caryclark@google.com>
The config fails to initialized in push_sinks because readbacks in test
draw don't match (as the GrContext is already in abandoned mode).
Change-Id: I73c68ab52df3e83b688ab43fbe74cf4c1ae8c19d
Reviewed-on: https://skia-review.googlesource.com/141562
Auto-Submit: Brian Salomon <bsalomon@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Change-Id: I181ed6d294774a9c6886fde8252bfdde99adc572
Reviewed-on: https://skia-review.googlesource.com/140982
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Salomon <bsalomon@google.com>
All Golo devices except 2 are AWOL. We added 6 to the Skolo, but they
can't keep up.
This CL moves the following jobs off Nexus5x:
- All ASAN jobs to Pixel. Most other devices we have can't be rooted,
so they can't run ASAN.
- Vulkan NoGPUThreads arm64 jobs to NVIDIA_Shield. Of the devices that
support Vulkan, this has the most available capacity.
- I removed the arm32 Vulkan NoGPUThreads jobs because it seems a
bit esoteric.
- Also removed arm32 Vulkan Debug Perf because I can't imagine it
would ever give us info that arm64 doesn't already provide. (We
ignore Perf data from Debug builds.)
- Other arm32 Vulkan jobs stayed on Nexus5x because we don't have
any other Vulkan-capable devices with spare capacity.
- Most arm32 jobs to MotoG4 and de-duplicated. This device has an
ARMv8 processor but runs everything in arm32 mode, which is exactly
what these jobs are trying to test. We have a bunch of MotoG4s in
the Skolo; however, they don't support Vulkan or ASAN.
- NoGPUThreads and NativeFonts arm64 jobs to GalaxyS6. There are only
6 in the Skolo, but for some reason these devices are particularly
fast. They also don't support Vulkan or ASAN.
- Currently GalaxyS6 CPU arm64 jobs fail, possibly due to OOM. I
will try to move those jobs in a future CL.
Sorted jobs.json.
Change-Id: If02584a9c04906f4b9c31b4ba3f1da0289a5b8a9
Reviewed-on: https://skia-review.googlesource.com/141281
Commit-Queue: Joe Gregorio <jcgregorio@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Add NEO driver to opencl_linux asset.
Bug: skia:8081
Change-Id: Ic36c670f3cabd73119845e94fcb2f54525e83443
Reviewed-on: https://skia-review.googlesource.com/138264
Commit-Queue: Mike Klein <mtklein@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
This reverts commit d5161b9a62.
Reason for revert: This should be okay again.
Original change's description:
> Reland "Revert "Update GPU DM/nanobench configs for color testing""
>
> This reverts commit c0fb3c8b9f.
>
> Reason for revert: Had to revert dependent CL.
> Original change's description:
> > Revert "Revert "Update GPU DM/nanobench configs for color testing""
> >
> > This reverts commit 2a284de7a5.
> >
> > Re-landing without esrgb configs. Going to enable those separately,
> > as they require some blacklisting.
> >
> > Bug: skia:
> > Change-Id: I2315849d4f5c45c2e5512f84f9f867d02ff6fe30
> > Reviewed-on: https://skia-review.googlesource.com/136589
> > Reviewed-by: Brian Osman <brianosman@google.com>
> > Commit-Queue: Brian Osman <brianosman@google.com>
>
> TBR=mtklein@google.com,bsalomon@google.com,brianosman@google.com
>
> Change-Id: I942d4235c17100fd933bf9cbfe923e3ae80657b1
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: skia:
> Reviewed-on: https://skia-review.googlesource.com/137721
> Reviewed-by: Brian Osman <brianosman@google.com>
> Commit-Queue: Brian Osman <brianosman@google.com>
TBR=mtklein@google.com,bsalomon@google.com,brianosman@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: skia:
Change-Id: I280dac49eb74d6c1cbf7c2f52a3f6a68fa2510ae
Reviewed-on: https://skia-review.googlesource.com/138301
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
We updated them to use Clang a while ago but forgot to rename the bots.
Change-Id: I01c353b48d5aff7a30d63d382bdf6bdfb3e0b196
Reviewed-on: https://skia-review.googlesource.com/137887
Auto-Submit: Mike Klein <mtklein@chromium.org>
Reviewed-by: Kevin Lubick <kjlubick@google.com>
This reverts commit c0fb3c8b9f.
Reason for revert: Had to revert dependent CL.
Original change's description:
> Revert "Revert "Update GPU DM/nanobench configs for color testing""
>
> This reverts commit 2a284de7a5.
>
> Re-landing without esrgb configs. Going to enable those separately,
> as they require some blacklisting.
>
> Bug: skia:
> Change-Id: I2315849d4f5c45c2e5512f84f9f867d02ff6fe30
> Reviewed-on: https://skia-review.googlesource.com/136589
> Reviewed-by: Brian Osman <brianosman@google.com>
> Commit-Queue: Brian Osman <brianosman@google.com>
TBR=mtklein@google.com,bsalomon@google.com,brianosman@google.com
Change-Id: I942d4235c17100fd933bf9cbfe923e3ae80657b1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: skia:
Reviewed-on: https://skia-review.googlesource.com/137721
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
This reverts commit 2a284de7a5.
Re-landing without esrgb configs. Going to enable those separately,
as they require some blacklisting.
Bug: skia:
Change-Id: I2315849d4f5c45c2e5512f84f9f867d02ff6fe30
Reviewed-on: https://skia-review.googlesource.com/136589
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
This reverts commit 5e11777a06.
Reason for revert: Bots with no F16 support failing (as expected).
Original change's description:
> Update GPU DM/nanobench configs for color testing
>
> - Rename '*srgbnl' to '*srgb'.
> - Add '*narrow' and '*enarrow' for testing a gamut narrower than sRGB.
> - Stopped blacklisting glessrgb. It no longer uses sRGB pixel configs,
> so should work fine on all GPUs.
> - Also runs 'esrgb' (F16 sRGB) everywhere. This is likely to fail on
> some devices that have poor (or no) F16 support.
>
> Change-Id: I941d0758adb3b0c895c7a8a85ff0ae4aa5bff6e7
> Reviewed-on: https://skia-review.googlesource.com/136171
> Commit-Queue: Brian Osman <brianosman@google.com>
> Reviewed-by: Brian Salomon <bsalomon@google.com>
> Reviewed-by: Mike Klein <mtklein@google.com>
TBR=mtklein@google.com,bsalomon@google.com,brianosman@google.com
Change-Id: I091269b93bc9e76c9d42f701d2018610b9c33a89
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://skia-review.googlesource.com/136300
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
- Rename '*srgbnl' to '*srgb'.
- Add '*narrow' and '*enarrow' for testing a gamut narrower than sRGB.
- Stopped blacklisting glessrgb. It no longer uses sRGB pixel configs,
so should work fine on all GPUs.
- Also runs 'esrgb' (F16 sRGB) everywhere. This is likely to fail on
some devices that have poor (or no) F16 support.
Change-Id: I941d0758adb3b0c895c7a8a85ff0ae4aa5bff6e7
Reviewed-on: https://skia-review.googlesource.com/136171
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Salomon <bsalomon@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
Bug: skia:6473
Change-Id: Iee2f195ddf4bbcdabc1580f2a021d2e9a07ff0b2
Reviewed-on: https://skia-review.googlesource.com/133441
Commit-Queue: Eric Boren <borenet@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
Bug: skia:6951
Change-Id: I094b542c6930e0789db71474650235b3ce991054
Reviewed-on: https://skia-review.googlesource.com/136022
Reviewed-by: Ravi Mistry <rmistry@google.com>
Commit-Queue: Eric Boren <borenet@google.com>
- Rename 'srgbnl' to just 'srgb'.
- Add 'narrow' and 'enarrow' for testing a gamut narrower than sRGB.
Tested by running xfermodes2 in DM... all look different, what a mess.
I also ran a few nanobenches and they seemed somewhat sane.
Change-Id: Iacdc391dc0eef4153a76f5b4f78d72c57a4371ee
Reviewed-on: https://skia-review.googlesource.com/135871
Commit-Queue: Mike Klein <mtklein@chromium.org>
Auto-Submit: Mike Klein <mtklein@chromium.org>
Reviewed-by: Brian Osman <brianosman@google.com>
We expect .BAT and .EXE to be on PATHEXT. Also, although Python
CreateProcess doesn't honor PATHEXT, it will try .EXE. This allows most
commands to be platform-independent, and also resolves the issue with
the cpython package not including python.bat.
No-Tree-Checks: true
Docs-Preview: https://skia.org/?cl=135626
Bug: chromium:852581
Change-Id: Iab4189407df44ff4ad4d37da07ff52414229d397
Reviewed-on: https://skia-review.googlesource.com/135626
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
Auto-Submit: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
depot_tools:
https://crrev.com/cd8d4a659a2b622f4cc2674896ace670c57e9ad3 gclient recipe: use env_suffixes insetad of custom ENV manipulation. (tandrii@chromium.org)
R=benjaminwagner@google.com
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I668725d3215e19b55b86900edb2813cade6a1dbc
Reviewed-on: https://skia-review.googlesource.com/134662
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
