...since we have so many of those machines (which were recently racked).
Change-Id: I61baa5d7b5140633d707411ab3a7a9ecc376b042
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/506777
Reviewed-by: Ben Wagner <bungeman@google.com>
This CL adds the task driver and plumbing necessary to run CI tasks that upload build statistics to https://codesize.skia.org.
For now, a single CodeSize-dm-Debian10-Clang-x86_64-Release task is defined, which as the name suggests:
- Depends on Build-Debian10-Clang-x86_64-Release.
- Takes the "dm" binary built by the aforementioned job.
- Runs Bloaty against said binary.
- Uploads the Bloaty output, along with a JSON file with metadata, to the GCS bucket where https://codesize.skia.org gets its data from.
If one wishes to upload codesize statistics for another binary, the steps are as follows:
- Identify the target binary, for example "fm".
- Identify a compile task that builds said binary with the desired settings, for example "Build-Debian10-Clang-x86_64-Debug".
- Add a new job in //infra/bots/jobs.json named "CodeSize-<binary>-<compile task>", where <compile task> is the name of the compile task without the "Build-" prefix, for example "CodeSize-fm-Debian10-Clang-x86_64-Debug".
- Run "make train" from the //infra/bots directory.
Eventually the codesize.skia.org webserver will automatically pick up the output of any such CodeSize-* task and show it in the UI, with no additional steps needed.
One caveat is that the binary file name, and therefore the "<binary>" part of the task name, cannot contain dashes. This is due to how //infra/bots/gen_tasks_logic.go works. See comments in said Go file for ideas on how to work around this if we ever need to.
Bug: skia:12151
Change-Id: If406944ca7660c4dd15c8e6b8f34e48c65cbbe2f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/502788
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Reviewed-by: Eric Boren <borenet@google.com>
Commit-Queue: Leandro Lovisolo <lovisolo@google.com>
This is a reland of 3225c8cc46
Original change's description:
> Add kR8_unorm_SkColorType
>
> Change-Id: I97b5bc7f90715664f233ca7b7c41c0ecbfc29ac4
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/505679
> Reviewed-by: Michael Ludwig <michaelludwig@google.com>
> Reviewed-by: Greg Daniel <egdaniel@google.com>
> Commit-Queue: Brian Osman <brianosman@google.com>
Change-Id: I73fa17625d57e0e58da1b70e2e59ba200383cfe7
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/506460
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Brian Osman <brianosman@google.com>
...based on conversation with jvanverth@ on 2022-02-04 in Skia GChat.
Change-Id: I463737e33e87f2052326f6231781a801ad5c320c
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/506877
Reviewed-by: Jim Van Verth <jvanverth@google.com>
Commit-Queue: Erik Rose <erikrose@google.com>
Change-Id: I97b5bc7f90715664f233ca7b7c41c0ecbfc29ac4
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/505679
Reviewed-by: Michael Ludwig <michaelludwig@google.com>
Reviewed-by: Greg Daniel <egdaniel@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
GPUs that failed continued to fail when I put in error bars like
`distance(a, b) <= 0.001`, so they're just disabled entirely now.
Presumably their results are very busted.
Change-Id: I0f1b80f661563a20630740f8cfb6ef69f2a47934
Bug: skia:11209, skia:12858
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503817
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: John Stiles <johnstiles@google.com>
This is a reland of e138730000
I fixed the tryjobs to move the compiled files into the right
location.
Original change's description:
> [pathkit] Reorganize to be more like CanvasKit
>
> Change-Id: I64226b57161f86277e0635592c2c11442888335a
> Bug: skia:8216
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/502817
> Reviewed-by: Nathaniel Nifong <nifong@google.com>
Bug: skia:8216
Change-Id: I535f906b43816d3143e95ff9350ba20b998d817d
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503826
Reviewed-by: Nathaniel Nifong <nifong@google.com>
These Minis have a new CPU (I5-4278U) we've never cared about before, so
add that too. (We had some other Minis of the same model in the Golo,
but we didn't care about their CPUs because we were running only GPU
jobs on them.)
If you want to see a good diff of those, use FileMerge or something that
uses a similar algorithm; git makes a total hash of it.
Change-Id: Id7c4f6bc06f83d2ed4a0034e9174b9e69d3d15b2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/495424
Reviewed-by: Ben Wagner <bungeman@google.com>
Commit-Queue: Erik Rose <erikrose@google.com>
This is a reland of 805acda3f3
It fixes the #if SK_GL which was causing the Android roll
to fail.
This disables unit tests on Test-Ubuntu18-Clang-Golo-GPU-QuadroP400-x86_64-Release-All-TSAN_Vulkan
which were consistently crashing with OOM.
Original change's description:
> [fuzzer] Remove GL from (now-Vulkan) build
>
> The fuzzer runs against the Vulkan version of Swiftshader.
> There are no libGL.so (etc) on the fuzz runtime, so we
> want to avoid linking against those.
>
> The GL code that is #ifdef'd out is still necessary to
> avoid timeouts on TSAN with our NVIDIA jobs.
> https://skia-review.googlesource.com/c/skia/+/502638
>
> Procedure for testing this locally (and iterating):
> 1. In oss-fuzz checkout, run
> `python infra/helper.py shell skia`
> to pull up local interactive version of Docker
> fuzzer build image.
> 2. Run `compile` in fuzzer shell. Stop after
> the swiftshader compiles and is copied into /out
> with Ctrl + C.
> 3. Comment out the swiftshader compilation part [1]
> (no need to re-do this when modifying Skia code).
> `apt-get install nano -y`
> `nano ../build.sh`
> 4. Make change to Skia repo using normal methods.
> 5. Run the following in the Skia repo
> `git diff origin main > foo.patch`
> Copy the patch into the Docker shell using Ctrl+C
> and nano.
> 6. Apply the patch inside the Docker shell
> `git apply foo.patch`
> and re-compile (which should skip right to
> building the fuzzer libs)
> `compile`
> 7. Repeat 4-7 or make small changes directly in
> the Docker shell via nano.
> 8. When compilation and link succeeds, run
> `ldd /out/api_mock_gpu_canvas`
> to verify GL and friends were not dynamically linked.
>
> [1] https://github.com/google/oss-fuzz/pull/7214/files#diff-76f13875e33875cdd372f1f0933206be599cd87952f1bd1eaa57ca928ee9e3e1R49-R53
>
> Change-Id: Idf569820527c1304b0e5a68fd36295be89dfa2a0
> Bug: oss-fuzz:44132
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503016
> Reviewed-by: Brian Osman <brianosman@google.com>
> Commit-Queue: Kevin Lubick <kjlubick@google.com>
Bug: oss-fuzz:44132, skia:12900
Change-Id: Ia2eff9403b0035e7f86098f296d7d9b1bbfd4876
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503716
Reviewed-by: John Stiles <johnstiles@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
I have trouble remembering how to interpret the skip
strings, so I documented it more and required the use of
four named arguments. Regex helped transform the old calls
into the new ones.
Change-Id: I2b970053b95b9a4d3d90a718b5790b0b25ce8dae
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503736
Reviewed-by: Eric Boren <borenet@google.com>
This reverts commit 805acda3f3.
Reason for revert: Looks to be breaking the Android roll. See https://android-build.googleplex.com/builds/pending/P29733268/aosp_bramble-userdebug/latest/view/logs/build_error.log
Original change's description:
> [fuzzer] Remove GL from (now-Vulkan) build
>
> The fuzzer runs against the Vulkan version of Swiftshader.
> There are no libGL.so (etc) on the fuzz runtime, so we
> want to avoid linking against those.
>
> The GL code that is #ifdef'd out is still necessary to
> avoid timeouts on TSAN with our NVIDIA jobs.
> https://skia-review.googlesource.com/c/skia/+/502638
>
> Procedure for testing this locally (and iterating):
> 1. In oss-fuzz checkout, run
> `python infra/helper.py shell skia`
> to pull up local interactive version of Docker
> fuzzer build image.
> 2. Run `compile` in fuzzer shell. Stop after
> the swiftshader compiles and is copied into /out
> with Ctrl + C.
> 3. Comment out the swiftshader compilation part [1]
> (no need to re-do this when modifying Skia code).
> `apt-get install nano -y`
> `nano ../build.sh`
> 4. Make change to Skia repo using normal methods.
> 5. Run the following in the Skia repo
> `git diff origin main > foo.patch`
> Copy the patch into the Docker shell using Ctrl+C
> and nano.
> 6. Apply the patch inside the Docker shell
> `git apply foo.patch`
> and re-compile (which should skip right to
> building the fuzzer libs)
> `compile`
> 7. Repeat 4-7 or make small changes directly in
> the Docker shell via nano.
> 8. When compilation and link succeeds, run
> `ldd /out/api_mock_gpu_canvas`
> to verify GL and friends were not dynamically linked.
>
> [1] https://github.com/google/oss-fuzz/pull/7214/files#diff-76f13875e33875cdd372f1f0933206be599cd87952f1bd1eaa57ca928ee9e3e1R49-R53
>
> Change-Id: Idf569820527c1304b0e5a68fd36295be89dfa2a0
> Bug: oss-fuzz:44132
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503016
> Reviewed-by: Brian Osman <brianosman@google.com>
> Commit-Queue: Kevin Lubick <kjlubick@google.com>
Bug: oss-fuzz:44132
Change-Id: I3832417c60ff425572717d37dc9609419922b18e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503351
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Brian Osman <brianosman@google.com>
These tests are not guaranteed to pass in ES2 (as ES2 specifically does
not require NaN to be implemented), so they are now run in GPU+ES3 and
on the CPU.
Change-Id: Ica78e15a06b3b00b651c5fabd0decf751853a83f
Bug: skia:12858
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/501238
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: John Stiles <johnstiles@google.com>
The fuzzer runs against the Vulkan version of Swiftshader.
There are no libGL.so (etc) on the fuzz runtime, so we
want to avoid linking against those.
The GL code that is #ifdef'd out is still necessary to
avoid timeouts on TSAN with our NVIDIA jobs.
https://skia-review.googlesource.com/c/skia/+/502638
Procedure for testing this locally (and iterating):
1. In oss-fuzz checkout, run
`python infra/helper.py shell skia`
to pull up local interactive version of Docker
fuzzer build image.
2. Run `compile` in fuzzer shell. Stop after
the swiftshader compiles and is copied into /out
with Ctrl + C.
3. Comment out the swiftshader compilation part [1]
(no need to re-do this when modifying Skia code).
`apt-get install nano -y`
`nano ../build.sh`
4. Make change to Skia repo using normal methods.
5. Run the following in the Skia repo
`git diff origin main > foo.patch`
Copy the patch into the Docker shell using Ctrl+C
and nano.
6. Apply the patch inside the Docker shell
`git apply foo.patch`
and re-compile (which should skip right to
building the fuzzer libs)
`compile`
7. Repeat 4-7 or make small changes directly in
the Docker shell via nano.
8. When compilation and link succeeds, run
`ldd /out/api_mock_gpu_canvas`
to verify GL and friends were not dynamically linked.
[1] https://github.com/google/oss-fuzz/pull/7214/files#diff-76f13875e33875cdd372f1f0933206be599cd87952f1bd1eaa57ca928ee9e3e1R49-R53
Change-Id: Idf569820527c1304b0e5a68fd36295be89dfa2a0
Bug: oss-fuzz:44132
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/503016
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
This required an update to emscripten, due to there being
a few bug fixes regarding the non-dynamic code and the
Closure compiler/minifier.
Change-Id: Icc922bd98cdd52a6923a9367da3747dac2b897b3
Bug: skia:12795
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/492916
Reviewed-by: Nathaniel Nifong <nifong@google.com>
Change-Id: Ia218fb2249abd479db9d27527b965fd0b8ad3367
Bug: skia:12858
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/501276
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: John Stiles <johnstiles@google.com>
Auto-Submit: John Stiles <johnstiles@google.com>
This should give us access to newer system headers. We are upgrading to
C++17, and in iOS 9, the <optional> header is buried in
<experimental/optional>.
Change-Id: I3e4d4861ddf06c6e02939f1c97857d9617284428
Bug: skia:12882
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/502305
Reviewed-by: Erik Rose <erikrose@google.com>
Auto-Submit: John Stiles <johnstiles@google.com>
Commit-Queue: John Stiles <johnstiles@google.com>
Test "InlinerHonorsGLSLOutParamSemantics" was failing on Wembley devices
and is now disabled on that GPU.
Also, it turns out that the inliner has ignored functions with out
params for a long time now, but our test names haven't been updated to
account for this. So, did some additional cleanup:
- "InlinerHonorsGLSLOutParamSemantics" (the test in question) has been
moved to shared/ and renamed to "OutParamsAreDistinct."
- Removed test "OutParamsNoInline" as it is functionally the same as
"OutParams".
Change-Id: I1431ed197b9216cb482eee4f5e4eb2579a5303f7
Bug: skia:12858
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/502303
Auto-Submit: John Stiles <johnstiles@google.com>
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Change-Id: If0ecc3c1080b8e44f17dbb34ff6cc1d66794ae0b
Bug: skia:11052, skia:11919, skia:12858
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/501843
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: John Stiles <johnstiles@google.com>
Auto-Submit: John Stiles <johnstiles@google.com>
Removed a special case from `writeFunctionCallArgument` which avoided
using a scratch variable for out params; now we always use the scratch
variable and copy it back to the original variable at the end.
Change-Id: I0e446a3fde6d19554943384210bd911f6f9c8cfa
Bug: skia:11052, skia:11919, skia:12858
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/501836
Auto-Submit: John Stiles <johnstiles@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
My hope is that with Bazel, we will be able to remove
this CIPD package in favor of a properly maintained/created
toolchain.
Change-Id: I6a7b5013da22395511d828bee857777fbe0bcd8c
Bug: skia:12878
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/502289
Reviewed-by: John Stiles <johnstiles@google.com>
Change-Id: I077e504be5aa0aaca615a83ac0b984ea9f651e6e
Bug: skia:12858
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/501478
Auto-Submit: John Stiles <johnstiles@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
This shakes out a few driver issues.
Change-Id: I35041a904fb762a5b933b3e970f2d7a668803dcd
Bug: skia:12858, skia:11919
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/501019
Auto-Submit: John Stiles <johnstiles@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: John Stiles <johnstiles@google.com>
This shook out a few driver issues.
Change-Id: If12f11c4b3b562fec402bddce25edf01a8be83b9
Bug: skia:12858, skia:11919
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/500816
Auto-Submit: John Stiles <johnstiles@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: John Stiles <johnstiles@google.com>
This hits a Vulkan driver bug in pipeline creation on older OSes.
Change-Id: I09016f1a779e4440a1893323db2d27f171132e86
Bug: skia:12869
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/499979
Auto-Submit: John Stiles <johnstiles@google.com>
Reviewed-by: Michael Ludwig <michaelludwig@google.com>
Commit-Queue: Michael Ludwig <michaelludwig@google.com>
The SwiftShader OpenGL frontend is officially unsupported. They strictly
support their Vulkan frontend only.
Bug: skia:12218
Bug: skia:12820
Bug: skia:12826
Bug: skia:12829
Change-Id: Ib2e43783e18fcabfa2df9268c29bcd610350e5a2
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/437149
Reviewed-by: Greg Daniel <egdaniel@google.com>
Commit-Queue: Christopher Dalton <csmartdalton@google.com>
Our ES2-fallback switch rewrite doesn't work well on Tegra3; it doesn't
always like breaks inside of for loops.
Change-Id: Iaaef0b7d0a94ce4fc954529e39f0810a6a34101c
Bug: skia:12477, skia:12828
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/494976
Auto-Submit: John Stiles <johnstiles@google.com>
Reviewed-by: Michael Ludwig <michaelludwig@google.com>
Commit-Queue: Michael Ludwig <michaelludwig@google.com>
* Using docker.New instead of docker.Login which needed configDir passed around.
* Also cleaned up unneeded references to infraCommonEnv and infraCommonBuildArgs.
Bug: skia:12812
Change-Id: Ie60ad77d75a5a17f9350e0028afb8c54bf4c63cb
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/494581
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
* Add support for fiddler's bazel build in the existing push_apps_from_skia_image task driver. After api is migrated to bazel it will be cleaned up.
* Cleanup push_bazel_apps_from_wasm_image.go by using utils from buildbot.git
* Make the two autodeployment bots be triggered only on the main branch.
Bug: skia:12788
Change-Id: Iaf2adaa5ba58a1992691bbc024558a0d7392f41d
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/493037
Reviewed-by: Kevin Lubick <kjlubick@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
Bug: skia:12813
Change-Id: I282d16d67624e14e5e02d69fdc18d729db81a16f
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/493856
Auto-Submit: Greg Daniel <egdaniel@google.com>
Reviewed-by: Michael Ludwig <michaelludwig@google.com>
Commit-Queue: Michael Ludwig <michaelludwig@google.com>
bazel run //example:hello_world --config=clang
causes a window to open and draws a circle and a square.
Text to follow in a future CL.
To make this work, I had to get rid of musl and use glibc.
All the shared libraries (.so files) that were pre-built
and available for download (e.g. from https://packages.debian.org/bullseye/amd64/libgl1/download)
were compiled against glibc. When I tried to run a
program statically linked with musl and dynamically linked
against things using glibc, I got a segmentation fault
on things like calloc().
Initial attempts to use glibc had failed because it was thought
that the libc.so.6 file could only be referred to by absolute
path (and thus Bazel would not be happy about it). As it turns out,
that was simply a misconfiguration of the builtin_sysroot
parameter to cc_common.create_cc_toolchain_config_info
(see //toolchain/clang_toolchain_config.bzl). By setting that
to `external/clang_linux_amd64` and not
`external/clang_linux_amd64/usr`, the libc binary which had
been extracted to `external/clang_linux_amd64/lib/x86_64-linux-gnu`
was perfectly reachable from
`external/clang_linux_amd64/usr/usr/lib/x86_64-linux-gnu/libc.so`
To bring in the shared libraries to link against (e.g. X11, GL)
I made build_toolchain.bzl easier to modify in that we simply need
to add a debian download url and sha256 hash to a list (rather than
having to plumb this through via arguments).
Recommended Review Order:
- example/BUILD.bazel (not sure if we always want to set bare
link arguments like that or if we want to use "features" to
pass those along to the toolchain).
- tools/sk_app/BUILD.bazel to see initial cc_library for
wrapping sk_app code.
- toolchain/build_toolchain.bzl to see removal of musl and
new list of debs.
- toolchain/clang_toolchain_config.bzl (where use of the
no-canonical-prefixes was key to compilation success).
Notice also that we statically linked libc++ (I did not
have any shared libraries for it locally, so I guessed
a typical developer might not either).
- Rest of toolchain/ for trivial renames.
- bazel/Makefile to see extra docs on those targets and
a new target that compiles all the exes so far for a
quick way to test the build.
- third_party/BUILD.bazel and src/gpu/BUILD.bazel which have
non-generated changes. (all other BUILD.bazel files do).
- go.mod, which needed to update the infra repo version in
order to pick up http://review.skia.org/491736).
Change-Id: I8687bd227353040eca2dffa9465798d8bd395027
Bug: skia:12541
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/492117
Reviewed-by: Ben Wagner <bungeman@google.com>
Reviewed-by: Leandro Lovisolo <lovisolo@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
Bug: b/198566237
If the RLE data exceeds the width of the image, treat it as an incorrect
value in the number of pixels in the run. This allows decoding images
that are supported by other decoders (e.g. Mac Preview).
Add a test image to skimage tests.
Change-Id: I10955cbde3e28accf670417fa154a536c0732e6a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/488296
Reviewed-by: Derek Sollenberger <djsollen@google.com>
Commit-Queue: Leon Scroggins <scroggo@google.com>
Failures in this bot are easy to miss since it runs infrequently
(nightly/weekly).
This change reports build and asset creation successes/failures to new
metrics. Alerts will be created from these metrics.
Bug: skia:12759
Change-Id: I07db58dcd5ee1929e16717cc7be66062f30d3670
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/487976
Reviewed-by: Joe Gregorio <jcgregorio@google.com>
Commit-Queue: Ravi Mistry <rmistry@google.com>
