Commit Graph

17 Commits

Author SHA1 Message Date
John Stiles
f2c2d3020e Migrate PersistentCacheUtils logic into a cpp file.
This code is not performance-sensitive, and there doesn't seem to be any
benefit to making it header-only.

Change-Id: Ic83be5cbb96f08c18a144954be7453aaabaa5a72
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/394900
Auto-Submit: John Stiles <johnstiles@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2021-04-12 14:09:59 +00:00
Brian Osman
9e1ef99d1e fm: Make --writeShaders dump fragment, vertex, and key files
Bug: skia:11372
Change-Id: Ib764b8d995655317e8968e57b1ea0528339d430b
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/379060
Reviewed-by: Mike Klein <mtklein@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
2021-03-03 20:10:06 +00:00
Brian Osman
f0de96f7b8 Reland "Redesign program key construction"
This is a reland of bbbf1a7f50

Original change's description:
> Redesign program key construction
>
> This does two things:
> 1) Moves responsibility for bit-packing portions of the key into the key
>    itself. A new GrKeyBuilder type manages adding bits, with asserts to
>    ensure a value always fits in the requested number. In theory this
>    will let us generate smaller keys overall, at the expense of slightly
>    more complex code during construction.
> 2) Adds a string label parameter for key methods that fold in data. For
>    new methods, the label is required. To ease migration, the old add32
>    does not require a label (yet). This will let us generate detailed,
>    human readable keys, either based on SK_DEBUG, or a runtime option
>    (if we're comfortable paying the cost).
>
> Bug: skia:11372
> Change-Id: Ib0f941551e0dbadabbd2a7de912b00e9e766b166
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/377876
> Commit-Queue: Brian Osman <brianosman@google.com>
> Reviewed-by: Brian Salomon <bsalomon@google.com>

Bug: skia:11372
Cq-Include-Trybots: luci.skia.skia.primary:Test-Win10-MSVC-Golo-GPU-QuadroP400-x86_64-Debug-All-Vulkan
Change-Id: I179ed581bc9ba772191e727274ac0ac6979ebdf3
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/378778
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
2021-03-03 17:44:00 +00:00
Brian Osman
acf26501fb Revert "Redesign program key construction"
This reverts commit bbbf1a7f50.

Reason for revert: D3D Failures

Original change's description:
> Redesign program key construction
>
> This does two things:
> 1) Moves responsibility for bit-packing portions of the key into the key
>    itself. A new GrKeyBuilder type manages adding bits, with asserts to
>    ensure a value always fits in the requested number. In theory this
>    will let us generate smaller keys overall, at the expense of slightly
>    more complex code during construction.
> 2) Adds a string label parameter for key methods that fold in data. For
>    new methods, the label is required. To ease migration, the old add32
>    does not require a label (yet). This will let us generate detailed,
>    human readable keys, either based on SK_DEBUG, or a runtime option
>    (if we're comfortable paying the cost).
>
> Bug: skia:11372
> Change-Id: Ib0f941551e0dbadabbd2a7de912b00e9e766b166
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/377876
> Commit-Queue: Brian Osman <brianosman@google.com>
> Reviewed-by: Brian Salomon <bsalomon@google.com>

TBR=bsalomon@google.com,robertphillips@google.com,brianosman@google.com

Change-Id: I7bfb20905c87083e84a1ea21bc53d63e882e2c68
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: skia:11372
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/378777
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
2021-03-03 13:35:22 +00:00
Brian Osman
bbbf1a7f50 Redesign program key construction
This does two things:
1) Moves responsibility for bit-packing portions of the key into the key
   itself. A new GrKeyBuilder type manages adding bits, with asserts to
   ensure a value always fits in the requested number. In theory this
   will let us generate smaller keys overall, at the expense of slightly
   more complex code during construction.
2) Adds a string label parameter for key methods that fold in data. For
   new methods, the label is required. To ease migration, the old add32
   does not require a label (yet). This will let us generate detailed,
   human readable keys, either based on SK_DEBUG, or a runtime option
   (if we're comfortable paying the cost).

Bug: skia:11372
Change-Id: Ib0f941551e0dbadabbd2a7de912b00e9e766b166
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/377876
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Salomon <bsalomon@google.com>
2021-03-02 23:46:48 +00:00
Brian Osman
9e4e4c7e97 Reland "Switch persistent cache to use SkReadBuffer/SkWriteBuffer"
These are much safer than SkReader32/SkWriter32 (they do validation and
ensure we never read past the end of the buffer).

Where we used to just assert that the contents of the cache were valid,
we now validate everything, and fail gracefully by discarding the cache
contents if it's corrupted or invalid.

Reland includes a new skipByteArray API. The previous technique for
reading into an std::string relied on data(), which doesn't return a
writeable pointer until the C++17 standard library.

Bug: skia:9402
Change-Id: I3b88efbf8ca590c8ad4f8164f7c07eee12696ec6
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/295441
Reviewed-by: Brian Salomon <bsalomon@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
2020-06-10 14:55:37 +00:00
Mike Reed
90e82904b5 Revert "Switch persistent cache to use SkReadBuffer/SkWriteBuffer"
This reverts commit 5fa11d4040.

Reason for revert: breaking some compiles..

https://chromium-swarm.appspot.com/task?id=4cb41acadb936310

Original change's description:
> Switch persistent cache to use SkReadBuffer/SkWriteBuffer
> 
> These are much safer than SkReader32/SkWriter32 (they do validation and
> ensure we never read past the end of the buffer).
> 
> Where we used to just assert that the contents of the cache were valid,
> we now validate everything, and fail gracefully by discarding the cache
> contents if it's corrupted or invalid.
> 
> Bug: skia:9402
> Change-Id: Ib893681f97f9413c28744f11075dc2e392364db6
> Reviewed-on: https://skia-review.googlesource.com/c/skia/+/294998
> Reviewed-by: Brian Salomon <bsalomon@google.com>
> Commit-Queue: Brian Osman <brianosman@google.com>

TBR=egdaniel@google.com,jvanverth@google.com,bsalomon@google.com,brianosman@google.com

Change-Id: Iabea26cde82043e3f3a23cde81503ea3abdd8398
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: skia:9402
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/295394
Reviewed-by: Mike Reed <reed@google.com>
2020-06-10 00:19:45 +00:00
Brian Osman
5fa11d4040 Switch persistent cache to use SkReadBuffer/SkWriteBuffer
These are much safer than SkReader32/SkWriter32 (they do validation and
ensure we never read past the end of the buffer).

Where we used to just assert that the contents of the cache were valid,
we now validate everything, and fail gracefully by discarding the cache
contents if it's corrupted or invalid.

Bug: skia:9402
Change-Id: Ib893681f97f9413c28744f11075dc2e392364db6
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/294998
Reviewed-by: Brian Salomon <bsalomon@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
2020-06-09 22:34:03 +00:00
Brian Osman
43f443f086 Fix caching of GL program binaries
Our "header" reading and writing code didn't agree, so we always failed
to recognize cached program binaries. The asserts in the testing sink
failed to notice, because we did get a 100% cache hit rate, but we
immediately discarded the data we received.

We now also check that we didn't insert anything into the cache, as a
proxy for doing any shader compile work. That change, plus the tweak to
set cached=false when the header fields are invalid (like we do if we
encounter problems further in the blob) detected the problem. Adding the
version tag to the start of the encoded blob fixes the test, and means
that program binary caching is actually working again.

This code still looks (and is) fragile. The next CL is going to rewrite
things to use SkReadBuffer and SkWriteBuffer, make the parsing code less
brittle, and give us a more robust way to detect failure anywhere in the
stream.

Bug: skia:9402
Change-Id: I0329f088e0afce3998494d91ef2206e5eb9cac42
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/294599
Reviewed-by: Brian Salomon <bsalomon@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
2020-06-05 15:54:19 +00:00
Brian Osman
1facd5e8d1 Make persistent cache serialization safer
Add a version tag at the start, reject any cache blobs without it

Bug: chromium:1062018
Change-Id: I18bb668e6b836e1247640e9286a69a3e745babfd
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/277376
Reviewed-by: Brian Salomon <bsalomon@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
2020-03-17 13:52:47 +00:00
Brian Osman
a66081d943 Move SkSL caching out of GR_TEST_UTILS, trim persistent cache options
Combines the two boolean options into a single tri-state enum. Old GLSL
option is still present (temporarily) until Chrome is switched over.

Also add a type tag for cached program binaries, so we can safely
detect cache entries of the wrong type.

Change-Id: I0ddeefa9180b27bc2c46e2e7b77e6c9cdf4a730a
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/238856
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Salomon <bsalomon@google.com>
2019-09-03 20:18:48 +00:00
Brian Osman
a085a41b9d Normalize storage of shader strings in the cache
Adds a tag that can be used to verify the expected type
of shaders. My follow-up CL that adds SkSL editing to
Vulkan relies heavily on this.

Change-Id: Ifda420c2dcbaff07cdf1b8157d0ece02b1ab6c78
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/210262
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Brian Salomon <bsalomon@google.com>
2019-04-25 15:27:59 +00:00
Mike Klein
c0bd9f9fe5 rewrite includes to not need so much -Ifoo
Current strategy: everything from the top

Things to look at first are the manual changes:

   - added tools/rewrite_includes.py
   - removed -Idirectives from BUILD.gn
   - various compile.sh simplifications
   - tweak tools/embed_resources.py
   - update gn/find_headers.py to write paths from the top
   - update gn/gn_to_bp.py SkUserConfig.h layout
     so that #include "include/config/SkUserConfig.h" always
     gets the header we want.

No-Presubmit: true
Change-Id: I73a4b181654e0e38d229bc456c0d0854bae3363e
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/209706
Commit-Queue: Mike Klein <mtklein@google.com>
Reviewed-by: Hal Canary <halcanary@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
Reviewed-by: Florin Malita <fmalita@chromium.org>
2019-04-24 16:27:11 +00:00
Brian Osman
be2062c430 Extend --writeShaders to output SPIR-V
Changed the python script to operate on all shaders in the directory,
handling SPIR-V appropriately, and collating data across backends.
Made the output CSV, so that it imports into spreadsheets directly
(and handles missing columns cleanly).

Removed all the JSON digest logic - this was overkill at the moment,
and it made it tricky once we were combining information from GL and
Vulkan. Also, the hit count data was probably misleading - it didn't
count hits at the program cache level, just the persistent cache.

Change-Id: If354cde943c96f84e7bcc20a137afefca3b59358
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/207960
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Greg Daniel <egdaniel@google.com>
2019-04-15 14:04:18 +00:00
Brian Osman
e6ef03d53b Make the persistent cache utility functions operate on SkData
This avoids boilerplate at every call-site (and more are being added
soon) to set up the reader/writer objects.

Change-Id: Idc632932b5a9be8d3164d249abab7dccbb905ca3
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/207704
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
2019-04-12 15:00:54 +00:00
Brian Osman
5aa11fb67a Shader serialization experiment (with Mali Offline Compiler analysis)
1) Adds a --writeShaders option to fm. When that's included, the GPU
backend is run with a persistent cache (and no binary caching).
Then we dump all of the fragment shaders (GLSL for now) that were
created to the passed-in directory, along with a JSON digest listing
the number of times each one was referenced in the cache.

2) Adds a python script that invokes the Mali Offline Compiler on a
directory generated from #1, scraping the output of each compile for
cycle counts and writing the collated information to stdout.

Change-Id: Ie4b58ddac4f62e936707c6fec44f4fe605c213fa
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/206162
Commit-Queue: Brian Osman <brianosman@google.com>
Reviewed-by: Mike Klein <mtklein@google.com>
2019-04-09 13:37:55 +00:00
Brian Salomon
00a5eb8c12 Add gltestpersistentcache config that tests GrContextOption's cache.
Uses a new GPU sink that runs each test twice, once to populate the
cache and then again with a new GrContext but a warmed cache. It
verifies that the two generated images are the same.

Change-Id: Iaba195a69751f14ea946afe7174228a813b83a63
Reviewed-on: https://skia-review.googlesource.com/140567
Commit-Queue: Brian Salomon <bsalomon@google.com>
Reviewed-by: Brian Osman <brianosman@google.com>
2018-07-11 20:06:33 +00:00