skia2/infra/bots/recipes
Brian Osman ed58e004e0 Add support for pre-compiling cached SkSL shaders
The client can do a test run of their application with
a persistent cache set to SkSL mode. They store the key
and data blobs that are produced.

Ship those blobs with the application. At startup, call
GrContext::precompileShader for each key/data pair. This
compiles the shaders, and stores the GL program ID, plus
a small amount of metadata in our runtime program cache.

Caveats:
* Currently only implemented for the GL backend. Other
  backends will require more metadata to do any useful
  amount of work. Metal may need a more drastic workflow
  change, involving offline compilation of the shaders.
* Currently only implemented for cached SkSL (not GLSL
  or program binaries). Supporting other formats again
  requires more metadata, and the cached shaders become
  increasingly specialized to GPU and driver versions.
* Reusing the cached SkSL on different hardware is not
  supported. Many driver workarounds are implemented in
  the SkSL -> GLSL transformation, but some are higher
  level. Limiting device variance by artificially hiding
  extensions may help, but there are no guarantees.

* The 'gltestprecompile' DM config exercises this code
  similarly to 'gltestpersistentcache', ensuring that
  results are visually identical when precompiling, and
  that no cache misses occur after precompiling.

Change-Id: Id314c5d5f5a58fe503a0505a613bd4a540cc3589
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/239438
Reviewed-by: Greg Daniel <egdaniel@google.com>
Reviewed-by: Brian Salomon <bsalomon@google.com>
Commit-Queue: Brian Osman <brianosman@google.com>
2019-09-06 19:45:09 +00:00
..
android_compile.expected Roll recipe dependencies (nontrivial). 2019-06-21 12:03:54 +00:00
calmbench.expected Roll recipe dependencies (nontrivial). 2019-06-25 11:43:53 +00:00
check_generated_files.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
compile.expected [infra] Move SKQP recipe test to correct recipe 2019-07-19 18:48:31 +00:00
compute_buildstats.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
compute_test.expected Roll recipe dependencies (nontrivial). 2019-06-25 11:43:53 +00:00
g3_compile.expected Roll recipe dependencies (nontrivial). 2019-06-21 12:03:54 +00:00
housekeeper.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
infra.expected [infra] Make infra tests idempotent 2019-07-09 19:04:47 +00:00
perf_canvaskit.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
perf_pathkit.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
perf_skottietrace.expected Roll recipe dependencies (nontrivial). 2019-06-25 11:43:53 +00:00
perf_skottiewasm_lottieweb.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
perf.expected Reland "Merge GrOpList and GrRTOpList and rename to GrOpsTask." 2019-08-22 20:52:09 +00:00
recreate_skps.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
skpbench.expected Add a kind of asset for MSKPs so the skpbench bot can read them. 2019-08-01 20:29:52 +00:00
skqp_test.expected Roll recipe dependencies (nontrivial). 2019-06-03 11:25:16 +00:00
sync_and_compile.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
test_canvaskit.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
test_lottie_web.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
test_pathkit.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
test_skqp_emulator.expected Roll recipe dependencies (nontrivial). 2019-08-14 11:16:56 +00:00
test.expected Add support for pre-compiling cached SkSL shaders 2019-09-06 19:45:09 +00:00
upload_buildstats_results.expected Roll recipe dependencies (nontrivial). 2019-06-03 11:25:16 +00:00
upload_calmbench_results.expected Roll recipe dependencies (nontrivial). 2019-06-03 11:25:16 +00:00
upload_dm_results.expected Roll recipe dependencies (nontrivial). 2019-06-03 11:25:16 +00:00
upload_nano_results.expected Roll recipe dependencies (nontrivial). 2019-06-03 11:25:16 +00:00
upload_skiaserve.expected Roll recipe dependencies (nontrivial). 2019-06-03 11:25:16 +00:00
android_compile.py [Android compile bot] Add links to force sync the checkout 2019-05-09 13:00:07 +00:00
calmbench.py Reland [recipes] Isolate build outputs with no subdirs 2018-06-01 15:18:46 +00:00
check_generated_files.py rewrite *.gn and #includes on CheckGeneratedFiles bot 2019-07-25 20:48:33 +00:00
compile.py [infra] Move SKQP recipe test to correct recipe 2019-07-19 18:48:31 +00:00
compute_buildstats.py Build code size treemaps for skottie_tool 2019-02-12 13:54:00 +00:00
compute_test.py Add OpenCL test job that runs hello-opencl. 2018-06-29 13:25:17 +00:00
g3_compile.py Add CL link in G3 trybot 2019-06-17 16:16:47 +00:00
housekeeper.py Remove old code size analysis 2018-10-10 14:25:17 +00:00
infra.py [infra] Make infra tests idempotent 2019-07-09 19:04:47 +00:00
perf_canvaskit.py [pathkit/canvaskit] Address race when uploading results 2019-03-12 21:45:04 +00:00
perf_pathkit.py [pathkit/canvaskit] Address race when uploading results 2019-03-12 21:45:04 +00:00
perf_skottietrace.py Add new SkottieTracing GPU bot to match SkottieWASM and LottieWeb 2019-06-22 12:28:38 +00:00
perf_skottiewasm_lottieweb.py [perf_skottiewasm_lottieweb] Start tracking timings of drawn frames and count aborted frames 2019-07-25 21:28:03 +00:00
perf.py Reland "Merge GrOpList and GrRTOpList and rename to GrOpsTask." 2019-08-22 20:52:09 +00:00
README.md Re-re-land "added GrSkSLFP and converted DitherEffect to use it" 2018-07-31 15:18:03 +00:00
recreate_skps.py Have the RecreateSKPs bot upload to partner bucket 2019-02-06 18:35:12 +00:00
skpbench.py Add a kind of asset for MSKPs so the skpbench bot can read them. 2019-08-01 20:29:52 +00:00
skqp_test.py [recipes] Rename some modules and files 2018-05-24 14:06:55 +00:00
sync_and_compile.py [infra] Move SKQP recipe test to correct recipe 2019-07-19 18:48:31 +00:00
test_canvaskit.py [pathkit/canvaskit] Address race when uploading results 2019-03-12 21:45:04 +00:00
test_lottie_web.py Update android-skqp Docker image to accept licenses 2019-01-22 23:34:58 +00:00
test_pathkit.py [pathkit/canvaskit] Address race when uploading results 2019-03-12 21:45:04 +00:00
test_skqp_emulator.py Add Build and Test jobs for SKQP using docker. 2018-10-08 19:17:25 +00:00
test.py Add support for pre-compiling cached SkSL shaders 2019-09-06 19:45:09 +00:00
upload_buildstats_results.py Use right bucket for buildstats 2018-10-08 19:23:56 +00:00
upload_calmbench_results.py [recipes] Rename some modules and files 2018-05-24 14:06:55 +00:00
upload_dm_results.py Fix upload_dm_results 2018-08-20 16:44:24 +00:00
upload_nano_results.py Add BuildStats pipeline for PathKit binaries 2018-10-08 18:15:32 +00:00
upload_skiaserve.py Reland [recipes] Isolate build outputs with no subdirs 2018-06-01 15:18:46 +00:00

Skia Recipes

These are the top-level scripts which run inside of Swarming tasks to perform all of Skia's automated testing.

To run a recipe locally:

$ python infra/bots/recipes.py run --workdir=/tmp/<workdir> <recipe name without .py> key1=value1 key2=value2 ...

Each recipe may have its own required properties which must be entered as key/value pairs in the command.

When you change a recipe, you generally need to re-train the simulation test:

$ python infra/bots/recipes.py test train

Or:

    $ cd infra/bots; make train

The test generates expectations files for the tests contained within each recipe which illustrate which steps would run, given a particular set of inputs. Pay attention to the diffs in these files when making changes to ensure that your change has the intended effect.