skia2/fuzz
Kevin Lubick 62f57a02b3 [includes] Enforce IWYU on src/utils
Client changes:
 - cl/443664347
 - cl/444232853
 - http://ag/17915718
 - http://ag/17913178
 - https://crrev.com/c/3602239

Change-Id: I16bcdbbe2b13bbf52f007b0f131e9ee0c14ed237
Bug: skia:13052
Reviewed-on: https://skia-review.googlesource.com/c/skia/+/532257
Reviewed-by: Brian Osman <brianosman@google.com>
Commit-Queue: Kevin Lubick <kjlubick@google.com>
2022-04-28 17:32:20 +00:00
..
oss_fuzz Move all Ganesh source files into ganesh subdirectory. 2022-04-07 21:06:50 +00:00
coverage
Fuzz.cpp Hide SkImageFilter::CropRect 2021-01-30 16:10:29 +00:00
Fuzz.h Stop using filterquality 2021-07-18 15:48:36 +00:00
FuzzCanvas.cpp Move all Ganesh source files into ganesh subdirectory. 2022-04-07 21:06:50 +00:00
FuzzCommon.cpp Control growth of FuzzNicePath with chain of path transforms 2022-02-14 19:11:54 +00:00
FuzzCommon.h rewrite includes to not need so much -Ifoo 2019-04-24 16:27:11 +00:00
FuzzCreateDDL.cpp Move Ganesh specific headers into include/private/gpu/ganesh 2022-04-12 17:30:07 +00:00
FuzzDDLThreading.cpp Finish GrMipMapped->GrMipmapped 2022-04-15 15:43:52 +00:00
FuzzDrawFunctions.cpp Use SkClipOp::kFoo instead of kFoo_SkClipOp from SkClipOpPriv 2021-08-03 18:08:21 +00:00
FuzzEncoders.cpp Reland "[includes] Remove link between SkImage.h and SkImageEncoder.h" 2022-03-10 04:47:51 +00:00
FuzzGradients.cpp [graphite] Add SkTileModeToStr 2022-01-11 16:19:53 +00:00
FuzzMain.cpp Reland "Fix compilation w/ "skia_enable_svg = false" (take 2)" 2021-09-01 17:54:08 +00:00
FuzzParsePath.cpp Fix additional cases of variable shadowing. 2021-08-16 17:47:14 +00:00
FuzzPath.cpp add SKPath::readFromMemory() fuzzer 2020-08-12 17:40:16 +00:00
FuzzPathMeasure.cpp [fuzz] Standardize, document, and backport fuzzing defines. 2020-09-14 13:36:10 +00:00
FuzzPathop.cpp Revert "Revert "switch to new filltype for SkPath"" 2019-11-26 17:43:14 +00:00
FuzzPolyUtils.cpp [includes] Enforce IWYU on src/utils 2022-04-28 17:32:20 +00:00
FuzzRegionOp.cpp [fuzz] Expose Region Op fuzzing to oss-fuzz. 2021-03-29 16:05:05 +00:00
FuzzRRect.cpp add SkRRect::readFromMemory() fuzzer 2020-08-11 20:26:28 +00:00
FuzzSkParagraph.cpp Fix compilation with "skia_enable_skparagraph = false" 2021-08-31 18:50:20 +00:00
FuzzTriangulation.cpp Move all Ganesh source files into ganesh subdirectory. 2022-04-07 21:06:50 +00:00
README.md [fuzz] Update docs with better OSS-Fuzz link 2021-08-31 14:04:02 +00:00

#Fuzzing In this folder, we keep our fuzzers (bits of code that takes a randomized input and executes code randomly, focusing on specific APIs). For example, we have a codec fuzzer which takes a mutated png/jpeg or similar file and attempts to turn it into an SkImage. We also have a canvas fuzzer which takes in a random set of bytes and turns them into calls on SkCanvas.

Executables

These fuzzers are packaged in two different ways (see //BUILD.gn). There is a fuzz executable that contains all fuzzers and is a convenient way to reproduce fuzzer-reported bugs. There are also single fuzzer executables containing exactly one fuzzer, which are convenient to build with libfuzzer.

See [../site/dev/testing/fuzz.md] for more information on building and running fuzzers using the fuzz executable.

Continuous Running

We fuzz Skia using OSS-Fuzz, which in turn uses fuzzing engines such as libfuzzer, afl-fuzz, hong-fuzz, and others to fuzz Skia. OSS-fuzz will automatically file and close bugs when it finds issues.

There is a Skia folder in the OSS-Fuzz repo that we make changes to when we want to add/remove/change the fuzzers that are automatically run. This describes how to test the OSS-Fuzz build and fuzzers locally using Docker.

When enabling a fuzzer in OSS-Fuzz, we typically need to follow these steps:

  1. *Add a seed corpus to gs://skia-fuzzer/oss-fuzz/ (in the skia-public project). Make sure the corpus file is public-readable. It is easiest to add this permission via the web UI. This is done by granting the allUsers "name" the Reader role to the zip file. See the infra team if you do not have access to this bucket.
  2. *Update the Dockerfile to download the seed corpus to the build image.
  3. Update build.sh to build the desired fuzzer target and move it into $OUT. If there is a seed corpus, move it into $OUT and make sure it is the same name as the fuzzer executable with _seed_corpus.zip as a suffix.

*For fuzzers who depend strongly on the format of the randomized data, e.g. image decoding, SkSL parsing. These are called binary fuzzers, as opposed to API fuzzers.

Example PRs for adding fuzzers: binary, API

There is also an OSS-fuzz folder set up for the skcms repo. The build process is similar, except instead of compiling using GN targets, the build.sh script compiles the fuzz executables directly.

OSS-Fuzz dashboard

https://oss-fuzz.com/fuzzer-stats is useful to see metrics on how our fuzzers are running. It shows things like executions per second (higher is better), edge coverage percent per fuzzer, what percent of fuzzing runs end in OOM/timeout/crash, the entire corpus of fuzzed inputs (corpus_backup), etc. Contact aarya@ to get permission to view this dashboard if necessary. Here are some example dashboards:

That dashboard also has a Coverage Report. Even though it appears the Coverage report is per fuzzer, the reports always show the aggregated coverage from all fuzzers. Example coverage report from 2021 Aug 22

See Also