17c7c2f4b1
Change-Id: I1be6bb537139c50342602aacd978585151a4a480 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/443916 Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
4.5 KiB
#Fuzzing
In this folder, we keep our fuzzers (bits of code that takes a randomized input and executes code
randomly, focusing on specific APIs). For example, we have a codec fuzzer which takes a mutated
png/jpeg or similar file and attempts to turn it into an SkImage. We also have a canvas fuzzer
which takes in a random set of bytes and turns them into calls on SkCanvas.
Executables
These fuzzers are packaged in two different ways (see //BUILD.gn). There is a fuzz executable
that contains all fuzzers and is a convenient way to reproduce fuzzer-reported bugs. There are also
single fuzzer executables containing exactly one fuzzer, which are convenient to build with
libfuzzer.
See [../site/dev/testing/fuzz.md] for more information on building and running fuzzers using the
fuzz executable.
Continuous Running
We fuzz Skia using OSS-Fuzz, which in turn uses fuzzing engines such as libfuzzer, afl-fuzz, hong-fuzz, and others to fuzz Skia. OSS-fuzz will automatically file and close bugs when it finds issues.
There is a Skia folder in the OSS-Fuzz repo that we make changes to when we want to add/remove/change the fuzzers that are automatically run. This describes how to test the OSS-Fuzz build and fuzzers locally using Docker.
When enabling a fuzzer in OSS-Fuzz, we typically need to follow these steps:
- *Add a seed corpus to
gs://skia-fuzzer/oss-fuzz/(in the skia-public project). Make sure the corpus file is public-readable. It is easiest to add this permission via the web UI. This is done by granting the allUsers "name" the Reader role to the zip file. See the infra team if you do not have access to this bucket. - *Update the Dockerfile to download the seed corpus to the build image.
- Update build.sh
to build the desired fuzzer target and move it into $OUT. If there is a seed corpus, move
it into $OUT and make sure it is the same name as the fuzzer executable with
_seed_corpus.zipas a suffix.
*For fuzzers who depend strongly on the format of the randomized data, e.g. image decoding, SkSL parsing. These are called binary fuzzers, as opposed to API fuzzers.
Example PRs for adding fuzzers: binary, API
There is also an OSS-fuzz folder set up for the skcms repo. The build process is similar, except instead of compiling using GN targets, the build.sh script compiles the fuzz executables directly.
OSS-Fuzz dashboard
https://oss-fuzz.com/fuzzer-stats is useful to see metrics on how our fuzzers are running. It shows things like executions per second (higher is better), edge coverage percent per fuzzer, what percent of fuzzing runs end in OOM/timeout/crash, the entire corpus of fuzzed inputs (corpus_backup), etc. Contact aarya@ to get permission to view this dashboard if necessary. Here are some example dashboards:
- Per Fuzzer summary for all Skia fuzzers driven by libFuzzer
- Five day summary of sksl2glsl driven by afl-fuzz
That dashboard also has a Coverage Report. Even though it appears the Coverage report is per fuzzer, the reports always show the aggregated coverage from all fuzzers. Example coverage report from 2021 Aug 22