v8/test/fuzzer
Andreas Haas 8f07a87df0 [wasm][fuzzer] Do not execute code with potential non-determinism
The WebAssembly spec is not fully deterministic: the sign bit of NaN
can be arbitrary. This sign bit can be observed by several WebAssembly
opcodes. In the testcase the sign bit of NaN makes the difference
between terminating code and an infinite loop.

In the libfuzzer fuzzer we have to prevent infinite loops ourselves.
At the moment we do this by only execute generated code of WebAssembly
modules for which the interpretation of the code ends in a limited
number of steps. With the non-determinism described above we cannot
guarantee the absence of infinite loops with this method. Therefore
we stop now to execute generated code of WebAssembly modules for which
we observe possible non-determinism in the interpreter.

R=clemensh@chromium.org

Bug: chromium:863829
Change-Id: I461d67df87d672bed25d6c915ba7ea5134cb5890
Reviewed-on: https://chromium-review.googlesource.com/1141945
Reviewed-by: Clemens Hammacher <clemensh@chromium.org>
Commit-Queue: Andreas Haas <ahaas@chromium.org>
Cr-Commit-Position: refs/heads/master@{#54541}
2018-07-19 08:55:55 +00:00
..
json
multi_return [turbofan] Add fuzzer to test different signatures for multi-returns 2018-01-12 12:20:27 +00:00
parser
regexp
regexp_builtins [regexp] Initial go at a builtins fuzzer 2018-01-18 11:02:57 +00:00
wasm
wasm_async [wasm] Avoid executing infinite loops in the wasm fuzzers 2017-09-07 12:35:45 +00:00
wasm_code
wasm_compile
wasm_data_section
wasm_function_sigs_section
wasm_globals_section
wasm_imports_section
wasm_memory_section
wasm_names_section
wasm_types_section
BUILD.gn [build] Add data deps for executable tests 2018-03-26 13:37:03 +00:00
DEPS
fuzzer-support.cc [cleanup] Deprecate EnsureEventLoopInitialized 2018-02-28 15:32:28 +00:00
fuzzer-support.h [fuzzer] Use std::unique_ptr for the FuzzerSupport 2017-11-29 16:36:23 +00:00
fuzzer.cc [test/fuzzer] Fix cpplint complaints 2017-09-04 10:45:21 +00:00
fuzzer.status
json.cc [fuzzer] Fix OOM in v8_json_parser_fuzzer due to unnecessary long input. 2018-07-17 14:25:27 +00:00
multi-return.cc [fuzzer] Switch code kind for multi-return fuzzer. 2018-07-13 12:09:24 +00:00
parser.cc [fuzzer] Fix timeout in v8_script_parser_fuzzer due to unnecessary long inputs. 2018-07-16 14:29:24 +00:00
README.md [gyp] move build targets for tests to gypfiles. 2018-01-30 06:31:00 +00:00
regexp-builtins.cc Reland: [cleanup] Refactor the Factory 2018-04-09 19:52:22 +00:00
regexp.cc [explicit isolates] Remove GetIsolate from objects.cc 2018-07-13 12:42:14 +00:00
testcfg.py [wasm] [fuzzer] Remove wasm_call fuzzer 2018-02-01 14:30:49 +00:00
wasm_corpus.tar.gz.sha1
wasm-async.cc [wasm] Reimplement WebAssembly.instantiate without desugaring 2018-05-24 22:15:52 +00:00
wasm-code.cc [wasm] "Vectorize" fuzzer 2018-07-12 12:44:39 +00:00
wasm-compile.cc [wasm] "Vectorize" fuzzer 2018-07-12 12:44:39 +00:00
wasm-data-section.cc
wasm-function-sigs-section.cc
wasm-fuzzer-common.cc [wasm][fuzzer] Do not execute code with potential non-determinism 2018-07-19 08:55:55 +00:00
wasm-fuzzer-common.h [wasm] "Vectorize" fuzzer 2018-07-12 12:44:39 +00:00
wasm-globals-section.cc
wasm-imports-section.cc
wasm-memory-section.cc
wasm-names-section.cc
wasm-types-section.cc
wasm.cc [wasm] wasm-fuzzer: Add ability to generate test case 2018-06-21 12:53:58 +00:00

How to make a libFuzzer fuzzer in V8

This document describes how to make a new libFuzzer fuzzer for V8. A general introduction to libFuzzer can be found here. In short, libFuzzer is an in-process coverage-driven evolutionary fuzzer. libFuzzer serves you with a sequence of byte arrays that you can use to test your code. libFuzzer tries to generate this sequence of byte arrays in a way that maximizes test coverage.

Warning: By itself libFuzzer typically does not generate valid JavaScript code.

Changes to V8

tldr: Do the same as https://codereview.chromium.org/2280623002 to introduce a new fuzzer to V8.

This is a step by step guide on how to make a new fuzzer in V8. In the example the fuzzer is called foo.

  1. Copy one of the existing fuzzer implementations in test/fuzzer/, e.g. cp wasm.cc foo.cc

    • Copying an existing fuzzer is a good idea to get all the required setup, e.g. setting up the isolate
  2. Create a directory called foo in test/fuzzer/ which contains at least one file

    • The file is used by the trybots to check whether the fuzzer actually compiles and runs
  3. Copy the build rules of an existing fuzzer in BUILD.gn, e.g. the build rules for the wasm.cc fuzzer are v8_source_set("wasm_fuzzer") and v8_fuzzer("wasm_fuzzer"). Note that the name has to be the name of the directory created in Step 2 + _fuzzer so that the scripts on the trybots work

  4. Now you can already compile the fuzzer, e.g. with ninja -j 1000 -C out/x64.debug/v8_simple_foo_fuzzer

    • Use this binary to reproduce issues found by cluster fuzz, e.g. out/x64.debug/v8_simple_foo_fuzzer testcase.foo
  5. Copy the binary name and the test directory name in test/fuzzer/fuzzer.isolate

  6. Add the fuzzer to the FuzzerTestSuite in test/fuzzer/testcfg.py

    • This step is needed to run the fuzzer with the files created in Step 2 on the trybots
  7. Commit the changes described above to the V8 repository

Changes to Chromium

tldr: Do the same as https://codereview.chromium.org/2344823002 to add the new fuzzer to cluster fuzz.

  1. Copy the build rules of an existing fuzzer in testing/libfuzzer/fuzzers/BUILD.gn, e.g. the build rule for the wasm.cc fuzzer is v8_wasm_fuzzer. There is no need to set a dictionary , or a seed_corpus. See chromium-fuzzing-getting-started for more information.

  2. Compile the fuzzer in chromium (for different configurations see: https://chromium.googlesource.com/chromium/src/+/master/testing/libfuzzer/reference.md):

    • gn gen out/libfuzzer '--args=use_libfuzzer=true is_asan=true is_debug=false enable_nacl=false'

    • ninja -j 1000 -C out/libfuzzer/ v8_foo_fuzzer

  3. Run the fuzzer locally

    • mkdir /tmp/empty_corpus && out/libfuzzer/v8_foo_fuzzer /tmp/empty_corpus