T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
10dbb19983cf77b5d8c92ca93fa9b4ede3c94fc9
wasmtime/fuzz
History
Alex Crichton 10dbb19983 Various improvements to differential fuzzing (#4845) 
* Improve wasmi differential fuzzer

* Support modules with a `start` function
* Implement trap-matching to ensure that wasmi and Wasmtime both report
  the same flavor of trap.

* Support differential fuzzing where no engines match

Locally I was attempting to run against just one wasm engine with
`ALLOWED_ENGINES=wasmi` but the fuzzer quickly panicked because the
generated test case didn't match wasmi's configuration. This commit
updates engine-selection in the differential fuzzer to return `None` if
no engine is applicable, throwing out the test case. This won't be hit
at all with oss-fuzz-based runs but for local runs it'll be useful to
have.

* Improve proposal support in differential fuzzer

* De-prioritize unstable wasm proposals such as multi-memory and
  memory64 by making them more unlikely with `Unstructured::ratio`.
* Allow fuzzing multi-table (reference types) and multi-memory by
  avoiding setting their maximums to 1 in `set_differential_config`.
* Update selection of the pooling strategy to unconditionally support
  the selected module config rather than the other way around.

* Improve handling of traps in differential fuzzing

This commit fixes an issue found via local fuzzing where engines were
reporting different results but the underlying reason for this was that
one engine was hitting stack overflow before the other. To fix the
underlying issue I updated the execution to check for stack overflow
and, if hit, it discards the entire fuzz test case from then on.

The rationale behind this is that each engine can have unique limits for
stack overflow. One test case I was looking at for example would stack
overflow at less than 1000 frames with epoch interruption enabled but
would stack overflow at more than 1000 frames with it disabled. This
means that the state after the trap started to diverge and it looked
like the engines produced different results.

While I was at it I also improved the "function call returned a trap"
case to compare traps to make sure the same trap reason popped out.

* Fix fuzzer tests
2022-09-02 14:16:02 -05:00
..
fuzz_targets
Various improvements to differential fuzzing (#4845)
2022-09-02 14:16:02 -05:00
.gitignore
fuzzing: remove the in-repo corpus
2019-11-26 15:49:07 -08:00
build.rs
Don't let fuzz targets import arbitrary directly (#4806)
2022-08-29 23:06:41 +00:00
Cargo.toml
Don't let fuzz targets import arbitrary directly (#4806)
2022-08-29 23:06:41 +00:00
README.md
Port v8 fuzzer to the new framework (#4739)
2022-08-19 19:19:00 +00:00

README.md

cargo fuzz Targets for Wasmtime

This crate defines various libFuzzer fuzzing targets for Wasmtime, which can be run via cargo fuzz.

These fuzz targets just glue together pre-defined test case generators with oracles and pass libFuzzer-provided inputs to them. The test case generators and oracles themselves are independent from the fuzzing engine that is driving the fuzzing process and are defined in wasmtime/crates/fuzzing.

Example

To start fuzzing run the following command, where $MY_FUZZ_TARGET is one of the available fuzz targets:

cargo fuzz run $MY_FUZZ_TARGET

Available Fuzz Targets

At the time of writing, we have the following fuzz targets:

  • api_calls: stress the Wasmtime API by executing sequences of API calls; only the subset of the API is currently supported.
  • compile: Attempt to compile libFuzzer's raw input bytes with Wasmtime.
  • compile-maybe-invalid: Attempt to compile a wasm-smith-generated Wasm module with code sequences that may be invalid.
  • cranelift-fuzzgen: Generate a Cranelift function and check that it returns the same results when compiled to the host and when using the Cranelift interpreter; only a subset of Cranelift IR is currently supported.
  • cranelift-icache: Generate a Cranelift function A, applies a small mutation to its source, yielding a function A', and checks that A compiled + incremental compilation generates the same machine code as if A' was compiled from scratch.
  • differential: Generate a Wasm module, evaluate each exported function with random inputs, and check that Wasmtime returns the same results as a choice of another engine: the Wasm spec interpreter (see the wasm-spec-interpreter crate), the wasmi interpreter, V8 (through the v8 crate), or Wasmtime itself run with a different configuration.
  • instantiate: Generate a Wasm module and Wasmtime configuration and attempt to compile and instantiate with them.
  • instantiate-many: Generate many Wasm modules and attempt to compile and instantiate them concurrently.
  • spectests: Pick a random spec test and run it with a generated configuration.
  • table_ops: Generate a sequence of externref table operations and run them in a GC environment.

The canonical list of fuzz targets is the .rs files in the fuzz_targets directory:

ls wasmtime/fuzz/fuzz_targets/

Corpora

While you can start from scratch, libFuzzer will work better if it is given a corpus of seed inputs to kick start the fuzzing process. We maintain a corpus for each of these fuzz targets in a dedicated repo on github.

You can use our corpora by cloning it and placing it at wasmtime/fuzz/corpus:

git clone \
    https://github.com/bytecodealliance/wasmtime-libfuzzer-corpus.git \
    wasmtime/fuzz/corpus

Reproducing a Fuzz Bug

When investigating a fuzz bug (especially one found by OSS-Fuzz), use the following steps to reproduce it locally:

  1. Download the test case (either the "Minimized Testcase" or "Unminimized Testcase" from OSS-Fuzz will do).
  2. Run the test case in the correct fuzz target: 
    cargo +nightly fuzz run <target> <test case>
    If all goes well, the bug should reproduce and libFuzzer will dump the failure stack trace to stdout
  3. For more debugging information, run the command above with RUST_LOG=debug to print the configuration and WebAssembly input used by the test case (see uses of log_wasm in the wasmtime-fuzzing crate).