T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
3a2b32bf4df83ab3d624f8c187116d02adb2c69c
wasmtime/fuzz
History
Jamey Sharp bd870a9d6c Shrink all SmallVecs by 8 bytes (#4951) 
We weren't using the "union" cargo feature for the smallvec crate, which
reduces the size of a SmallVec by one machine word. This feature
requires Rust 1.49 but we already require much newer versions.

When using Wasmtime to compile pulldown-cmark from Sightglass, this
saves a decent amount of memory allocations and writes. According to
`valgrind --tool=dhat`:

- 6.2MiB (3.69%) less memory allocated over the program's lifetime
- 0.5MiB (4.13%) less memory allocated at maximum heap size
- 5.5MiB (1.88%) fewer bytes written to
- 0.44% fewer instructions executed

Sightglass reports a statistically significant runtime improvement too:

compilation :: cycles :: benchmarks/pulldown-cmark/benchmark.wasm

  Δ = 24379323.60 ± 20051394.04 (confidence = 99%)

  shrink-abiarg-0406da67c.so is 1.01x to 1.13x faster than main-be690a468.so!

  [227506364 355007998.78 423280514] main-be690a468.so
  [227686018 330628675.18 406025344] shrink-abiarg-0406da67c.so

compilation :: cycles :: benchmarks/spidermonkey/benchmark.wasm

  Δ = 360151622.56 ± 278294316.90 (confidence = 99%)

  shrink-abiarg-0406da67c.so is 1.01x to 1.07x faster than main-be690a468.so!

  [8709162212 8911001926.44 9535111576] main-be690a468.so
  [5058015392 8550850303.88 9282148438] shrink-abiarg-0406da67c.so

compilation :: cycles :: benchmarks/bz2/benchmark.wasm

  Δ = 6936570.28 ± 6897696.38 (confidence = 99%)

  shrink-abiarg-0406da67c.so is 1.00x to 1.08x faster than main-be690a468.so!

  [155810934 175260571.20 234737344] main-be690a468.so
  [119128240 168324000.92 257451074] shrink-abiarg-0406da67c.so
2022-09-23 16:32:13 -07:00
..
fuzz_targets
fuzzgen: Disable verifier after NaN Canonicalization (#4914)
2022-09-15 17:18:15 +00:00
.gitignore
fuzzing: remove the in-repo corpus
2019-11-26 15:49:07 -08:00
build.rs
Don't let fuzz targets import arbitrary directly (#4806)
2022-08-29 23:06:41 +00:00
Cargo.toml
Shrink all SmallVecs by 8 bytes (#4951)
2022-09-23 16:32:13 -07:00
README.md
Port v8 fuzzer to the new framework (#4739)
2022-08-19 19:19:00 +00:00

README.md

cargo fuzz Targets for Wasmtime

This crate defines various libFuzzer fuzzing targets for Wasmtime, which can be run via cargo fuzz.

These fuzz targets just glue together pre-defined test case generators with oracles and pass libFuzzer-provided inputs to them. The test case generators and oracles themselves are independent from the fuzzing engine that is driving the fuzzing process and are defined in wasmtime/crates/fuzzing.

Example

To start fuzzing run the following command, where $MY_FUZZ_TARGET is one of the available fuzz targets:

cargo fuzz run $MY_FUZZ_TARGET

Available Fuzz Targets

At the time of writing, we have the following fuzz targets:

  • api_calls: stress the Wasmtime API by executing sequences of API calls; only the subset of the API is currently supported.
  • compile: Attempt to compile libFuzzer's raw input bytes with Wasmtime.
  • compile-maybe-invalid: Attempt to compile a wasm-smith-generated Wasm module with code sequences that may be invalid.
  • cranelift-fuzzgen: Generate a Cranelift function and check that it returns the same results when compiled to the host and when using the Cranelift interpreter; only a subset of Cranelift IR is currently supported.
  • cranelift-icache: Generate a Cranelift function A, applies a small mutation to its source, yielding a function A', and checks that A compiled + incremental compilation generates the same machine code as if A' was compiled from scratch.
  • differential: Generate a Wasm module, evaluate each exported function with random inputs, and check that Wasmtime returns the same results as a choice of another engine: the Wasm spec interpreter (see the wasm-spec-interpreter crate), the wasmi interpreter, V8 (through the v8 crate), or Wasmtime itself run with a different configuration.
  • instantiate: Generate a Wasm module and Wasmtime configuration and attempt to compile and instantiate with them.
  • instantiate-many: Generate many Wasm modules and attempt to compile and instantiate them concurrently.
  • spectests: Pick a random spec test and run it with a generated configuration.
  • table_ops: Generate a sequence of externref table operations and run them in a GC environment.

The canonical list of fuzz targets is the .rs files in the fuzz_targets directory:

ls wasmtime/fuzz/fuzz_targets/

Corpora

While you can start from scratch, libFuzzer will work better if it is given a corpus of seed inputs to kick start the fuzzing process. We maintain a corpus for each of these fuzz targets in a dedicated repo on github.

You can use our corpora by cloning it and placing it at wasmtime/fuzz/corpus:

git clone \
    https://github.com/bytecodealliance/wasmtime-libfuzzer-corpus.git \
    wasmtime/fuzz/corpus

Reproducing a Fuzz Bug

When investigating a fuzz bug (especially one found by OSS-Fuzz), use the following steps to reproduce it locally:

  1. Download the test case (either the "Minimized Testcase" or "Unminimized Testcase" from OSS-Fuzz will do).
  2. Run the test case in the correct fuzz target: 
    cargo +nightly fuzz run <target> <test case>
    If all goes well, the bug should reproduce and libFuzzer will dump the failure stack trace to stdout
  3. For more debugging information, run the command above with RUST_LOG=debug to print the configuration and WebAssembly input used by the test case (see uses of log_wasm in the wasmtime-fuzzing crate).