T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Replace binaryen -ttf based fuzzing with wasm-smith (#2336)

Browse Source 
This commit removes the binaryen support for fuzzing from wasmtime,
instead switching over to `wasm-smith`. In general it's great to have
what fuzzing we can, but our binaryen support suffers from a few issues:

* The Rust crate, binaryen-sys, seems largely unmaintained at this
  point. While we could likely take ownership and/or send PRs to update
  the crate it seems like the maintenance is largely on us at this point.

* Currently the binaryen-sys crate doesn't support fuzzing anything
  beyond MVP wasm, but we're interested at least in features like bulk
  memory and reference types. Additionally we'll also be interested in
  features like module-linking. New features would require either
  implementation work in binaryen or the binaryen-sys crate to support.

* We have 4-5 fuzz-bugs right now related to timeouts simply in
  generating a module for wasmtime to fuzz. One investigation along
  these lines in the past revealed a bug in binaryen itself, and in any
  case these bugs would otherwise need to get investigated, reported,
  and possibly fixed ourselves in upstream binaryen.

Overall I'm not sure at this point if maintaining binaryen fuzzing is
worth it with the advent of `wasm-smith` which has similar goals for
wasm module generation, but is much more readily maintainable on our
end.

Additonally in this commit I've added a fuzzer for wasm-smith's
`SwarmConfig`-based fuzzer which should expand the coverage of tested
modules.

Closes #2163
This commit is contained in:
Alex Crichton
2020-10-29 10:02:59 -05:00
committed by GitHub
parent 3cd9d52d32
commit b73b831892
10 changed files with 41 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
crates/fuzzing/src/generators.rs
View File

@@ -8,55 +8,12 @@
//! wrapper over an external tool, such that the wrapper implements the
//! `Arbitrary` trait for the wrapped external tool.
#[cfg(feature = "binaryen")]
pub mod api;
pub mod table_ops;
use arbitrary::{Arbitrary, Unstructured};
/// A Wasm test case generator that is powered by Binaryen's `wasm-opt -ttf`.
#[derive(Clone)]
#[cfg(feature = "binaryen")]
pub struct WasmOptTtf {
/// The raw, encoded Wasm bytes.
pub wasm: Vec<u8>,
}
#[cfg(feature = "binaryen")]
impl std::fmt::Debug for WasmOptTtf {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(
f,
"WasmOptTtf {{ wasm: wat::parse_str(r###\"\n{}\n\"###).unwrap() }}",
wasmprinter::print_bytes(&self.wasm).expect("valid wasm should always disassemble")
)
}
}
#[cfg(feature = "binaryen")]
impl Arbitrary for WasmOptTtf {
fn arbitrary(input: &mut arbitrary::Unstructured) -> arbitrary::Result<Self> {
crate::init_fuzzing();
let seed: Vec<u8> = Arbitrary::arbitrary(input)?;
let module = binaryen::tools::translate_to_fuzz_mvp(&seed);
let wasm = module.write();
Ok(WasmOptTtf { wasm })
}
fn arbitrary_take_rest(input: arbitrary::Unstructured) -> arbitrary::Result<Self> {
crate::init_fuzzing();
let seed: Vec<u8> = Arbitrary::arbitrary_take_rest(input)?;
let module = binaryen::tools::translate_to_fuzz_mvp(&seed);
let wasm = module.write();
Ok(WasmOptTtf { wasm })
}
fn size_hint(depth: usize) -> (usize, Option<usize>) {
<Vec<u8> as Arbitrary>::size_hint(depth)
}
}
/// A description of configuration options that we should do differential
/// testing between.
#[derive(Arbitrary, Clone, Debug, PartialEq, Eq, Hash)]