T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Replace binaryen -ttf based fuzzing with wasm-smith (#2336)

Browse Source 
This commit removes the binaryen support for fuzzing from wasmtime,
instead switching over to `wasm-smith`. In general it's great to have
what fuzzing we can, but our binaryen support suffers from a few issues:

* The Rust crate, binaryen-sys, seems largely unmaintained at this
  point. While we could likely take ownership and/or send PRs to update
  the crate it seems like the maintenance is largely on us at this point.

* Currently the binaryen-sys crate doesn't support fuzzing anything
  beyond MVP wasm, but we're interested at least in features like bulk
  memory and reference types. Additionally we'll also be interested in
  features like module-linking. New features would require either
  implementation work in binaryen or the binaryen-sys crate to support.

* We have 4-5 fuzz-bugs right now related to timeouts simply in
  generating a module for wasmtime to fuzz. One investigation along
  these lines in the past revealed a bug in binaryen itself, and in any
  case these bugs would otherwise need to get investigated, reported,
  and possibly fixed ourselves in upstream binaryen.

Overall I'm not sure at this point if maintaining binaryen fuzzing is
worth it with the advent of `wasm-smith` which has similar goals for
wasm module generation, but is much more readily maintainable on our
end.

Additonally in this commit I've added a fuzzer for wasm-smith's
`SwarmConfig`-based fuzzer which should expand the coverage of tested
modules.

Closes #2163
This commit is contained in:
Alex Crichton
2020-10-29 10:02:59 -05:00
committed by GitHub
parent 3cd9d52d32
commit b73b831892
10 changed files with 41 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
crates/fuzzing/src/oracles.rs
View File

@@ -119,9 +119,8 @@ pub fn compile(wasm: &[u8], strategy: Strategy) {
/// exports. Modulo OOM, non-canonical NaNs, and usage of Wasm features that are
/// or aren't enabled for different configs, we should get the same results when
/// we call the exported functions for all of our different configs.
#[cfg(feature = "binaryen")]
pub fn differential_execution(
ttf: &crate::generators::WasmOptTtf,
module: &wasm_smith::Module,
configs: &[crate::generators::DifferentialConfig],
) {
use std::collections::{HashMap, HashSet};
@@ -144,13 +143,14 @@ pub fn differential_execution(
};
let mut export_func_results: HashMap<String, Result<Box<[Val]>, Trap>> = Default::default();
log_wasm(&ttf.wasm);
let wasm = module.to_bytes();
log_wasm(&wasm);
for config in &configs {
let engine = Engine::new(config);
let store = Store::new(&engine);
let module = match Module::new(&engine, &ttf.wasm) {
let module = match Module::new(&engine, &wasm) {
Ok(module) => module,
// The module might rely on some feature that our config didn't
// enable or something like that.
@@ -278,7 +278,6 @@ pub fn differential_execution(
}
/// Invoke the given API calls.
#[cfg(feature = "binaryen")]
pub fn make_api_calls(api: crate::generators::api::ApiCalls) {
use crate::generators::api::ApiCall;
use std::collections::HashMap;
@@ -323,8 +322,9 @@ pub fn make_api_calls(api: crate::generators::api::ApiCalls) {
ApiCall::ModuleNew { id, wasm } => {
log::debug!("creating module: {}", id);
log_wasm(&wasm.wasm);
let module = match Module::new(engine.as_ref().unwrap(), &wasm.wasm) {
let wasm = wasm.to_bytes();
log_wasm(&wasm);
let module = match Module::new(engine.as_ref().unwrap(), &wasm) {
Ok(m) => m,
Err(_) => continue,
};