[fuzz] Add a meta-differential fuzz target (#4515)
* [fuzz] Add `Module` enum, refactor `ModuleConfig` This change adds a way to create either a single-instruction module or a regular (big) `wasm-smith` module. It has some slight refactorings in preparation for the use of this new code. * [fuzz] Add `DiffValue` for differential evaluation In order to evaluate functions with randomly-generated values, we needed a common way to generate these values. Using the Wasmtime `Val` type is not great because we would like to be able to implement various traits on the new value type, e.g., to convert `Into` and `From` boxed values of other engines we differentially fuzz against. This new type, `DiffValue`, gives us a common ground for all the conversions and comparisons between the other engine types. * [fuzz] Add interface for differential engines In order to randomly choose an engine to fuzz against, we expect all of the engines to meet a common interface. The traits in this commit allow us to instantiate a module from its binary form, evaluate exported functions, and (possibly) hash the exported items of the instance. This change has some missing pieces, though: - the `wasm-spec-interpreter` needs some work to be able to create instances, evaluate a function by name, and expose exported items - the `v8` engine is not implemented yet due to the complexity of its Rust lifetimes * [fuzz] Use `ModuleFeatures` instead of existing configuration When attempting to use both wasm-smith and single-instruction modules, there is a mismatch in how we communicate what an engine must be able to support. In the first case, we could use the `ModuleConfig`, a wrapper for wasm-smith's `SwarmConfig`, but single-instruction modules do not have a `SwarmConfig`--the many options simply don't apply. Here, we instead add `ModuleFeatures` and adapt a `ModuleConfig` to that. `ModuleFeatures` then becomes the way to communicate what features an engine must support to evaluate functions in a module. * [fuzz] Add a new fuzz target using the meta-differential oracle This change adds the `differential_meta` target to the list of fuzz targets. I expect that sometime soon this could replace the other `differential*` targets, as it almost checks all the things those check. The major missing piece is that currently it only chooses single-instruction modules instead of also generating arbitrary modules using `wasm-smith`. Also, this change adds the concept of an ignorable error: some differential engines will choke with certain inputs (e.g., `wasmi` might have an old opcode mapping) which we do not want to flag as fuzz bugs. Here we wrap those errors in `DiffIgnoreError` and then use a new helper trait, `DiffIgnorable`, to downcast and inspect the `anyhow` error to only panic on non-ignorable errors; the ignorable errors are converted to one of the `arbitrary::Error` variants, which we already ignore. * [fuzz] Compare `DiffValue` NaNs more leniently Because arithmetic NaNs can contain arbitrary payload bits, checking that two differential executions should produce the same result should relax the comparison of the `F32` and `F64` types (and eventually `V128` as well... TODO). This change adds several considerations, however, so that in the future we make the comparison a bit stricter, e.g., re: canonical NaNs. This change, however, just matches the current logic used by other fuzz targets. * review: allow hashing mutate the instance state @alexcrichton requested that the interface be adapted to accommodate Wasmtime's API, in which even reading from an instance could trigger mutation of the store. * review: refactor where configurations are made compatible See @alexcrichton's [suggestion](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928974376). * review: convert `DiffValueType` using `TryFrom` See @alexcrichton's [comment](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928962394). * review: adapt target implementation to Wasmtime-specific RHS This change is joint work with @alexcrichton to adapt the structure of the fuzz target to his comments [here](https://github.com/bytecodealliance/wasmtime/pull/4515#pullrequestreview-1073247791). This change: - removes `ModuleFeatures` and the `Module` enum (for big and small modules) - upgrades `SingleInstModule` to filter out cases that are not valid for a given `ModuleConfig` - adds `DiffEngine::name()` - constructs each `DiffEngine` using a `ModuleConfig`, eliminating `DiffIgnoreError` completely - prints an execution rate to the `differential_meta` target Still TODO: - `get_exported_function_signatures` could be re-written in terms of the Wasmtime API instead `wasmparser` - the fuzzer crashes eventually, we think due to the signal handler interference between OCaml and Wasmtime - the spec interpreter has several cases that we skip for now but could be fuzzed with further work Co-authored-by: Alex Crichton <alex@alexcrichton.com> * fix: avoid SIGSEGV by explicitly initializing OCaml runtime first * review: use Wasmtime's API to retrieve exported functions Co-authored-by: Alex Crichton <alex@alexcrichton.com>
This commit is contained in:
Andrew Brown
159
crates/fuzzing/src/oracles/diff_wasmtime.rs
Normal file
159
crates/fuzzing/src/oracles/diff_wasmtime.rs
Normal file
@@ -0,0 +1,159 @@
|
||||
//! Evaluate an exported Wasm function using Wasmtime.
|
||||
|
||||
use crate::generators::{self, DiffValue};
|
||||
use crate::oracles::engine::DiffInstance;
|
||||
use crate::oracles::{compile_module, engine::DiffEngine, instantiate_with_dummy, StoreLimits};
|
||||
use anyhow::{Context, Result};
|
||||
use std::hash::Hash;
|
||||
use std::slice;
|
||||
use wasmtime::{AsContextMut, Extern, FuncType, Instance, Module, Store, Val};
|
||||
|
||||
/// A wrapper for using Wasmtime as a [`DiffEngine`].
|
||||
pub struct WasmtimeEngine {
|
||||
pub(crate) config: generators::Config,
|
||||
}
|
||||
|
||||
impl WasmtimeEngine {
|
||||
/// Merely store the configuration; the engine is actually constructed
|
||||
/// later. Ideally the store and engine could be built here but
|
||||
/// `compile_module` takes a [`generators::Config`]; TODO re-factor this if
|
||||
/// that ever changes.
|
||||
pub fn new(config: &generators::Config) -> Result<Box<Self>> {
|
||||
Ok(Box::new(Self {
|
||||
config: config.clone(),
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
impl DiffEngine for WasmtimeEngine {
|
||||
fn name(&self) -> &'static str {
|
||||
"wasmtime"
|
||||
}
|
||||
|
||||
fn instantiate(&self, wasm: &[u8]) -> Result<Box<dyn DiffInstance>> {
|
||||
let store = self.config.to_store();
|
||||
let module = compile_module(store.engine(), wasm, true, &self.config).unwrap();
|
||||
let instance = WasmtimeInstance::new(store, module)?;
|
||||
Ok(Box::new(instance))
|
||||
}
|
||||
}
|
||||
|
||||
/// A wrapper around a Wasmtime instance.
|
||||
///
|
||||
/// The Wasmtime engine constructs a new store and compiles an instance of a
|
||||
/// Wasm module.
|
||||
pub struct WasmtimeInstance {
|
||||
store: Store<StoreLimits>,
|
||||
instance: Instance,
|
||||
}
|
||||
|
||||
impl WasmtimeInstance {
|
||||
/// Instantiate a new Wasmtime instance.
|
||||
pub fn new(mut store: Store<StoreLimits>, module: Module) -> Result<Self> {
|
||||
let instance = instantiate_with_dummy(&mut store, &module)
|
||||
.context("unable to instantiate module in wasmtime")?;
|
||||
Ok(Self { store, instance })
|
||||
}
|
||||
|
||||
/// Retrieve the names and types of all exported functions in the instance.
|
||||
///
|
||||
/// This is useful for evaluating each exported function with different
|
||||
/// values. The [`DiffInstance`] trait asks for the function name and we
|
||||
/// need to know the function signature in order to pass in the right
|
||||
/// arguments.
|
||||
pub fn exported_functions(&mut self) -> Vec<(String, FuncType)> {
|
||||
let exported_functions = self
|
||||
.instance
|
||||
.exports(&mut self.store)
|
||||
.map(|e| (e.name().to_owned(), e.into_func()))
|
||||
.filter_map(|(n, f)| f.map(|f| (n, f)))
|
||||
.collect::<Vec<_>>();
|
||||
exported_functions
|
||||
.into_iter()
|
||||
.map(|(n, f)| (n, f.ty(&self.store)))
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
impl DiffInstance for WasmtimeInstance {
|
||||
fn name(&self) -> &'static str {
|
||||
"wasmtime"
|
||||
}
|
||||
|
||||
fn evaluate(&mut self, function_name: &str, arguments: &[DiffValue]) -> Result<Vec<DiffValue>> {
|
||||
let arguments: Vec<_> = arguments.iter().map(Val::from).collect();
|
||||
|
||||
let function = self
|
||||
.instance
|
||||
.get_func(&mut self.store, function_name)
|
||||
.expect("unable to access exported function");
|
||||
let ty = function.ty(&self.store);
|
||||
let mut results = vec![Val::I32(0); ty.results().len()];
|
||||
function.call(&mut self.store, &arguments, &mut results)?;
|
||||
|
||||
let results = results.into_iter().map(Val::into).collect();
|
||||
Ok(results)
|
||||
}
|
||||
|
||||
fn is_hashable(&self) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
fn hash(&mut self, state: &mut std::collections::hash_map::DefaultHasher) -> Result<()> {
|
||||
let exports: Vec<_> = self
|
||||
.instance
|
||||
.exports(self.store.as_context_mut())
|
||||
.map(|e| e.into_extern())
|
||||
.collect();
|
||||
for e in exports {
|
||||
match e {
|
||||
Extern::Global(g) => {
|
||||
let val: DiffValue = g.get(&mut self.store).into();
|
||||
val.hash(state)
|
||||
}
|
||||
Extern::Memory(m) => {
|
||||
let data = m.data(&mut self.store);
|
||||
data.hash(state)
|
||||
}
|
||||
Extern::SharedMemory(m) => {
|
||||
let data = unsafe { slice::from_raw_parts(m.data() as *mut u8, m.data_size()) };
|
||||
data.hash(state)
|
||||
}
|
||||
Extern::Table(_) => {
|
||||
// TODO: it's unclear whether it is worth it to iterate
|
||||
// through the table and hash the values.
|
||||
}
|
||||
Extern::Func(_) => {
|
||||
// Note: no need to hash exported functions.
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl From<&DiffValue> for Val {
|
||||
fn from(v: &DiffValue) -> Self {
|
||||
match *v {
|
||||
DiffValue::I32(n) => Val::I32(n),
|
||||
DiffValue::I64(n) => Val::I64(n),
|
||||
DiffValue::F32(n) => Val::F32(n),
|
||||
DiffValue::F64(n) => Val::F64(n),
|
||||
DiffValue::V128(n) => Val::V128(n),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Into<DiffValue> for Val {
|
||||
fn into(self) -> DiffValue {
|
||||
match self {
|
||||
Val::I32(n) => DiffValue::I32(n),
|
||||
Val::I64(n) => DiffValue::I64(n),
|
||||
Val::F32(n) => DiffValue::F32(n),
|
||||
Val::F64(n) => DiffValue::F64(n),
|
||||
Val::V128(n) => DiffValue::V128(n),
|
||||
Val::FuncRef(_) => unimplemented!(),
|
||||
Val::ExternRef(_) => unimplemented!(),
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user