Files
wasmtime/crates/fuzzing/wasm-spec-interpreter/src/with_library.rs
Andrew Brown 5ec92d59d2 [fuzz] Add a meta-differential fuzz target (#4515)
* [fuzz] Add `Module` enum, refactor `ModuleConfig`

This change adds a way to create either a single-instruction module or a
regular (big) `wasm-smith` module. It has some slight refactorings in
preparation for the use of this new code.

* [fuzz] Add `DiffValue` for differential evaluation

In order to evaluate functions with randomly-generated values, we needed
a common way to generate these values. Using the Wasmtime `Val` type is
not great because we would like to be able to implement various traits
on the new value type, e.g., to convert `Into` and `From` boxed values
of other engines we differentially fuzz against. This new type,
`DiffValue`, gives us a common ground for all the conversions and
comparisons between the other engine types.

* [fuzz] Add interface for differential engines

In order to randomly choose an engine to fuzz against, we expect all of
the engines to meet a common interface. The traits in this commit allow
us to instantiate a module from its binary form, evaluate exported
functions, and (possibly) hash the exported items of the instance.

This change has some missing pieces, though:
 - the `wasm-spec-interpreter` needs some work to be able to create
   instances, evaluate a function by name, and expose exported items
 - the `v8` engine is not implemented yet due to the complexity of its
   Rust lifetimes

* [fuzz] Use `ModuleFeatures` instead of existing configuration

When attempting to use both wasm-smith and single-instruction modules,
there is a mismatch in how we communicate what an engine must be able to
support. In the first case, we could use the `ModuleConfig`, a wrapper
for wasm-smith's `SwarmConfig`, but single-instruction modules do not
have a `SwarmConfig`--the many options simply don't apply. Here, we
instead add `ModuleFeatures` and adapt a `ModuleConfig` to that.
`ModuleFeatures` then becomes the way to communicate what features an
engine must support to evaluate functions in a module.

* [fuzz] Add a new fuzz target using the meta-differential oracle

This change adds the `differential_meta` target to the list of fuzz
targets. I expect that sometime soon this could replace the other
`differential*` targets, as it almost checks all the things those check.
The major missing piece is that currently it only chooses
single-instruction modules instead of also generating arbitrary modules
using `wasm-smith`.

Also, this change adds the concept of an ignorable error: some
differential engines will choke with certain inputs (e.g., `wasmi` might
have an old opcode mapping) which we do not want to flag as fuzz bugs.
Here we wrap those errors in `DiffIgnoreError` and then use a new helper
trait, `DiffIgnorable`, to downcast and inspect the `anyhow` error to
only panic on non-ignorable errors; the ignorable errors are converted
to one of the `arbitrary::Error` variants, which we already ignore.

* [fuzz] Compare `DiffValue` NaNs more leniently

Because arithmetic NaNs can contain arbitrary payload bits, checking
that two differential executions should produce the same result should
relax the comparison of the `F32` and `F64` types (and eventually `V128`
as well... TODO). This change adds several considerations, however, so
that in the future we make the comparison a bit stricter, e.g., re:
canonical NaNs. This change, however, just matches the current logic
used by other fuzz targets.

* review: allow hashing mutate the instance state

@alexcrichton requested that the interface be adapted to accommodate
Wasmtime's API, in which even reading from an instance could trigger
mutation of the store.

* review: refactor where configurations are made compatible

See @alexcrichton's
[suggestion](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928974376).

* review: convert `DiffValueType` using `TryFrom`

See @alexcrichton's
[comment](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928962394).

* review: adapt target implementation to Wasmtime-specific RHS

This change is joint work with @alexcrichton to adapt the structure of
the fuzz target to his comments
[here](https://github.com/bytecodealliance/wasmtime/pull/4515#pullrequestreview-1073247791).

This change:
- removes `ModuleFeatures` and the `Module` enum (for big and small
  modules)
- upgrades `SingleInstModule` to filter out cases that are not valid for
  a given `ModuleConfig`
- adds `DiffEngine::name()`
- constructs each `DiffEngine` using a `ModuleConfig`, eliminating
  `DiffIgnoreError` completely
- prints an execution rate to the `differential_meta` target

Still TODO:
- `get_exported_function_signatures` could be re-written in terms of the
  Wasmtime API instead `wasmparser`
- the fuzzer crashes eventually, we think due to the signal handler
  interference between OCaml and Wasmtime
- the spec interpreter has several cases that we skip for now but could
  be fuzzed with further work

Co-authored-by: Alex Crichton <alex@alexcrichton.com>

* fix: avoid SIGSEGV by explicitly initializing OCaml runtime first

* review: use Wasmtime's API to retrieve exported functions

Co-authored-by: Alex Crichton <alex@alexcrichton.com>
2022-08-18 19:22:58 -05:00

144 lines
5.3 KiB
Rust

//! Interpret WebAssembly modules using the OCaml spec interpreter.
//! ```
//! # use wasm_spec_interpreter::{Value, interpret};
//! let module = wat::parse_file("tests/add.wat").unwrap();
//! let parameters = vec![Value::I32(42), Value::I32(1)];
//! let results = interpret(&module, Some(parameters)).unwrap();
//! assert_eq!(results, &[Value::I32(43)]);
//! ```
use crate::Value;
use ocaml_interop::{OCamlRuntime, ToOCaml};
use once_cell::sync::Lazy;
use std::sync::Mutex;
static INTERPRET: Lazy<Mutex<()>> = Lazy::new(|| Mutex::new(()));
/// Interpret the first function in the passed WebAssembly module (in Wasm form,
/// currently, not WAT), optionally with the given parameters. If no parameters
/// are provided, the function is invoked with zeroed parameters.
pub fn interpret(module: &[u8], opt_parameters: Option<Vec<Value>>) -> Result<Vec<Value>, String> {
// The OCaml runtime is not re-entrant
// (https://ocaml.org/manual/intfc.html#ss:parallel-execution-long-running-c-code).
// We need to make sure that only one Rust thread is executing at a time
// (using this lock) or we can observe `SIGSEGV` failures while running
// `cargo test`.
let _lock = INTERPRET.lock().unwrap();
// Here we use an unsafe approach to initializing the `OCamlRuntime` based
// on the discussion in https://github.com/tezedge/ocaml-interop/issues/35.
// This was the recommendation to resolve seeing errors like `boxroot is not
// setup` followed by a `SIGSEGV`; this is similar to the testing approach
// in
// https://github.com/tezedge/ocaml-interop/blob/master/testing/rust-caller/src/lib.rs
// and is only as safe as the OCaml code running underneath.
OCamlRuntime::init_persistent();
let ocaml_runtime = unsafe { OCamlRuntime::recover_handle() };
// Parse and execute, returning results converted to Rust.
let module = module.to_boxroot(ocaml_runtime);
let opt_parameters = opt_parameters.to_boxroot(ocaml_runtime);
let results = ocaml_bindings::interpret(ocaml_runtime, &module, &opt_parameters);
results.to_rust(ocaml_runtime)
}
// Here we declare which functions we will use from the OCaml library. See
// https://docs.rs/ocaml-interop/0.8.4/ocaml_interop/index.html#example.
mod ocaml_bindings {
use super::*;
use ocaml_interop::{
impl_conv_ocaml_variant, ocaml, OCamlBytes, OCamlInt32, OCamlInt64, OCamlList,
};
// Using this macro converts the enum both ways: Rust to OCaml and OCaml to
// Rust. See
// https://docs.rs/ocaml-interop/0.8.4/ocaml_interop/macro.impl_conv_ocaml_variant.html.
impl_conv_ocaml_variant! {
Value {
Value::I32(i: OCamlInt32),
Value::I64(i: OCamlInt64),
Value::F32(i: OCamlInt32),
Value::F64(i: OCamlInt64),
Value::V128(i: OCamlBytes),
}
}
// These functions must be exposed from OCaml with:
// `Callback.register "interpret" interpret`
//
// In Rust, this function becomes:
// `pub fn interpret(_: &mut OCamlRuntime, ...: OCamlRef<...>) -> BoxRoot<...>;`
ocaml! {
pub fn interpret(module: OCamlBytes, params: Option<OCamlList<Value>>) -> Result<OCamlList<Value>, String>;
}
}
/// Initialize a persistent OCaml runtime.
///
/// When used for fuzzing differentially with engines that also use signal
/// handlers, this function provides a way to explicitly set up the OCaml
/// runtime and configure its signal handlers.
pub fn setup_ocaml_runtime() {
let _lock = INTERPRET.lock().unwrap();
OCamlRuntime::init_persistent();
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn multiple() {
let module = wat::parse_file("tests/add.wat").unwrap();
let parameters1 = Some(vec![Value::I32(42), Value::I32(1)]);
let results1 = interpret(&module, parameters1.clone()).unwrap();
let parameters2 = Some(vec![Value::I32(1), Value::I32(42)]);
let results2 = interpret(&module, parameters2.clone()).unwrap();
assert_eq!(results1, results2);
let parameters3 = Some(vec![Value::I32(20), Value::I32(23)]);
let results3 = interpret(&module, parameters3.clone()).unwrap();
assert_eq!(results2, results3);
}
#[test]
fn oob() {
let module = wat::parse_file("tests/oob.wat").unwrap();
let results = interpret(&module, None);
assert_eq!(
results,
Err("Error(_, \"(Isabelle) trap: load\")".to_string())
);
}
#[test]
fn simd_not() {
let module = wat::parse_file("tests/simd_not.wat").unwrap();
let parameters = Some(vec![Value::V128(vec![
0, 255, 0, 0, 255, 0, 0, 0, 0, 255, 0, 0, 0, 0, 0, 0,
])]);
let results = interpret(&module, parameters.clone()).unwrap();
assert_eq!(
results,
vec![Value::V128(vec![
255, 0, 255, 255, 0, 255, 255, 255, 255, 0, 255, 255, 255, 255, 255, 255
])]
);
}
// See issue https://github.com/bytecodealliance/wasmtime/issues/4671.
#[test]
fn order_of_params() {
let module = wat::parse_file("tests/shr_s.wat").unwrap();
let parameters = Some(vec![Value::I32(1795123818), Value::I32(-2147483648)]);
let results = interpret(&module, parameters.clone()).unwrap();
assert_eq!(results, vec![Value::I32(1795123818)]);
}
}