T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[fuzz] Add a meta-differential fuzz target (#4515)

Browse Source 
* [fuzz] Add `Module` enum, refactor `ModuleConfig`

This change adds a way to create either a single-instruction module or a
regular (big) `wasm-smith` module. It has some slight refactorings in
preparation for the use of this new code.

* [fuzz] Add `DiffValue` for differential evaluation

In order to evaluate functions with randomly-generated values, we needed
a common way to generate these values. Using the Wasmtime `Val` type is
not great because we would like to be able to implement various traits
on the new value type, e.g., to convert `Into` and `From` boxed values
of other engines we differentially fuzz against. This new type,
`DiffValue`, gives us a common ground for all the conversions and
comparisons between the other engine types.

* [fuzz] Add interface for differential engines

In order to randomly choose an engine to fuzz against, we expect all of
the engines to meet a common interface. The traits in this commit allow
us to instantiate a module from its binary form, evaluate exported
functions, and (possibly) hash the exported items of the instance.

This change has some missing pieces, though:
 - the `wasm-spec-interpreter` needs some work to be able to create
   instances, evaluate a function by name, and expose exported items
 - the `v8` engine is not implemented yet due to the complexity of its
   Rust lifetimes

* [fuzz] Use `ModuleFeatures` instead of existing configuration

When attempting to use both wasm-smith and single-instruction modules,
there is a mismatch in how we communicate what an engine must be able to
support. In the first case, we could use the `ModuleConfig`, a wrapper
for wasm-smith's `SwarmConfig`, but single-instruction modules do not
have a `SwarmConfig`--the many options simply don't apply. Here, we
instead add `ModuleFeatures` and adapt a `ModuleConfig` to that.
`ModuleFeatures` then becomes the way to communicate what features an
engine must support to evaluate functions in a module.

* [fuzz] Add a new fuzz target using the meta-differential oracle

This change adds the `differential_meta` target to the list of fuzz
targets. I expect that sometime soon this could replace the other
`differential*` targets, as it almost checks all the things those check.
The major missing piece is that currently it only chooses
single-instruction modules instead of also generating arbitrary modules
using `wasm-smith`.

Also, this change adds the concept of an ignorable error: some
differential engines will choke with certain inputs (e.g., `wasmi` might
have an old opcode mapping) which we do not want to flag as fuzz bugs.
Here we wrap those errors in `DiffIgnoreError` and then use a new helper
trait, `DiffIgnorable`, to downcast and inspect the `anyhow` error to
only panic on non-ignorable errors; the ignorable errors are converted
to one of the `arbitrary::Error` variants, which we already ignore.

* [fuzz] Compare `DiffValue` NaNs more leniently

Because arithmetic NaNs can contain arbitrary payload bits, checking
that two differential executions should produce the same result should
relax the comparison of the `F32` and `F64` types (and eventually `V128`
as well... TODO). This change adds several considerations, however, so
that in the future we make the comparison a bit stricter, e.g., re:
canonical NaNs. This change, however, just matches the current logic
used by other fuzz targets.

* review: allow hashing mutate the instance state

@alexcrichton requested that the interface be adapted to accommodate
Wasmtime's API, in which even reading from an instance could trigger
mutation of the store.

* review: refactor where configurations are made compatible

See @alexcrichton's
[suggestion](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928974376).

* review: convert `DiffValueType` using `TryFrom`

See @alexcrichton's
[comment](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928962394).

* review: adapt target implementation to Wasmtime-specific RHS

This change is joint work with @alexcrichton to adapt the structure of
the fuzz target to his comments
[here](https://github.com/bytecodealliance/wasmtime/pull/4515#pullrequestreview-1073247791).

This change:
- removes `ModuleFeatures` and the `Module` enum (for big and small
  modules)
- upgrades `SingleInstModule` to filter out cases that are not valid for
  a given `ModuleConfig`
- adds `DiffEngine::name()`
- constructs each `DiffEngine` using a `ModuleConfig`, eliminating
  `DiffIgnoreError` completely
- prints an execution rate to the `differential_meta` target

Still TODO:
- `get_exported_function_signatures` could be re-written in terms of the
  Wasmtime API instead `wasmparser`
- the fuzzer crashes eventually, we think due to the signal handler
  interference between OCaml and Wasmtime
- the spec interpreter has several cases that we skip for now but could
  be fuzzed with further work

Co-authored-by: Alex Crichton <alex@alexcrichton.com>

* fix: avoid SIGSEGV by explicitly initializing OCaml runtime first

* review: use Wasmtime's API to retrieve exported functions

Co-authored-by: Alex Crichton <alex@alexcrichton.com>
This commit is contained in:
Andrew Brown
2022-08-18 17:22:58 -07:00
committed by GitHub
parent 8b6019909b
commit 5ec92d59d2
14 changed files with 1046 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

178
crates/fuzzing/src/oracles/diff_wasmi.rs Normal file
View File

@@ -0,0 +1,178 @@
//! Evaluate an exported Wasm function using the wasmi interpreter.
use crate::generators::{DiffValue, ModuleConfig};
use crate::oracles::engine::{DiffEngine, DiffInstance};
use anyhow::{bail, Context, Result};
use std::hash::Hash;
/// A wrapper for `wasmi` as a [`DiffEngine`].
pub struct WasmiEngine;
impl WasmiEngine {
/// Build a new [`WasmiEngine`] but only if the configuration does not rely
/// on features that `wasmi` does not support.
pub fn new(config: &ModuleConfig) -> Result<Box<Self>> {
if config.config.reference_types_enabled {
bail!("wasmi does not support reference types")
}
if config.config.simd_enabled {
bail!("wasmi does not support SIMD")
}
if config.config.multi_value_enabled {
bail!("wasmi does not support multi-value")
}
if config.config.saturating_float_to_int_enabled {
bail!("wasmi does not support saturating float-to-int conversions")
}
if config.config.sign_extension_enabled {
bail!("wasmi does not support sign-extension")
}
Ok(Box::new(Self))
}
}
impl DiffEngine for WasmiEngine {
fn name(&self) -> &'static str {
"wasmi"
}
fn instantiate(&self, wasm: &[u8]) -> Result<Box<dyn DiffInstance>> {
let module = wasmi::Module::from_buffer(wasm).context("unable to validate Wasm module")?;
let instance = wasmi::ModuleInstance::new(&module, &wasmi::ImportsBuilder::default())
.context("unable to instantiate module in wasmi")?;
let instance = instance.assert_no_start();
let exports = list_export_names(wasm);
Ok(Box::new(WasmiInstance {
module,
exports,
instance,
}))
}
}
/// A wrapper for `wasmi` Wasm instances.
struct WasmiInstance {
#[allow(dead_code)] // reason = "the module must live as long as its reference"
module: wasmi::Module,
instance: wasmi::ModuleRef,
/// `wasmi`'s instances have no way of listing their exports so, in order to
/// properly hash the instance, we keep track of the export names.
exports: Vec<String>,
}
impl DiffInstance for WasmiInstance {
fn name(&self) -> &'static str {
"wasmi"
}
fn evaluate(&mut self, function_name: &str, arguments: &[DiffValue]) -> Result<Vec<DiffValue>> {
let arguments: Vec<_> = arguments.iter().map(wasmi::RuntimeValue::from).collect();
let export = self
.instance
.export_by_name(function_name)
.context(format!(
"unable to find function '{}' in wasmi instance",
function_name
))?;
let function = export.as_func().context("wasmi export is not a function")?;
let result = wasmi::FuncInstance::invoke(&function, &arguments, &mut wasmi::NopExternals)
.context("failed while invoking function in wasmi")?;
Ok(if let Some(result) = result {
vec![result.into()]
} else {
vec![]
})
}
fn is_hashable(&self) -> bool {
true
}
fn hash(&mut self, state: &mut std::collections::hash_map::DefaultHasher) -> Result<()> {
for export_name in &self.exports {
if let Some(export) = self.instance.export_by_name(export_name) {
match export {
wasmi::ExternVal::Func(_) => {}
wasmi::ExternVal::Table(_) => {} // TODO eventually we can hash whether the values are null or non-null.
wasmi::ExternVal::Memory(m) => {
// `wasmi` memory may be stored non-contiguously; copy
// it out to a contiguous chunk.
let mut buffer: Vec<u8> = vec![0; m.current_size().0 * 65536];
m.get_into(0, &mut buffer[..])
.expect("can access wasmi memory");
buffer.hash(state)
}
wasmi::ExternVal::Global(g) => {
let val: DiffValue = g.get().into();
val.hash(state);
}
}
} else {
panic!("unable to find export: {}", export_name)
}
}
Ok(())
}
}
/// List the names of all exported items in a binary Wasm module.
fn list_export_names(wasm: &[u8]) -> Vec<String> {
let mut exports = vec![];
for payload in wasmparser::Parser::new(0).parse_all(&wasm) {
match payload.unwrap() {
wasmparser::Payload::ExportSection(s) => {
for export in s {
exports.push(export.unwrap().name.to_string());
}
}
_ => {
// Ignore any other sections.
}
}
}
exports
}
impl From<&DiffValue> for wasmi::RuntimeValue {
fn from(v: &DiffValue) -> Self {
use wasmi::RuntimeValue::*;
match *v {
DiffValue::I32(n) => I32(n),
DiffValue::I64(n) => I64(n),
DiffValue::F32(n) => F32(wasmi::nan_preserving_float::F32::from_bits(n)),
DiffValue::F64(n) => F64(wasmi::nan_preserving_float::F64::from_bits(n)),
DiffValue::V128(_) => unimplemented!(),
}
}
}
impl Into<DiffValue> for wasmi::RuntimeValue {
fn into(self) -> DiffValue {
use wasmi::RuntimeValue::*;
match self {
I32(n) => DiffValue::I32(n),
I64(n) => DiffValue::I64(n),
F32(n) => DiffValue::F32(n.to_bits()),
F64(n) => DiffValue::F64(n.to_bits()),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_list_export_names() {
let wat = r#"(module
(func (export "a") (result i32) (i32.const 42))
(global (export "b") (mut i32) (i32.const 42))
(memory (export "c") 1 2 shared)
)"#;
let wasm = wat::parse_str(wat).unwrap();
assert_eq!(
list_export_names(&wasm),
vec!["a".to_string(), "b".to_string(), "c".to_string()],
);
}
}