T0b1/wasmtime
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[fuzz] Add a meta-differential fuzz target (#4515)

Browse Source 
* [fuzz] Add `Module` enum, refactor `ModuleConfig`

This change adds a way to create either a single-instruction module or a
regular (big) `wasm-smith` module. It has some slight refactorings in
preparation for the use of this new code.

* [fuzz] Add `DiffValue` for differential evaluation

In order to evaluate functions with randomly-generated values, we needed
a common way to generate these values. Using the Wasmtime `Val` type is
not great because we would like to be able to implement various traits
on the new value type, e.g., to convert `Into` and `From` boxed values
of other engines we differentially fuzz against. This new type,
`DiffValue`, gives us a common ground for all the conversions and
comparisons between the other engine types.

* [fuzz] Add interface for differential engines

In order to randomly choose an engine to fuzz against, we expect all of
the engines to meet a common interface. The traits in this commit allow
us to instantiate a module from its binary form, evaluate exported
functions, and (possibly) hash the exported items of the instance.

This change has some missing pieces, though:
 - the `wasm-spec-interpreter` needs some work to be able to create
   instances, evaluate a function by name, and expose exported items
 - the `v8` engine is not implemented yet due to the complexity of its
   Rust lifetimes

* [fuzz] Use `ModuleFeatures` instead of existing configuration

When attempting to use both wasm-smith and single-instruction modules,
there is a mismatch in how we communicate what an engine must be able to
support. In the first case, we could use the `ModuleConfig`, a wrapper
for wasm-smith's `SwarmConfig`, but single-instruction modules do not
have a `SwarmConfig`--the many options simply don't apply. Here, we
instead add `ModuleFeatures` and adapt a `ModuleConfig` to that.
`ModuleFeatures` then becomes the way to communicate what features an
engine must support to evaluate functions in a module.

* [fuzz] Add a new fuzz target using the meta-differential oracle

This change adds the `differential_meta` target to the list of fuzz
targets. I expect that sometime soon this could replace the other
`differential*` targets, as it almost checks all the things those check.
The major missing piece is that currently it only chooses
single-instruction modules instead of also generating arbitrary modules
using `wasm-smith`.

Also, this change adds the concept of an ignorable error: some
differential engines will choke with certain inputs (e.g., `wasmi` might
have an old opcode mapping) which we do not want to flag as fuzz bugs.
Here we wrap those errors in `DiffIgnoreError` and then use a new helper
trait, `DiffIgnorable`, to downcast and inspect the `anyhow` error to
only panic on non-ignorable errors; the ignorable errors are converted
to one of the `arbitrary::Error` variants, which we already ignore.

* [fuzz] Compare `DiffValue` NaNs more leniently

Because arithmetic NaNs can contain arbitrary payload bits, checking
that two differential executions should produce the same result should
relax the comparison of the `F32` and `F64` types (and eventually `V128`
as well... TODO). This change adds several considerations, however, so
that in the future we make the comparison a bit stricter, e.g., re:
canonical NaNs. This change, however, just matches the current logic
used by other fuzz targets.

* review: allow hashing mutate the instance state

@alexcrichton requested that the interface be adapted to accommodate
Wasmtime's API, in which even reading from an instance could trigger
mutation of the store.

* review: refactor where configurations are made compatible

See @alexcrichton's
[suggestion](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928974376).

* review: convert `DiffValueType` using `TryFrom`

See @alexcrichton's
[comment](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928962394).

* review: adapt target implementation to Wasmtime-specific RHS

This change is joint work with @alexcrichton to adapt the structure of
the fuzz target to his comments
[here](https://github.com/bytecodealliance/wasmtime/pull/4515#pullrequestreview-1073247791).

This change:
- removes `ModuleFeatures` and the `Module` enum (for big and small
  modules)
- upgrades `SingleInstModule` to filter out cases that are not valid for
  a given `ModuleConfig`
- adds `DiffEngine::name()`
- constructs each `DiffEngine` using a `ModuleConfig`, eliminating
  `DiffIgnoreError` completely
- prints an execution rate to the `differential_meta` target

Still TODO:
- `get_exported_function_signatures` could be re-written in terms of the
  Wasmtime API instead `wasmparser`
- the fuzzer crashes eventually, we think due to the signal handler
  interference between OCaml and Wasmtime
- the spec interpreter has several cases that we skip for now but could
  be fuzzed with further work

Co-authored-by: Alex Crichton <alex@alexcrichton.com>

* fix: avoid SIGSEGV by explicitly initializing OCaml runtime first

* review: use Wasmtime's API to retrieve exported functions

Co-authored-by: Alex Crichton <alex@alexcrichton.com>
This commit is contained in:
Andrew Brown
2022-08-18 17:22:58 -07:00
committed by GitHub
parent 8b6019909b
commit 5ec92d59d2
14 changed files with 1046 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
crates/fuzzing/src/generators.rs
View File

@@ -15,18 +15,20 @@ mod config;
mod instance_allocation_strategy;
mod instance_limits;
mod memory;
mod module_config;
mod module;
mod single_inst_module;
mod spec_test;
mod stacks;
pub mod table_ops;
mod value;
pub use codegen_settings::CodegenSettings;
pub use config::{Config, WasmtimeConfig};
pub use instance_allocation_strategy::InstanceAllocationStrategy;
pub use instance_limits::InstanceLimits;
pub use memory::{MemoryConfig, NormalMemoryConfig, UnalignedMemory, UnalignedMemoryCreator};
pub use module_config::ModuleConfig;
pub use module::ModuleConfig;
pub use single_inst_module::SingleInstModule;
pub use spec_test::SpecTest;
pub use stacks::Stacks;
pub use value::{DiffValue, DiffValueType};

38
crates/fuzzing/src/generators/config.rs
View File

@@ -92,6 +92,8 @@ impl Config {
limits.tables = 1;
limits.table_elements = 1_000;
limits.size = 1_000_000;
match &mut self.wasmtime.memory_config {
MemoryConfig::Normal(config) => {
config.static_memory_maximum_size = Some(limits.memory_pages * 0x10000);
@@ -101,6 +103,34 @@ impl Config {
}
}
/// Force `self` to be a configuration compatible with `other`. This is
/// useful for differential execution to avoid unhelpful fuzz crashes when
/// one engine has a feature enabled and the other does not.
pub fn make_compatible_with(&mut self, other: &Self) {
// Use the same `wasm-smith` configuration as `other` because this is
// used for determining what Wasm features are enabled in the engine
// (see `to_wasmtime`).
self.module_config = other.module_config.clone();
// Use the same allocation strategy between the two configs.
//
// Ideally this wouldn't be necessary, but, during differential
// evaluation, if the `lhs` is using ondemand and the `rhs` is using the
// pooling allocator (or vice versa), then the module may have been
// generated in such a way that is incompatible with the other
// allocation strategy.
//
// We can remove this in the future when it's possible to access the
// fields of `wasm_smith::Module` to constrain the pooling allocator
// based on what was actually generated.
self.wasmtime.strategy = other.wasmtime.strategy.clone();
if let InstanceAllocationStrategy::Pooling { .. } = &other.wasmtime.strategy {
// Also use the same memory configuration when using the pooling
// allocator.
self.wasmtime.memory_config = other.wasmtime.memory_config.clone();
}
}
/// Uses this configuration and the supplied source of data to generate
/// a wasm module.
///
@@ -112,13 +142,7 @@ impl Config {
input: &mut Unstructured<'_>,
default_fuel: Option<u32>,
) -> arbitrary::Result<wasm_smith::Module> {
let mut module = wasm_smith::Module::new(self.module_config.config.clone(), input)?;
if let Some(default_fuel) = default_fuel {
module.ensure_termination(default_fuel);
}
Ok(module)
self.module_config.generate(input, default_fuel)
}
/// Indicates that this configuration should be spec-test-compliant,

25
crates/fuzzing/src/generators/module_config.rs → crates/fuzzing/src/generators/module.rs
View File

@@ -1,4 +1,4 @@
//! Generate a configuration for generating a Wasm module.
//! Generate a Wasm module and the configuration for generating it.
use arbitrary::{Arbitrary, Unstructured};
use wasm_smith::SwarmConfig;
@@ -36,3 +36,26 @@ impl<'a> Arbitrary<'a> for ModuleConfig {
Ok(ModuleConfig { config })
}
}
impl ModuleConfig {
/// Uses this configuration and the supplied source of data to generate a
/// Wasm module.
///
/// If a `default_fuel` is provided, the resulting module will be configured
/// to ensure termination; as doing so will add an additional global to the
/// module, the pooling allocator, if configured, must also have its globals
/// limit updated.
pub fn generate(
&self,
input: &mut Unstructured<'_>,
default_fuel: Option<u32>,
) -> arbitrary::Result<wasm_smith::Module> {
let mut module = wasm_smith::Module::new(self.config.clone(), input)?;
if let Some(default_fuel) = default_fuel {
module.ensure_termination(default_fuel);
}
Ok(module)
}
}

102
crates/fuzzing/src/generators/single_inst_module.rs
View File

@@ -1,6 +1,7 @@
//! Generate Wasm modules that contain a single instruction.
use arbitrary::{Arbitrary, Unstructured};
use super::ModuleConfig;
use arbitrary::Unstructured;
use wasm_encoder::{
CodeSection, ExportKind, ExportSection, Function, FunctionSection, Instruction, Module,
TypeSection, ValType,
@@ -13,17 +14,38 @@ const FUNCTION_NAME: &'static str = "test";
///
/// By explicitly defining the parameter and result types (versus generating the
/// module directly), we can more easily generate values of the right type.
#[derive(Clone, Debug)]
#[derive(Clone)]
pub struct SingleInstModule<'a> {
instruction: Instruction<'a>,
parameters: &'a [ValType],
results: &'a [ValType],
feature: fn(&ModuleConfig) -> bool,
}
impl<'a> SingleInstModule<'a> {
/// Generate a binary Wasm module with a single exported function, `test`,
/// Choose a single-instruction module that matches `config`.
pub fn new(u: &mut Unstructured<'a>, config: &mut ModuleConfig) -> arbitrary::Result<&'a Self> {
// To avoid skipping modules unnecessarily during fuzzing, fix up the
// `ModuleConfig` to match the inherent limits of a single-instruction
// module.
config.config.min_funcs = 1;
config.config.max_funcs = 1;
config.config.min_tables = 0;
config.config.max_tables = 0;
config.config.min_memories = 0;
config.config.max_memories = 0;
// Only select instructions that match the `ModuleConfig`.
let instructions = &INSTRUCTIONS
.iter()
.filter(|i| (i.feature)(config))
.collect::<Vec<_>>();
u.choose(&instructions[..]).copied()
}
/// Encode a binary Wasm module with a single exported function, `test`,
/// that executes the single instruction.
pub fn encode(&self) -> Vec<u8> {
pub fn to_bytes(&self) -> Vec<u8> {
let mut module = Module::new();
// Encode the type section.
@@ -61,12 +83,6 @@ impl<'a> SingleInstModule<'a> {
}
}
impl<'a> Arbitrary<'a> for &SingleInstModule<'_> {
fn arbitrary(u: &mut Unstructured<'a>) -> arbitrary::Result<Self> {
u.choose(&INSTRUCTIONS)
}
}
// MACROS
//
// These macros make it a bit easier to define the instructions available for
@@ -91,39 +107,52 @@ macro_rules! valtype {
macro_rules! binary {
($inst:ident, $rust_ty:tt) => {
binary! { $inst, valtype!($rust_ty), valtype!($rust_ty) }
binary! { $inst, $rust_ty, $rust_ty }
};
($inst:ident, $arguments_ty:expr, $result_ty:expr) => {
($inst:ident, $arguments_ty:tt, $result_ty:tt) => {
SingleInstModule {
instruction: Instruction::$inst,
parameters: &[$arguments_ty, $arguments_ty],
results: &[$result_ty],
parameters: &[valtype!($arguments_ty), valtype!($arguments_ty)],
results: &[valtype!($result_ty)],
feature: |_| true,
}
};
}
macro_rules! compare {
($inst:ident, $rust_ty:tt) => {
binary! { $inst, valtype!($rust_ty), ValType::I32 }
binary! { $inst, $rust_ty, i32 }
};
}
macro_rules! unary {
($inst:ident, $rust_ty:tt) => {
unary! { $inst, valtype!($rust_ty), valtype!($rust_ty) }
unary! { $inst, $rust_ty, $rust_ty }
};
($inst:ident, $argument_ty:expr, $result_ty:expr) => {
($inst:ident, $argument_ty:tt, $result_ty:tt) => {
SingleInstModule {
instruction: Instruction::$inst,
parameters: &[$argument_ty],
results: &[$result_ty],
parameters: &[valtype!($argument_ty)],
results: &[valtype!($result_ty)],
feature: |_| true,
}
};
($inst:ident, $argument_ty:tt, $result_ty:tt, $feature:expr) => {
SingleInstModule {
instruction: Instruction::$inst,
parameters: &[valtype!($argument_ty)],
results: &[valtype!($result_ty)],
feature: $feature,
}
};
}
macro_rules! convert {
($inst:ident, $from_ty:tt -> $to_ty:tt) => {
unary! { $inst, valtype!($from_ty), valtype!($to_ty) }
unary! { $inst, $from_ty, $to_ty }
};
($inst:ident, $from_ty:tt -> $to_ty:tt, $feature:expr) => {
unary! { $inst, $from_ty, $to_ty, $feature }
};
}
@@ -172,7 +201,7 @@ static INSTRUCTIONS: &[SingleInstModule] = &[
binary!(I64Rotr, i64),
// Integer comparison.
unary!(I32Eqz, i32),
unary!(I64Eqz, ValType::I64, ValType::I32),
unary!(I64Eqz, i64, i32),
compare!(I32Eq, i32),
compare!(I64Eq, i64),
compare!(I32Ne, i32),
@@ -236,11 +265,11 @@ static INSTRUCTIONS: &[SingleInstModule] = &[
compare!(F32Ge, f32),
compare!(F64Ge, f64),
// Integer conversions ("to integer").
unary!(I32Extend8S, i32),
unary!(I32Extend16S, i32),
unary!(I64Extend8S, i64),
unary!(I64Extend16S, i64),
convert!(I64Extend32S, i64 -> i64),
unary!(I32Extend8S, i32, i32, |c| c.config.sign_extension_enabled),
unary!(I32Extend16S, i32, i32, |c| c.config.sign_extension_enabled),
unary!(I64Extend8S, i64, i64, |c| c.config.sign_extension_enabled),
unary!(I64Extend16S, i64, i64, |c| c.config.sign_extension_enabled),
convert!(I64Extend32S, i64 -> i64, |c| c.config.sign_extension_enabled),
convert!(I32WrapI64, i64 -> i32),
convert!(I64ExtendI32S, i32 -> i64),
convert!(I64ExtendI32U, i32 -> i64),
@@ -252,14 +281,14 @@ static INSTRUCTIONS: &[SingleInstModule] = &[
convert!(I64TruncF32U, f32 -> i64),
convert!(I64TruncF64S, f64 -> i64),
convert!(I64TruncF64U, f64 -> i64),
convert!(I32TruncSatF32S, f32 -> i32),
convert!(I32TruncSatF32U, f32 -> i32),
convert!(I32TruncSatF64S, f64 -> i32),
convert!(I32TruncSatF64U, f64 -> i32),
convert!(I64TruncSatF32S, f32 -> i64),
convert!(I64TruncSatF32U, f32 -> i64),
convert!(I64TruncSatF64S, f64 -> i64),
convert!(I64TruncSatF64U, f64 -> i64),
convert!(I32TruncSatF32S, f32 -> i32, |c| c.config.saturating_float_to_int_enabled),
convert!(I32TruncSatF32U, f32 -> i32, |c| c.config.saturating_float_to_int_enabled),
convert!(I32TruncSatF64S, f64 -> i32, |c| c.config.saturating_float_to_int_enabled),
convert!(I32TruncSatF64U, f64 -> i32, |c| c.config.saturating_float_to_int_enabled),
convert!(I64TruncSatF32S, f32 -> i64, |c| c.config.saturating_float_to_int_enabled),
convert!(I64TruncSatF32U, f32 -> i64, |c| c.config.saturating_float_to_int_enabled),
convert!(I64TruncSatF64S, f64 -> i64, |c| c.config.saturating_float_to_int_enabled),
convert!(I64TruncSatF64U, f64 -> i64, |c| c.config.saturating_float_to_int_enabled),
convert!(I32ReinterpretF32, f32 -> i32),
convert!(I64ReinterpretF64, f64 -> i64),
// Floating-point conversions ("to float").
@@ -287,8 +316,9 @@ mod test {
instruction: Instruction::I32Add,
parameters: &[ValType::I32, ValType::I32],
results: &[ValType::I32],
feature: |_| true,
};
let wasm = sut.encode();
let wasm = sut.to_bytes();
let wat = wasmprinter::print_bytes(wasm).unwrap();
assert_eq!(
wat,
@@ -307,7 +337,7 @@ mod test {
#[test]
fn instructions_encode_to_valid_modules() {
for inst in INSTRUCTIONS {
assert!(wat::parse_bytes(&inst.encode()).is_ok());
assert!(wat::parse_bytes(&inst.to_bytes()).is_ok());
}
}
}

177
crates/fuzzing/src/generators/value.rs Normal file
View File

@@ -0,0 +1,177 @@
//! Generate Wasm values, primarily for differential execution.
use arbitrary::{Arbitrary, Unstructured};
use std::hash::Hash;
/// A value passed to and from evaluation. Note that reference types are not
/// (yet) supported.
#[derive(Clone, Debug)]
#[allow(missing_docs)]
pub enum DiffValue {
I32(i32),
I64(i64),
F32(u32),
F64(u64),
V128(u128),
}
impl DiffValue {
fn ty(&self) -> DiffValueType {
match self {
DiffValue::I32(_) => DiffValueType::I32,
DiffValue::I64(_) => DiffValueType::I64,
DiffValue::F32(_) => DiffValueType::F32,
DiffValue::F64(_) => DiffValueType::F64,
DiffValue::V128(_) => DiffValueType::V128,
}
}
/// Generate a [`DiffValue`] of the given `ty` type.
///
/// This function will bias the returned value 50% of the time towards one
/// of a set of known values (e.g., NaN, -1, 0, infinity, etc.).
pub fn arbitrary_of_type(
u: &mut Unstructured<'_>,
ty: DiffValueType,
) -> arbitrary::Result<Self> {
use DiffValueType::*;
let val = match ty {
I32 => DiffValue::I32(biased_arbitrary_value(u, KNOWN_I32_VALUES)?),
I64 => DiffValue::I64(biased_arbitrary_value(u, KNOWN_I64_VALUES)?),
F32 => {
// TODO once `to_bits` is stable as a `const` function, move
// this to a `const` definition.
let known_f32_values = &[
f32::NAN.to_bits(),
f32::INFINITY.to_bits(),
f32::NEG_INFINITY.to_bits(),
f32::MIN.to_bits(),
(-1.0f32).to_bits(),
(0.0f32).to_bits(),
(1.0f32).to_bits(),
f32::MAX.to_bits(),
];
DiffValue::F32(biased_arbitrary_value(u, known_f32_values)?)
}
F64 => {
// TODO once `to_bits` is stable as a `const` function, move
// this to a `const` definition.
let known_f64_values = &[
f64::NAN.to_bits(),
f64::INFINITY.to_bits(),
f64::NEG_INFINITY.to_bits(),
f64::MIN.to_bits(),
(-1.0f64).to_bits(),
(0.0f64).to_bits(),
(1.0f64).to_bits(),
f64::MAX.to_bits(),
];
DiffValue::F64(biased_arbitrary_value(u, known_f64_values)?)
}
V128 => DiffValue::V128(biased_arbitrary_value(u, KNOWN_U128_VALUES)?),
};
arbitrary::Result::Ok(val)
}
}
const KNOWN_I32_VALUES: &[i32] = &[i32::MIN, -1, 0, 1, i32::MAX];
const KNOWN_I64_VALUES: &[i64] = &[i64::MIN, -1, 0, 1, i64::MAX];
const KNOWN_U128_VALUES: &[u128] = &[u128::MIN, 1, u128::MAX];
/// Helper function to pick a known value from the list of `known_values` half
/// the time.
fn biased_arbitrary_value<'a, T>(
u: &mut Unstructured<'a>,
known_values: &[T],
) -> arbitrary::Result<T>
where
T: Arbitrary<'a> + Copy,
{
let pick_from_known_values: bool = u.arbitrary()?;
if pick_from_known_values {
Ok(*u.choose(known_values)?)
} else {
u.arbitrary()
}
}
impl<'a> Arbitrary<'a> for DiffValue {
fn arbitrary(u: &mut Unstructured<'a>) -> arbitrary::Result<Self> {
let ty: DiffValueType = u.arbitrary()?;
DiffValue::arbitrary_of_type(u, ty)
}
}
impl Hash for DiffValue {
fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
self.ty().hash(state);
match self {
DiffValue::I32(n) => n.hash(state),
DiffValue::I64(n) => n.hash(state),
DiffValue::F32(n) => n.hash(state),
DiffValue::F64(n) => n.hash(state),
DiffValue::V128(n) => n.hash(state),
}
}
}
/// Implement equality checks. Note that floating-point values are not compared
/// bit-for-bit in the case of NaNs: because Wasm floating-point numbers may be
/// [arithmetic NaNs with arbitrary payloads] and Wasm operations are [not
/// required to propagate NaN payloads], we simply check that both sides are
/// NaNs here. We could be more strict, though: we could check that the NaN
/// signs are equal and that [canonical NaN payloads remain canonical].
///
/// [arithmetic NaNs with arbitrary payloads]:
/// https://webassembly.github.io/spec/core/bikeshed/index.html#floating-point%E2%91%A0
/// [not required to propagate NaN payloads]:
/// https://webassembly.github.io/spec/core/bikeshed/index.html#floating-point-operations%E2%91%A0
/// [canonical NaN payloads remain canonical]:
/// https://webassembly.github.io/spec/core/bikeshed/index.html#nan-propagation%E2%91%A0
impl PartialEq for DiffValue {
fn eq(&self, other: &Self) -> bool {
match (self, other) {
(Self::I32(l0), Self::I32(r0)) => l0 == r0,
(Self::I64(l0), Self::I64(r0)) => l0 == r0,
(Self::V128(l0), Self::V128(r0)) => l0 == r0,
(Self::F32(l0), Self::F32(r0)) => {
let l0 = f32::from_bits(*l0);
let r0 = f32::from_bits(*r0);
l0 == r0 || (l0.is_nan() && r0.is_nan())
}
(Self::F64(l0), Self::F64(r0)) => {
let l0 = f64::from_bits(*l0);
let r0 = f64::from_bits(*r0);
l0 == r0 || (l0.is_nan() && r0.is_nan())
}
_ => false,
}
}
}
/// Enumerate the supported value types.
#[derive(Clone, Debug, Arbitrary, Hash)]
#[allow(missing_docs)]
pub enum DiffValueType {
I32,
I64,
F32,
F64,
V128,
}
impl TryFrom<wasmtime::ValType> for DiffValueType {
type Error = &'static str;
fn try_from(ty: wasmtime::ValType) -> Result<Self, Self::Error> {
use wasmtime::ValType::*;
match ty {
I32 => Ok(Self::I32),
I64 => Ok(Self::I64),
F32 => Ok(Self::F32),
F64 => Ok(Self::F64),
V128 => Ok(Self::V128),
FuncRef => Err("unable to convert reference types"),
ExternRef => Err("unable to convert reference types"),
}
}
}

87
crates/fuzzing/src/oracles.rs
View File

@@ -10,14 +10,23 @@
//! When an oracle finds a bug, it should report it to the fuzzing engine by
//! panicking.
#[cfg(feature = "fuzz-spec-interpreter")]
pub mod diff_spec;
pub mod diff_wasmi;
pub mod diff_wasmtime;
pub mod dummy;
pub mod engine;
mod stacks;
use crate::generators;
use self::diff_wasmtime::WasmtimeInstance;
use self::engine::DiffInstance;
use crate::generators::{self, DiffValue};
use arbitrary::Arbitrary;
use log::debug;
pub use stacks::check_stacks;
use std::cell::Cell;
use std::collections::hash_map::DefaultHasher;
use std::hash::Hasher;
use std::rc::Rc;
use std::sync::atomic::{AtomicUsize, Ordering::SeqCst};
use std::sync::{Arc, Condvar, Mutex};
@@ -240,9 +249,10 @@ fn compile_module(
if let generators::InstanceAllocationStrategy::Pooling { .. } =
&config.wasmtime.strategy
{
// When using the pooling allocator, accept failures to compile when arbitrary
// table element limits have been exceeded as there is currently no way
// to constrain the generated module table types.
// When using the pooling allocator, accept failures to compile
// when arbitrary table element limits have been exceeded as
// there is currently no way to constrain the generated module
// table types.
let string = e.to_string();
if string.contains("minimum element size") {
return None;
@@ -250,7 +260,7 @@ fn compile_module(
// Allow modules-failing-to-compile which exceed the requested
// size for each instance. This is something that is difficult
// to control and ensure it always suceeds, so we simply have a
// to control and ensure it always succeeds, so we simply have a
// "random" instance size limit and if a module doesn't fit we
// move on to the next fuzz input.
if string.contains("instance allocation for this module requires") {
@@ -263,7 +273,17 @@ fn compile_module(
}
}
fn instantiate_with_dummy(store: &mut Store<StoreLimits>, module: &Module) -> Option<Instance> {
/// Create a Wasmtime [`Instance`] from a [`Module`] and fill in all imports
/// with dummy values (e.g., zeroed values, immediately-trapping functions).
/// Also, this function catches certain fuzz-related instantiation failures and
/// returns `None` instead of panicking.
///
/// TODO: we should implement tracing versions of these dummy imports that
/// record a trace of the order that imported functions were called in and with
/// what values. Like the results of exported functions, calls to imports should
/// also yield the same values for each configuration, and we should assert
/// that.
pub fn instantiate_with_dummy(store: &mut Store<StoreLimits>, module: &Module) -> Option<Instance> {
// Creation of imports can fail due to resource limit constraints, and then
// instantiation can naturally fail for a number of reasons as well. Bundle
// the two steps together to match on the error below.
@@ -279,12 +299,14 @@ fn instantiate_with_dummy(store: &mut Store<StoreLimits>, module: &Module) -> Op
// expected that fuzz-generated programs try to allocate lots of
// stuff.
if store.data().0.oom.get() {
log::debug!("failed to instantiate: OOM");
return None;
}
// Allow traps which can happen normally with `unreachable` or a
// timeout or such
if e.downcast_ref::<Trap>().is_some() {
if let Some(trap) = e.downcast_ref::<Trap>() {
log::debug!("failed to instantiate: {}", trap);
return None;
}
@@ -296,11 +318,13 @@ fn instantiate_with_dummy(store: &mut Store<StoreLimits>, module: &Module) -> Op
// rather than positional-based resolution
|| string.contains("incompatible import type")
{
log::debug!("failed to instantiate: {}", string);
return None;
}
// Also allow failures to instantiate as a result of hitting instance limits
if string.contains("concurrent instances has been reached") {
log::debug!("failed to instantiate: {}", string);
return None;
}
@@ -308,6 +332,55 @@ fn instantiate_with_dummy(store: &mut Store<StoreLimits>, module: &Module) -> Op
panic!("failed to instantiate: {:?}", e);
}
/// Evaluate the function identified by `name` in two different engine
/// instances--`lhs` and `rhs`.
///
/// # Panics
///
/// This will panic if the evaluation is different between engines (e.g.,
/// results are different, hashed instance is different, one side traps, etc.).
pub fn differential(
lhs: &mut dyn DiffInstance,
rhs: &mut WasmtimeInstance,
name: &str,
args: &[DiffValue],
) -> anyhow::Result<()> {
log::debug!("Evaluating: {}({:?})", name, args);
let lhs_results = lhs.evaluate(name, args);
log::debug!(" -> results on {}: {:?}", lhs.name(), &lhs_results);
let rhs_results = rhs.evaluate(name, args);
log::debug!(" -> results on {}: {:?}", rhs.name(), &rhs_results);
match (lhs_results, rhs_results) {
// If the evaluation succeeds, we compare the results.
(Ok(lhs_results), Ok(rhs_results)) => assert_eq!(lhs_results, rhs_results),
// Both sides failed--this is an acceptable result (e.g., both sides
// trap at a divide by zero). We could compare the error strings perhaps
// (since the `lhs` and `rhs` could be failing for different reasons)
// but this seems good enough for now.
(Err(_), Err(_)) => {}
// A real bug is found if only one side fails.
(Ok(_), Err(_)) => panic!("only the `rhs` ({}) failed for this input", rhs.name()),
(Err(_), Ok(_)) => panic!("only the `lhs` ({}) failed for this input", lhs.name()),
};
let hash = |i: &mut dyn DiffInstance| -> anyhow::Result<u64> {
let mut hasher = DefaultHasher::new();
i.hash(&mut hasher)?;
Ok(hasher.finish())
};
if lhs.is_hashable() && rhs.is_hashable() {
log::debug!("Hashing instances:");
let lhs_hash = hash(lhs)?;
log::debug!(" -> hash of {}: {:?}", lhs.name(), lhs_hash);
let rhs_hash = hash(rhs)?;
log::debug!(" -> hash of {}: {:?}", rhs.name(), rhs_hash);
assert_eq!(lhs_hash, rhs_hash);
}
Ok(())
}
/// Instantiate the given Wasm module with each `Config` and call all of its
/// exports. Modulo OOM, non-canonical NaNs, and usage of Wasm features that are
/// or aren't enabled for different configs, we should get the same results when

123
crates/fuzzing/src/oracles/diff_spec.rs Normal file
View File

@@ -0,0 +1,123 @@
//! Evaluate an exported Wasm function using the WebAssembly specification
//! reference interpreter.
use crate::generators::{DiffValue, ModuleConfig};
use crate::oracles::engine::{DiffEngine, DiffInstance};
use anyhow::{anyhow, bail, Result};
use wasm_spec_interpreter::Value;
/// A wrapper for `wasm-spec-interpreter` as a [`DiffEngine`].
pub struct SpecInterpreter;
impl SpecInterpreter {
/// Build a new [`SpecInterpreter`] but only if the configuration does not
/// rely on features that the current bindings (i.e.,
/// `wasm-spec-interpreter`) do not support.
pub fn new(config: &ModuleConfig) -> Result<Box<Self>> {
if config.config.reference_types_enabled {
bail!("the spec interpreter bindings do not support reference types")
}
if config.config.max_funcs > 1 {
// TODO
bail!("the spec interpreter bindings can only support one function for now")
}
if config.config.max_tables > 0 {
// TODO
bail!("the spec interpreter bindings do not fail as they should with out-of-bounds table accesses")
}
Ok(Box::new(Self))
}
}
impl DiffEngine for SpecInterpreter {
fn name(&self) -> &'static str {
"spec"
}
fn instantiate(&self, wasm: &[u8]) -> Result<Box<dyn DiffInstance>> {
// TODO: ideally we would avoid copying the module bytes here.
Ok(Box::new(SpecInstance {
wasm: wasm.to_vec(),
}))
}
}
struct SpecInstance {
wasm: Vec<u8>,
}
impl DiffInstance for SpecInstance {
fn name(&self) -> &'static str {
"spec"
}
fn evaluate(
&mut self,
_function_name: &str,
arguments: &[DiffValue],
) -> Result<Vec<DiffValue>> {
// The spec interpreter needs some work before it can fully support this
// interface:
// - TODO adapt `wasm-spec-interpreter` to use function name to select
// function to run
// - TODO adapt `wasm-spec-interpreter` to expose an "instance" with
// so we can hash memory, globals, etc.
let arguments = arguments.iter().map(Value::from).collect();
match wasm_spec_interpreter::interpret(&self.wasm, Some(arguments)) {
Ok(results) => Ok(results.into_iter().map(Value::into).collect()),
Err(err) => Err(anyhow!(err)),
}
}
fn is_hashable(&self) -> bool {
false
}
fn hash(&mut self, _state: &mut std::collections::hash_map::DefaultHasher) -> Result<()> {
unimplemented!()
}
}
impl From<&DiffValue> for Value {
fn from(v: &DiffValue) -> Self {
match *v {
DiffValue::I32(n) => Value::I32(n),
DiffValue::I64(n) => Value::I64(n),
DiffValue::F32(n) => Value::F32(n as i32),
DiffValue::F64(n) => Value::F64(n as i64),
DiffValue::V128(n) => Value::V128(n.to_le_bytes().to_vec()),
}
}
}
impl Into<DiffValue> for Value {
fn into(self) -> DiffValue {
match self {
Value::I32(n) => DiffValue::I32(n),
Value::I64(n) => DiffValue::I64(n),
Value::F32(n) => DiffValue::F32(n as u32),
Value::F64(n) => DiffValue::F64(n as u64),
Value::V128(n) => {
assert_eq!(n.len(), 16);
DiffValue::V128(u128::from_le_bytes(n.as_slice().try_into().unwrap()))
}
}
}
}
/// Set up the OCaml runtime for triggering its signal handler configuration.
///
/// Because both the OCaml runtime and Wasmtime set up signal handlers, we must
/// carefully decide when to instantiate them; this function allows us to
/// control when. Wasmtime uses these signal handlers for catching various
/// WebAssembly failures. On certain OSes (e.g. Linux `x86_64`), the signal
/// handlers interfere, observable as an uncaught `SIGSEGV`--not even caught by
/// libFuzzer.
///
/// This failure can be mitigated by always running Wasmtime second in
/// differential fuzzing. In some cases, however, this is not possible because
/// which engine will execute first is unknown. This function can be explicitly
/// executed first, e.g., during global initialization, to avoid this issue.
pub fn setup_ocaml_runtime() {
wasm_spec_interpreter::setup_ocaml_runtime();
}

178
crates/fuzzing/src/oracles/diff_wasmi.rs Normal file
View File

@@ -0,0 +1,178 @@
//! Evaluate an exported Wasm function using the wasmi interpreter.
use crate::generators::{DiffValue, ModuleConfig};
use crate::oracles::engine::{DiffEngine, DiffInstance};
use anyhow::{bail, Context, Result};
use std::hash::Hash;
/// A wrapper for `wasmi` as a [`DiffEngine`].
pub struct WasmiEngine;
impl WasmiEngine {
/// Build a new [`WasmiEngine`] but only if the configuration does not rely
/// on features that `wasmi` does not support.
pub fn new(config: &ModuleConfig) -> Result<Box<Self>> {
if config.config.reference_types_enabled {
bail!("wasmi does not support reference types")
}
if config.config.simd_enabled {
bail!("wasmi does not support SIMD")
}
if config.config.multi_value_enabled {
bail!("wasmi does not support multi-value")
}
if config.config.saturating_float_to_int_enabled {
bail!("wasmi does not support saturating float-to-int conversions")
}
if config.config.sign_extension_enabled {
bail!("wasmi does not support sign-extension")
}
Ok(Box::new(Self))
}
}
impl DiffEngine for WasmiEngine {
fn name(&self) -> &'static str {
"wasmi"
}
fn instantiate(&self, wasm: &[u8]) -> Result<Box<dyn DiffInstance>> {
let module = wasmi::Module::from_buffer(wasm).context("unable to validate Wasm module")?;
let instance = wasmi::ModuleInstance::new(&module, &wasmi::ImportsBuilder::default())
.context("unable to instantiate module in wasmi")?;
let instance = instance.assert_no_start();
let exports = list_export_names(wasm);
Ok(Box::new(WasmiInstance {
module,
exports,
instance,
}))
}
}
/// A wrapper for `wasmi` Wasm instances.
struct WasmiInstance {
#[allow(dead_code)] // reason = "the module must live as long as its reference"
module: wasmi::Module,
instance: wasmi::ModuleRef,
/// `wasmi`'s instances have no way of listing their exports so, in order to
/// properly hash the instance, we keep track of the export names.
exports: Vec<String>,
}
impl DiffInstance for WasmiInstance {
fn name(&self) -> &'static str {
"wasmi"
}
fn evaluate(&mut self, function_name: &str, arguments: &[DiffValue]) -> Result<Vec<DiffValue>> {
let arguments: Vec<_> = arguments.iter().map(wasmi::RuntimeValue::from).collect();
let export = self
.instance
.export_by_name(function_name)
.context(format!(
"unable to find function '{}' in wasmi instance",
function_name
))?;
let function = export.as_func().context("wasmi export is not a function")?;
let result = wasmi::FuncInstance::invoke(&function, &arguments, &mut wasmi::NopExternals)
.context("failed while invoking function in wasmi")?;
Ok(if let Some(result) = result {
vec![result.into()]
} else {
vec![]
})
}
fn is_hashable(&self) -> bool {
true
}
fn hash(&mut self, state: &mut std::collections::hash_map::DefaultHasher) -> Result<()> {
for export_name in &self.exports {
if let Some(export) = self.instance.export_by_name(export_name) {
match export {
wasmi::ExternVal::Func(_) => {}
wasmi::ExternVal::Table(_) => {} // TODO eventually we can hash whether the values are null or non-null.
wasmi::ExternVal::Memory(m) => {
// `wasmi` memory may be stored non-contiguously; copy
// it out to a contiguous chunk.
let mut buffer: Vec<u8> = vec![0; m.current_size().0 * 65536];
m.get_into(0, &mut buffer[..])
.expect("can access wasmi memory");
buffer.hash(state)
}
wasmi::ExternVal::Global(g) => {
let val: DiffValue = g.get().into();
val.hash(state);
}
}
} else {
panic!("unable to find export: {}", export_name)
}
}
Ok(())
}
}
/// List the names of all exported items in a binary Wasm module.
fn list_export_names(wasm: &[u8]) -> Vec<String> {
let mut exports = vec![];
for payload in wasmparser::Parser::new(0).parse_all(&wasm) {
match payload.unwrap() {
wasmparser::Payload::ExportSection(s) => {
for export in s {
exports.push(export.unwrap().name.to_string());
}
}
_ => {
// Ignore any other sections.
}
}
}
exports
}
impl From<&DiffValue> for wasmi::RuntimeValue {
fn from(v: &DiffValue) -> Self {
use wasmi::RuntimeValue::*;
match *v {
DiffValue::I32(n) => I32(n),
DiffValue::I64(n) => I64(n),
DiffValue::F32(n) => F32(wasmi::nan_preserving_float::F32::from_bits(n)),
DiffValue::F64(n) => F64(wasmi::nan_preserving_float::F64::from_bits(n)),
DiffValue::V128(_) => unimplemented!(),
}
}
}
impl Into<DiffValue> for wasmi::RuntimeValue {
fn into(self) -> DiffValue {
use wasmi::RuntimeValue::*;
match self {
I32(n) => DiffValue::I32(n),
I64(n) => DiffValue::I64(n),
F32(n) => DiffValue::F32(n.to_bits()),
F64(n) => DiffValue::F64(n.to_bits()),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_list_export_names() {
let wat = r#"(module
(func (export "a") (result i32) (i32.const 42))
(global (export "b") (mut i32) (i32.const 42))
(memory (export "c") 1 2 shared)
)"#;
let wasm = wat::parse_str(wat).unwrap();
assert_eq!(
list_export_names(&wasm),
vec!["a".to_string(), "b".to_string(), "c".to_string()],
);
}
}

159
crates/fuzzing/src/oracles/diff_wasmtime.rs Normal file
View File

@@ -0,0 +1,159 @@
//! Evaluate an exported Wasm function using Wasmtime.
use crate::generators::{self, DiffValue};
use crate::oracles::engine::DiffInstance;
use crate::oracles::{compile_module, engine::DiffEngine, instantiate_with_dummy, StoreLimits};
use anyhow::{Context, Result};
use std::hash::Hash;
use std::slice;
use wasmtime::{AsContextMut, Extern, FuncType, Instance, Module, Store, Val};
/// A wrapper for using Wasmtime as a [`DiffEngine`].
pub struct WasmtimeEngine {
pub(crate) config: generators::Config,
}
impl WasmtimeEngine {
/// Merely store the configuration; the engine is actually constructed
/// later. Ideally the store and engine could be built here but
/// `compile_module` takes a [`generators::Config`]; TODO re-factor this if
/// that ever changes.
pub fn new(config: &generators::Config) -> Result<Box<Self>> {
Ok(Box::new(Self {
config: config.clone(),
}))
}
}
impl DiffEngine for WasmtimeEngine {
fn name(&self) -> &'static str {
"wasmtime"
}
fn instantiate(&self, wasm: &[u8]) -> Result<Box<dyn DiffInstance>> {
let store = self.config.to_store();
let module = compile_module(store.engine(), wasm, true, &self.config).unwrap();
let instance = WasmtimeInstance::new(store, module)?;
Ok(Box::new(instance))
}
}
/// A wrapper around a Wasmtime instance.
///
/// The Wasmtime engine constructs a new store and compiles an instance of a
/// Wasm module.
pub struct WasmtimeInstance {
store: Store<StoreLimits>,
instance: Instance,
}
impl WasmtimeInstance {
/// Instantiate a new Wasmtime instance.
pub fn new(mut store: Store<StoreLimits>, module: Module) -> Result<Self> {
let instance = instantiate_with_dummy(&mut store, &module)
.context("unable to instantiate module in wasmtime")?;
Ok(Self { store, instance })
}
/// Retrieve the names and types of all exported functions in the instance.
///
/// This is useful for evaluating each exported function with different
/// values. The [`DiffInstance`] trait asks for the function name and we
/// need to know the function signature in order to pass in the right
/// arguments.
pub fn exported_functions(&mut self) -> Vec<(String, FuncType)> {
let exported_functions = self
.instance
.exports(&mut self.store)
.map(|e| (e.name().to_owned(), e.into_func()))
.filter_map(|(n, f)| f.map(|f| (n, f)))
.collect::<Vec<_>>();
exported_functions
.into_iter()
.map(|(n, f)| (n, f.ty(&self.store)))
.collect()
}
}
impl DiffInstance for WasmtimeInstance {
fn name(&self) -> &'static str {
"wasmtime"
}
fn evaluate(&mut self, function_name: &str, arguments: &[DiffValue]) -> Result<Vec<DiffValue>> {
let arguments: Vec<_> = arguments.iter().map(Val::from).collect();
let function = self
.instance
.get_func(&mut self.store, function_name)
.expect("unable to access exported function");
let ty = function.ty(&self.store);
let mut results = vec![Val::I32(0); ty.results().len()];
function.call(&mut self.store, &arguments, &mut results)?;
let results = results.into_iter().map(Val::into).collect();
Ok(results)
}
fn is_hashable(&self) -> bool {
true
}
fn hash(&mut self, state: &mut std::collections::hash_map::DefaultHasher) -> Result<()> {
let exports: Vec<_> = self
.instance
.exports(self.store.as_context_mut())
.map(|e| e.into_extern())
.collect();
for e in exports {
match e {
Extern::Global(g) => {
let val: DiffValue = g.get(&mut self.store).into();
val.hash(state)
}
Extern::Memory(m) => {
let data = m.data(&mut self.store);
data.hash(state)
}
Extern::SharedMemory(m) => {
let data = unsafe { slice::from_raw_parts(m.data() as *mut u8, m.data_size()) };
data.hash(state)
}
Extern::Table(_) => {
// TODO: it's unclear whether it is worth it to iterate
// through the table and hash the values.
}
Extern::Func(_) => {
// Note: no need to hash exported functions.
}
}
}
Ok(())
}
}
impl From<&DiffValue> for Val {
fn from(v: &DiffValue) -> Self {
match *v {
DiffValue::I32(n) => Val::I32(n),
DiffValue::I64(n) => Val::I64(n),
DiffValue::F32(n) => Val::F32(n),
DiffValue::F64(n) => Val::F64(n),
DiffValue::V128(n) => Val::V128(n),
}
}
}
impl Into<DiffValue> for Val {
fn into(self) -> DiffValue {
match self {
Val::I32(n) => DiffValue::I32(n),
Val::I64(n) => DiffValue::I64(n),
Val::F32(n) => DiffValue::F32(n),
Val::F64(n) => DiffValue::F64(n),
Val::V128(n) => DiffValue::V128(n),
Val::FuncRef(_) => unimplemented!(),
Val::ExternRef(_) => unimplemented!(),
}
}
}

75
crates/fuzzing/src/oracles/engine.rs Normal file
View File

@@ -0,0 +1,75 @@
//! Define the interface for differential evaluation of Wasm functions.
use crate::generators::{Config, DiffValue};
use crate::oracles::{diff_wasmi::WasmiEngine, diff_wasmtime::WasmtimeEngine};
use arbitrary::Unstructured;
use std::collections::hash_map::DefaultHasher;
/// Pick one of the engines implemented in this module that is compatible with
/// the Wasm features passed in `features` and, when fuzzing Wasmtime against
/// itself, an existing `wasmtime_engine`.
pub fn choose(
u: &mut Unstructured<'_>,
existing_config: &Config,
) -> arbitrary::Result<Box<dyn DiffEngine>> {
// Filter out any engines that cannot match the given configuration.
let mut engines: Vec<Box<dyn DiffEngine>> = vec![];
let mut config: Config = u.arbitrary()?; // TODO change to WasmtimeConfig
config.make_compatible_with(&existing_config);
if let Result::Ok(e) = WasmtimeEngine::new(&config) {
engines.push(e)
}
if let Result::Ok(e) = WasmiEngine::new(&existing_config.module_config) {
engines.push(e)
}
#[cfg(feature = "fuzz-spec-interpreter")]
if let Result::Ok(e) =
crate::oracles::diff_spec::SpecInterpreter::new(&existing_config.module_config)
{
engines.push(e)
}
// Choose one of the remaining engines.
if !engines.is_empty() {
let index: usize = u.int_in_range(0..=engines.len() - 1)?;
let engine = engines.swap_remove(index);
log::debug!("selected engine: {}", engine.name());
Ok(engine)
} else {
panic!("no engines to pick from");
// Err(arbitrary::Error::EmptyChoose)
}
}
/// Provide a way to instantiate Wasm modules.
pub trait DiffEngine {
/// Return the name of the engine.
fn name(&self) -> &'static str;
/// Create a new instance with the given engine.
fn instantiate(&self, wasm: &[u8]) -> anyhow::Result<Box<dyn DiffInstance>>;
}
/// Provide a way to evaluate Wasm functions--a Wasm instance implemented by a
/// specific engine (i.e., compiler or interpreter).
pub trait DiffInstance {
/// Return the name of the engine behind this instance.
fn name(&self) -> &'static str;
/// Evaluate an exported function with the given values.
fn evaluate(
&mut self,
function_name: &str,
arguments: &[DiffValue],
) -> anyhow::Result<Vec<DiffValue>>;
/// Check if instances of this kind are actually hashable--not all engines
/// support this.
fn is_hashable(&self) -> bool;
/// If the instance `is_hashable()`, this method will try to hash the
/// following exported items in the instance: globals, memory.
///
/// TODO allow more types of hashers.
fn hash(&mut self, state: &mut DefaultHasher) -> anyhow::Result<()>;
}

10
crates/fuzzing/wasm-spec-interpreter/src/with_library.rs
View File

@@ -71,6 +71,16 @@ mod ocaml_bindings {
}
}
/// Initialize a persistent OCaml runtime.
///
/// When used for fuzzing differentially with engines that also use signal
/// handlers, this function provides a way to explicitly set up the OCaml
/// runtime and configure its signal handlers.
pub fn setup_ocaml_runtime() {
let _lock = INTERPRET.lock().unwrap();
OCamlRuntime::init_persistent();
}
#[cfg(test)]
mod tests {
use super::*;

3
crates/fuzzing/wasm-spec-interpreter/src/without_library.rs
View File

@@ -15,3 +15,6 @@ pub fn interpret(_module: &[u8], _parameters: Option<Vec<Value>>) -> Result<Vec<
library; re-compile with the dependencies listed in its README.md."
);
}
#[allow(dead_code)]
pub fn setup_ocaml_runtime() {}

7
fuzz/Cargo.toml
View File

@@ -60,6 +60,13 @@ path = "fuzz_targets/differential.rs"
test = false
doc = false
[[bin]]
name = "differential_meta"
path = "fuzz_targets/differential_meta.rs"
test = false
doc = false
[[bin]]
name = "differential_spec"
path = "fuzz_targets/differential_spec.rs"

109
fuzz/fuzz_targets/differential_meta.rs Normal file
View File

@@ -0,0 +1,109 @@
#![no_main]
use libfuzzer_sys::arbitrary::{Result, Unstructured};
use libfuzzer_sys::fuzz_target;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::SeqCst;
use std::sync::Once;
use wasmtime_fuzzing::generators::{Config, DiffValue, SingleInstModule};
use wasmtime_fuzzing::oracles::diff_spec;
use wasmtime_fuzzing::oracles::diff_wasmtime::WasmtimeInstance;
use wasmtime_fuzzing::oracles::{differential, engine, log_wasm};
// Upper limit on the number of invocations for each WebAssembly function
// executed by this fuzz target.
const NUM_INVOCATIONS: usize = 5;
// Keep track of how many WebAssembly modules we actually executed (i.e. ran to
// completion) versus how many were tried.
static TOTAL_INVOCATIONS: AtomicUsize = AtomicUsize::new(0);
static TOTAL_SUCCESSES: AtomicUsize = AtomicUsize::new(0);
static TOTAL_ATTEMPTED: AtomicUsize = AtomicUsize::new(0);
// The spec interpreter requires special one-time setup.
static SETUP: Once = Once::new();
fuzz_target!(|data: &[u8]| {
// To avoid a uncaught `SIGSEGV` due to signal handlers; see comments on
// `setup_ocaml_runtime`.
SETUP.call_once(|| diff_spec::setup_ocaml_runtime());
// Errors in `run` have to do with not enough input in `data`, which we
// ignore here since it doesn't affect how we'd like to fuzz.
drop(run(&data));
});
fn run(data: &[u8]) -> Result<()> {
let successes = TOTAL_SUCCESSES.load(SeqCst);
let attempts = TOTAL_ATTEMPTED.fetch_add(1, SeqCst);
if attempts > 1 && attempts % 1_000 == 0 {
println!("=== Execution rate ({} successes / {} attempted modules): {}% (total invocations: {}) ===",
successes,
attempts,
successes as f64 / attempts as f64 * 100f64,
TOTAL_INVOCATIONS.load(SeqCst)
);
}
let mut u = Unstructured::new(data);
let mut config: Config = u.arbitrary()?;
config.set_differential_config();
// Generate the Wasm module.
let wasm = if u.arbitrary()? {
// TODO figure out if this always eats up the rest of the unstructured;
// can we limit the number of instructions/functions.
let module = config.generate(&mut u, Some(1000))?;
module.to_bytes()
} else {
let module = SingleInstModule::new(&mut u, &mut config.module_config)?;
module.to_bytes()
};
log_wasm(&wasm);
// Choose a left-hand side Wasm engine.
let lhs = engine::choose(&mut u, &config)?;
let lhs_instance = lhs.instantiate(&wasm);
// Choose a right-hand side Wasm engine--this will always be Wasmtime.
let rhs_store = config.to_store();
let rhs_module = wasmtime::Module::new(rhs_store.engine(), &wasm).unwrap();
let rhs_instance = WasmtimeInstance::new(rhs_store, rhs_module);
// If we fail to instantiate, check that both sides do.
let (mut lhs_instance, mut rhs_instance) = match (lhs_instance, rhs_instance) {
(Ok(l), Ok(r)) => (l, r),
(Err(_), Err(_)) => return Ok(()), // TODO match the error messages.
(l, r) => panic!(
"failed to instantiate only one side: {:?} != {:?}",
l.err(),
r.err()
),
};
// Call each exported function with different sets of arguments.
for (name, signature) in rhs_instance.exported_functions() {
let mut invocations = 0;
loop {
let arguments = signature
.params()
.map(|t| DiffValue::arbitrary_of_type(&mut u, t.try_into().unwrap()))
.collect::<Result<Vec<_>>>()?;
differential(lhs_instance.as_mut(), &mut rhs_instance, &name, &arguments)
.expect("failed to run differential evaluation");
// We evaluate the same function with different arguments until we
// hit a predetermined limit or we run out of unstructured data--it
// does not make sense to re-evaluate the same arguments over and
// over.
invocations += 1;
TOTAL_INVOCATIONS.fetch_add(1, SeqCst);
if invocations > NUM_INVOCATIONS || u.is_empty() {
break;
}
}
}
TOTAL_SUCCESSES.fetch_add(1, SeqCst);
Ok(())
}