Files
wasmtime/crates/fuzzing/src/generators.rs
Alex Crichton 363cd2d20f Expose memory-related options in Config (#1513)
* Expose memory-related options in `Config`

This commit was initially motivated by looking more into #1501, but it
ended up balooning a bit after finding a few issues. The high-level
items in this commit are:

* New configuration options via `wasmtime::Config` are exposed to
  configure the tunable limits of how memories are allocated and such.
* The `MemoryCreator` trait has been updated to accurately reflect the
  required allocation characteristics that JIT code expects.
* A bug has been fixed in the cranelift wasm code generation where if no
  guard page was present bounds checks weren't accurately performed.

The new `Config` methods allow tuning the memory allocation
characteristics of wasmtime. Currently 64-bit platforms will reserve 6GB
chunks of memory for each linear memory, but by tweaking various config
options you can change how this is allocate, perhaps at the cost of
slower JIT code since it needs more bounds checks. The methods are
intended to be pretty thoroughly documented as to the effect they have
on the JIT code and what values you may wish to select. These new
methods have been added to the spectest fuzzer to ensure that various
configuration values for these methods don't affect correctness.

The `MemoryCreator` trait previously only allocated memories with a
`MemoryType`, but this didn't actually reflect the guarantees that JIT
code expected. JIT code is generated with an assumption about the
minimum size of the guard region, as well as whether memory is static or
dynamic (whether the base pointer can be relocated). These properties
must be upheld by custom allocation engines for JIT code to perform
correctly, so extra parameters have been added to
`MemoryCreator::new_memory` to reflect this.

Finally the fuzzing with `Config` turned up an issue where if no guard
pages present the wasm code wouldn't correctly bounds-check memory
accesses. The issue here was that with a guard page we only need to
bounds-check the first byte of access, but without a guard page we need
to bounds-check the last byte of access. This meant that the code
generation needed to account for the size of the memory operation
(load/store) and use this as the offset-to-check in the no-guard-page
scenario. I've attempted to make the various comments in cranelift a bit
more exhaustive too to hopefully make it a bit clearer for future
readers!

Closes #1501

* Review comments

* Update a comment
2020-04-29 17:10:00 -07:00

157 lines
5.1 KiB
Rust

//! Test case generators.
//!
//! Test case generators take raw, unstructured input from a fuzzer
//! (e.g. libFuzzer) and translate that into a structured test case (e.g. a
//! valid Wasm binary).
//!
//! These are generally implementations of the `Arbitrary` trait, or some
//! wrapper over an external tool, such that the wrapper implements the
//! `Arbitrary` trait for the wrapped external tool.
#[cfg(feature = "binaryen")]
pub mod api;
use arbitrary::{Arbitrary, Unstructured};
/// A Wasm test case generator that is powered by Binaryen's `wasm-opt -ttf`.
#[derive(Clone)]
#[cfg(feature = "binaryen")]
pub struct WasmOptTtf {
/// The raw, encoded Wasm bytes.
pub wasm: Vec<u8>,
}
#[cfg(feature = "binaryen")]
impl std::fmt::Debug for WasmOptTtf {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(
f,
"WasmOptTtf {{ wasm: wat::parse_str(r###\"\n{}\n\"###).unwrap() }}",
wasmprinter::print_bytes(&self.wasm).expect("valid wasm should always disassemble")
)
}
}
#[cfg(feature = "binaryen")]
impl Arbitrary for WasmOptTtf {
fn arbitrary(input: &mut arbitrary::Unstructured) -> arbitrary::Result<Self> {
crate::init_fuzzing();
let seed: Vec<u8> = Arbitrary::arbitrary(input)?;
let module = binaryen::tools::translate_to_fuzz_mvp(&seed);
let wasm = module.write();
Ok(WasmOptTtf { wasm })
}
fn arbitrary_take_rest(input: arbitrary::Unstructured) -> arbitrary::Result<Self> {
crate::init_fuzzing();
let seed: Vec<u8> = Arbitrary::arbitrary_take_rest(input)?;
let module = binaryen::tools::translate_to_fuzz_mvp(&seed);
let wasm = module.write();
Ok(WasmOptTtf { wasm })
}
fn size_hint(depth: usize) -> (usize, Option<usize>) {
<Vec<u8> as Arbitrary>::size_hint(depth)
}
}
/// A description of configuration options that we should do differential
/// testing between.
#[derive(Arbitrary, Clone, Debug, PartialEq, Eq, Hash)]
pub struct DifferentialConfig {
strategy: DifferentialStrategy,
opt_level: OptLevel,
}
impl DifferentialConfig {
/// Convert this differential fuzzing config into a `wasmtime::Config`.
pub fn to_wasmtime_config(&self) -> anyhow::Result<wasmtime::Config> {
let mut config = crate::fuzz_default_config(match self.strategy {
DifferentialStrategy::Cranelift => wasmtime::Strategy::Cranelift,
DifferentialStrategy::Lightbeam => wasmtime::Strategy::Lightbeam,
})?;
config.cranelift_opt_level(self.opt_level.to_wasmtime());
Ok(config)
}
}
#[derive(Arbitrary, Clone, Debug, PartialEq, Eq, Hash)]
enum DifferentialStrategy {
Cranelift,
Lightbeam,
}
#[derive(Arbitrary, Clone, Debug, PartialEq, Eq, Hash)]
enum OptLevel {
None,
Speed,
SpeedAndSize,
}
impl OptLevel {
fn to_wasmtime(&self) -> wasmtime::OptLevel {
match self {
OptLevel::None => wasmtime::OptLevel::None,
OptLevel::Speed => wasmtime::OptLevel::Speed,
OptLevel::SpeedAndSize => wasmtime::OptLevel::SpeedAndSize,
}
}
}
/// Implementation of generating a `wasmtime::Config` arbitrarily
#[derive(Arbitrary, Debug)]
pub struct Config {
opt_level: OptLevel,
debug_verifier: bool,
debug_info: bool,
canonicalize_nans: bool,
interruptable: bool,
// Note that we use 32-bit values here to avoid blowing the 64-bit address
// space by requesting ungodly-large sizes/guards.
static_memory_maximum_size: Option<u32>,
static_memory_guard_size: Option<u32>,
dynamic_memory_guard_size: Option<u32>,
}
impl Config {
/// Converts this to a `wasmtime::Config` object
pub fn to_wasmtime(&self) -> wasmtime::Config {
let mut cfg = wasmtime::Config::new();
cfg.debug_info(self.debug_info)
.static_memory_maximum_size(self.static_memory_maximum_size.unwrap_or(0).into())
.static_memory_guard_size(self.static_memory_guard_size.unwrap_or(0).into())
.dynamic_memory_guard_size(self.dynamic_memory_guard_size.unwrap_or(0).into())
.cranelift_nan_canonicalization(self.canonicalize_nans)
.cranelift_debug_verifier(self.debug_verifier)
.cranelift_opt_level(self.opt_level.to_wasmtime())
.interruptable(self.interruptable);
return cfg;
}
}
include!(concat!(env!("OUT_DIR"), "/spectests.rs"));
/// A spec test from the upstream wast testsuite, arbitrarily chosen from the
/// list of known spec tests.
#[derive(Debug)]
pub struct SpecTest {
/// The filename of the spec test
pub file: &'static str,
/// The `*.wast` contents of the spec test
pub contents: &'static str,
}
impl Arbitrary for SpecTest {
fn arbitrary(u: &mut Unstructured) -> arbitrary::Result<Self> {
// NB: this does get a uniform value in the provided range.
let i = u.int_in_range(0..=FILES.len() - 1)?;
let (file, contents) = FILES[i];
Ok(SpecTest { file, contents })
}
fn size_hint(_depth: usize) -> (usize, Option<usize>) {
(1, Some(std::mem::size_of::<usize>()))
}
}