* Expose memory-related options in `Config` This commit was initially motivated by looking more into #1501, but it ended up balooning a bit after finding a few issues. The high-level items in this commit are: * New configuration options via `wasmtime::Config` are exposed to configure the tunable limits of how memories are allocated and such. * The `MemoryCreator` trait has been updated to accurately reflect the required allocation characteristics that JIT code expects. * A bug has been fixed in the cranelift wasm code generation where if no guard page was present bounds checks weren't accurately performed. The new `Config` methods allow tuning the memory allocation characteristics of wasmtime. Currently 64-bit platforms will reserve 6GB chunks of memory for each linear memory, but by tweaking various config options you can change how this is allocate, perhaps at the cost of slower JIT code since it needs more bounds checks. The methods are intended to be pretty thoroughly documented as to the effect they have on the JIT code and what values you may wish to select. These new methods have been added to the spectest fuzzer to ensure that various configuration values for these methods don't affect correctness. The `MemoryCreator` trait previously only allocated memories with a `MemoryType`, but this didn't actually reflect the guarantees that JIT code expected. JIT code is generated with an assumption about the minimum size of the guard region, as well as whether memory is static or dynamic (whether the base pointer can be relocated). These properties must be upheld by custom allocation engines for JIT code to perform correctly, so extra parameters have been added to `MemoryCreator::new_memory` to reflect this. Finally the fuzzing with `Config` turned up an issue where if no guard pages present the wasm code wouldn't correctly bounds-check memory accesses. The issue here was that with a guard page we only need to bounds-check the first byte of access, but without a guard page we need to bounds-check the last byte of access. This meant that the code generation needed to account for the size of the memory operation (load/store) and use this as the offset-to-check in the no-guard-page scenario. I've attempted to make the various comments in cranelift a bit more exhaustive too to hopefully make it a bit clearer for future readers! Closes #1501 * Review comments * Update a comment
157 lines
5.1 KiB
Rust
157 lines
5.1 KiB
Rust
//! Test case generators.
|
|
//!
|
|
//! Test case generators take raw, unstructured input from a fuzzer
|
|
//! (e.g. libFuzzer) and translate that into a structured test case (e.g. a
|
|
//! valid Wasm binary).
|
|
//!
|
|
//! These are generally implementations of the `Arbitrary` trait, or some
|
|
//! wrapper over an external tool, such that the wrapper implements the
|
|
//! `Arbitrary` trait for the wrapped external tool.
|
|
|
|
#[cfg(feature = "binaryen")]
|
|
pub mod api;
|
|
|
|
use arbitrary::{Arbitrary, Unstructured};
|
|
|
|
/// A Wasm test case generator that is powered by Binaryen's `wasm-opt -ttf`.
|
|
#[derive(Clone)]
|
|
#[cfg(feature = "binaryen")]
|
|
pub struct WasmOptTtf {
|
|
/// The raw, encoded Wasm bytes.
|
|
pub wasm: Vec<u8>,
|
|
}
|
|
|
|
#[cfg(feature = "binaryen")]
|
|
impl std::fmt::Debug for WasmOptTtf {
|
|
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
|
|
write!(
|
|
f,
|
|
"WasmOptTtf {{ wasm: wat::parse_str(r###\"\n{}\n\"###).unwrap() }}",
|
|
wasmprinter::print_bytes(&self.wasm).expect("valid wasm should always disassemble")
|
|
)
|
|
}
|
|
}
|
|
|
|
#[cfg(feature = "binaryen")]
|
|
impl Arbitrary for WasmOptTtf {
|
|
fn arbitrary(input: &mut arbitrary::Unstructured) -> arbitrary::Result<Self> {
|
|
crate::init_fuzzing();
|
|
let seed: Vec<u8> = Arbitrary::arbitrary(input)?;
|
|
let module = binaryen::tools::translate_to_fuzz_mvp(&seed);
|
|
let wasm = module.write();
|
|
Ok(WasmOptTtf { wasm })
|
|
}
|
|
|
|
fn arbitrary_take_rest(input: arbitrary::Unstructured) -> arbitrary::Result<Self> {
|
|
crate::init_fuzzing();
|
|
let seed: Vec<u8> = Arbitrary::arbitrary_take_rest(input)?;
|
|
let module = binaryen::tools::translate_to_fuzz_mvp(&seed);
|
|
let wasm = module.write();
|
|
Ok(WasmOptTtf { wasm })
|
|
}
|
|
|
|
fn size_hint(depth: usize) -> (usize, Option<usize>) {
|
|
<Vec<u8> as Arbitrary>::size_hint(depth)
|
|
}
|
|
}
|
|
|
|
/// A description of configuration options that we should do differential
|
|
/// testing between.
|
|
#[derive(Arbitrary, Clone, Debug, PartialEq, Eq, Hash)]
|
|
pub struct DifferentialConfig {
|
|
strategy: DifferentialStrategy,
|
|
opt_level: OptLevel,
|
|
}
|
|
|
|
impl DifferentialConfig {
|
|
/// Convert this differential fuzzing config into a `wasmtime::Config`.
|
|
pub fn to_wasmtime_config(&self) -> anyhow::Result<wasmtime::Config> {
|
|
let mut config = crate::fuzz_default_config(match self.strategy {
|
|
DifferentialStrategy::Cranelift => wasmtime::Strategy::Cranelift,
|
|
DifferentialStrategy::Lightbeam => wasmtime::Strategy::Lightbeam,
|
|
})?;
|
|
config.cranelift_opt_level(self.opt_level.to_wasmtime());
|
|
Ok(config)
|
|
}
|
|
}
|
|
|
|
#[derive(Arbitrary, Clone, Debug, PartialEq, Eq, Hash)]
|
|
enum DifferentialStrategy {
|
|
Cranelift,
|
|
Lightbeam,
|
|
}
|
|
|
|
#[derive(Arbitrary, Clone, Debug, PartialEq, Eq, Hash)]
|
|
enum OptLevel {
|
|
None,
|
|
Speed,
|
|
SpeedAndSize,
|
|
}
|
|
|
|
impl OptLevel {
|
|
fn to_wasmtime(&self) -> wasmtime::OptLevel {
|
|
match self {
|
|
OptLevel::None => wasmtime::OptLevel::None,
|
|
OptLevel::Speed => wasmtime::OptLevel::Speed,
|
|
OptLevel::SpeedAndSize => wasmtime::OptLevel::SpeedAndSize,
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Implementation of generating a `wasmtime::Config` arbitrarily
|
|
#[derive(Arbitrary, Debug)]
|
|
pub struct Config {
|
|
opt_level: OptLevel,
|
|
debug_verifier: bool,
|
|
debug_info: bool,
|
|
canonicalize_nans: bool,
|
|
interruptable: bool,
|
|
|
|
// Note that we use 32-bit values here to avoid blowing the 64-bit address
|
|
// space by requesting ungodly-large sizes/guards.
|
|
static_memory_maximum_size: Option<u32>,
|
|
static_memory_guard_size: Option<u32>,
|
|
dynamic_memory_guard_size: Option<u32>,
|
|
}
|
|
|
|
impl Config {
|
|
/// Converts this to a `wasmtime::Config` object
|
|
pub fn to_wasmtime(&self) -> wasmtime::Config {
|
|
let mut cfg = wasmtime::Config::new();
|
|
cfg.debug_info(self.debug_info)
|
|
.static_memory_maximum_size(self.static_memory_maximum_size.unwrap_or(0).into())
|
|
.static_memory_guard_size(self.static_memory_guard_size.unwrap_or(0).into())
|
|
.dynamic_memory_guard_size(self.dynamic_memory_guard_size.unwrap_or(0).into())
|
|
.cranelift_nan_canonicalization(self.canonicalize_nans)
|
|
.cranelift_debug_verifier(self.debug_verifier)
|
|
.cranelift_opt_level(self.opt_level.to_wasmtime())
|
|
.interruptable(self.interruptable);
|
|
return cfg;
|
|
}
|
|
}
|
|
|
|
include!(concat!(env!("OUT_DIR"), "/spectests.rs"));
|
|
|
|
/// A spec test from the upstream wast testsuite, arbitrarily chosen from the
|
|
/// list of known spec tests.
|
|
#[derive(Debug)]
|
|
pub struct SpecTest {
|
|
/// The filename of the spec test
|
|
pub file: &'static str,
|
|
/// The `*.wast` contents of the spec test
|
|
pub contents: &'static str,
|
|
}
|
|
|
|
impl Arbitrary for SpecTest {
|
|
fn arbitrary(u: &mut Unstructured) -> arbitrary::Result<Self> {
|
|
// NB: this does get a uniform value in the provided range.
|
|
let i = u.int_in_range(0..=FILES.len() - 1)?;
|
|
let (file, contents) = FILES[i];
|
|
Ok(SpecTest { file, contents })
|
|
}
|
|
|
|
fn size_hint(_depth: usize) -> (usize, Option<usize>) {
|
|
(1, Some(std::mem::size_of::<usize>()))
|
|
}
|
|
}
|