5ec92d59d2
* [fuzz] Add `Module` enum, refactor `ModuleConfig` This change adds a way to create either a single-instruction module or a regular (big) `wasm-smith` module. It has some slight refactorings in preparation for the use of this new code. * [fuzz] Add `DiffValue` for differential evaluation In order to evaluate functions with randomly-generated values, we needed a common way to generate these values. Using the Wasmtime `Val` type is not great because we would like to be able to implement various traits on the new value type, e.g., to convert `Into` and `From` boxed values of other engines we differentially fuzz against. This new type, `DiffValue`, gives us a common ground for all the conversions and comparisons between the other engine types. * [fuzz] Add interface for differential engines In order to randomly choose an engine to fuzz against, we expect all of the engines to meet a common interface. The traits in this commit allow us to instantiate a module from its binary form, evaluate exported functions, and (possibly) hash the exported items of the instance. This change has some missing pieces, though: - the `wasm-spec-interpreter` needs some work to be able to create instances, evaluate a function by name, and expose exported items - the `v8` engine is not implemented yet due to the complexity of its Rust lifetimes * [fuzz] Use `ModuleFeatures` instead of existing configuration When attempting to use both wasm-smith and single-instruction modules, there is a mismatch in how we communicate what an engine must be able to support. In the first case, we could use the `ModuleConfig`, a wrapper for wasm-smith's `SwarmConfig`, but single-instruction modules do not have a `SwarmConfig`--the many options simply don't apply. Here, we instead add `ModuleFeatures` and adapt a `ModuleConfig` to that. `ModuleFeatures` then becomes the way to communicate what features an engine must support to evaluate functions in a module. * [fuzz] Add a new fuzz target using the meta-differential oracle This change adds the `differential_meta` target to the list of fuzz targets. I expect that sometime soon this could replace the other `differential*` targets, as it almost checks all the things those check. The major missing piece is that currently it only chooses single-instruction modules instead of also generating arbitrary modules using `wasm-smith`. Also, this change adds the concept of an ignorable error: some differential engines will choke with certain inputs (e.g., `wasmi` might have an old opcode mapping) which we do not want to flag as fuzz bugs. Here we wrap those errors in `DiffIgnoreError` and then use a new helper trait, `DiffIgnorable`, to downcast and inspect the `anyhow` error to only panic on non-ignorable errors; the ignorable errors are converted to one of the `arbitrary::Error` variants, which we already ignore. * [fuzz] Compare `DiffValue` NaNs more leniently Because arithmetic NaNs can contain arbitrary payload bits, checking that two differential executions should produce the same result should relax the comparison of the `F32` and `F64` types (and eventually `V128` as well... TODO). This change adds several considerations, however, so that in the future we make the comparison a bit stricter, e.g., re: canonical NaNs. This change, however, just matches the current logic used by other fuzz targets. * review: allow hashing mutate the instance state @alexcrichton requested that the interface be adapted to accommodate Wasmtime's API, in which even reading from an instance could trigger mutation of the store. * review: refactor where configurations are made compatible See @alexcrichton's [suggestion](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928974376). * review: convert `DiffValueType` using `TryFrom` See @alexcrichton's [comment](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928962394). * review: adapt target implementation to Wasmtime-specific RHS This change is joint work with @alexcrichton to adapt the structure of the fuzz target to his comments [here](https://github.com/bytecodealliance/wasmtime/pull/4515#pullrequestreview-1073247791). This change: - removes `ModuleFeatures` and the `Module` enum (for big and small modules) - upgrades `SingleInstModule` to filter out cases that are not valid for a given `ModuleConfig` - adds `DiffEngine::name()` - constructs each `DiffEngine` using a `ModuleConfig`, eliminating `DiffIgnoreError` completely - prints an execution rate to the `differential_meta` target Still TODO: - `get_exported_function_signatures` could be re-written in terms of the Wasmtime API instead `wasmparser` - the fuzzer crashes eventually, we think due to the signal handler interference between OCaml and Wasmtime - the spec interpreter has several cases that we skip for now but could be fuzzed with further work Co-authored-by: Alex Crichton <alex@alexcrichton.com> * fix: avoid SIGSEGV by explicitly initializing OCaml runtime first * review: use Wasmtime's API to retrieve exported functions Co-authored-by: Alex Crichton <alex@alexcrichton.com>
136 lines
2.8 KiB
TOML
136 lines
2.8 KiB
TOML
[package]
|
|
name = "wasmtime-fuzz"
|
|
version = "0.0.0"
|
|
authors = ["The Wasmtime Project Developers"]
|
|
edition = "2021"
|
|
publish = false
|
|
|
|
[package.metadata]
|
|
cargo-fuzz = true
|
|
|
|
[dependencies]
|
|
anyhow = { version = "1.0.19" }
|
|
arbitrary = { version = "1.1.0", features = ["derive"] }
|
|
cranelift-codegen = { path = "../cranelift/codegen", features = ["incremental-cache"] }
|
|
cranelift-reader = { path = "../cranelift/reader" }
|
|
cranelift-wasm = { path = "../cranelift/wasm" }
|
|
cranelift-filetests = { path = "../cranelift/filetests" }
|
|
cranelift-interpreter = { path = "../cranelift/interpreter" }
|
|
cranelift-fuzzgen = { path = "../cranelift/fuzzgen" }
|
|
libfuzzer-sys = "0.4.0"
|
|
target-lexicon = "0.12"
|
|
wasmtime = { path = "../crates/wasmtime" }
|
|
wasmtime-fuzzing = { path = "../crates/fuzzing" }
|
|
component-test-util = { path = "../crates/misc/component-test-util" }
|
|
component-fuzz-util = { path = "../crates/misc/component-fuzz-util" }
|
|
|
|
[build-dependencies]
|
|
anyhow = "1.0.19"
|
|
proc-macro2 = "1.0"
|
|
arbitrary = { version = "1.1.0", features = ["derive"] }
|
|
rand = { version = "0.8.0" }
|
|
quote = "1.0"
|
|
component-fuzz-util = { path = "../crates/misc/component-fuzz-util" }
|
|
|
|
[features]
|
|
default = ['fuzz-spec-interpreter']
|
|
fuzz-spec-interpreter = ['wasmtime-fuzzing/fuzz-spec-interpreter']
|
|
|
|
[[bin]]
|
|
name = "compile"
|
|
path = "fuzz_targets/compile.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "instantiate"
|
|
path = "fuzz_targets/instantiate.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "api_calls"
|
|
path = "fuzz_targets/api_calls.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "differential"
|
|
path = "fuzz_targets/differential.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "differential_meta"
|
|
path = "fuzz_targets/differential_meta.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
|
|
[[bin]]
|
|
name = "differential_spec"
|
|
path = "fuzz_targets/differential_spec.rs"
|
|
test = false
|
|
doc = false
|
|
required-features = ['fuzz-spec-interpreter']
|
|
|
|
[[bin]]
|
|
name = "differential_wasmi"
|
|
path = "fuzz_targets/differential_wasmi.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "differential_v8"
|
|
path = "fuzz_targets/differential_v8.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "spectests"
|
|
path = "fuzz_targets/spectests.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "table_ops"
|
|
path = "fuzz_targets/table_ops.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "stacks"
|
|
path = "fuzz_targets/stacks.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "compile-maybe-invalid"
|
|
path = "fuzz_targets/compile-maybe-invalid.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "cranelift-fuzzgen"
|
|
path = "fuzz_targets/cranelift-fuzzgen.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "instantiate-many"
|
|
path = "fuzz_targets/instantiate-many.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "component_api"
|
|
path = "fuzz_targets/component_api.rs"
|
|
test = false
|
|
doc = false
|
|
|
|
[[bin]]
|
|
name = "cranelift-icache"
|
|
path = "fuzz_targets/cranelift-icache.rs"
|
|
test = false
|
|
doc = false
|