* [fuzz] Add `Module` enum, refactor `ModuleConfig`
This change adds a way to create either a single-instruction module or a
regular (big) `wasm-smith` module. It has some slight refactorings in
preparation for the use of this new code.
* [fuzz] Add `DiffValue` for differential evaluation
In order to evaluate functions with randomly-generated values, we needed
a common way to generate these values. Using the Wasmtime `Val` type is
not great because we would like to be able to implement various traits
on the new value type, e.g., to convert `Into` and `From` boxed values
of other engines we differentially fuzz against. This new type,
`DiffValue`, gives us a common ground for all the conversions and
comparisons between the other engine types.
* [fuzz] Add interface for differential engines
In order to randomly choose an engine to fuzz against, we expect all of
the engines to meet a common interface. The traits in this commit allow
us to instantiate a module from its binary form, evaluate exported
functions, and (possibly) hash the exported items of the instance.
This change has some missing pieces, though:
- the `wasm-spec-interpreter` needs some work to be able to create
instances, evaluate a function by name, and expose exported items
- the `v8` engine is not implemented yet due to the complexity of its
Rust lifetimes
* [fuzz] Use `ModuleFeatures` instead of existing configuration
When attempting to use both wasm-smith and single-instruction modules,
there is a mismatch in how we communicate what an engine must be able to
support. In the first case, we could use the `ModuleConfig`, a wrapper
for wasm-smith's `SwarmConfig`, but single-instruction modules do not
have a `SwarmConfig`--the many options simply don't apply. Here, we
instead add `ModuleFeatures` and adapt a `ModuleConfig` to that.
`ModuleFeatures` then becomes the way to communicate what features an
engine must support to evaluate functions in a module.
* [fuzz] Add a new fuzz target using the meta-differential oracle
This change adds the `differential_meta` target to the list of fuzz
targets. I expect that sometime soon this could replace the other
`differential*` targets, as it almost checks all the things those check.
The major missing piece is that currently it only chooses
single-instruction modules instead of also generating arbitrary modules
using `wasm-smith`.
Also, this change adds the concept of an ignorable error: some
differential engines will choke with certain inputs (e.g., `wasmi` might
have an old opcode mapping) which we do not want to flag as fuzz bugs.
Here we wrap those errors in `DiffIgnoreError` and then use a new helper
trait, `DiffIgnorable`, to downcast and inspect the `anyhow` error to
only panic on non-ignorable errors; the ignorable errors are converted
to one of the `arbitrary::Error` variants, which we already ignore.
* [fuzz] Compare `DiffValue` NaNs more leniently
Because arithmetic NaNs can contain arbitrary payload bits, checking
that two differential executions should produce the same result should
relax the comparison of the `F32` and `F64` types (and eventually `V128`
as well... TODO). This change adds several considerations, however, so
that in the future we make the comparison a bit stricter, e.g., re:
canonical NaNs. This change, however, just matches the current logic
used by other fuzz targets.
* review: allow hashing mutate the instance state
@alexcrichton requested that the interface be adapted to accommodate
Wasmtime's API, in which even reading from an instance could trigger
mutation of the store.
* review: refactor where configurations are made compatible
See @alexcrichton's
[suggestion](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928974376).
* review: convert `DiffValueType` using `TryFrom`
See @alexcrichton's
[comment](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928962394).
* review: adapt target implementation to Wasmtime-specific RHS
This change is joint work with @alexcrichton to adapt the structure of
the fuzz target to his comments
[here](https://github.com/bytecodealliance/wasmtime/pull/4515#pullrequestreview-1073247791).
This change:
- removes `ModuleFeatures` and the `Module` enum (for big and small
modules)
- upgrades `SingleInstModule` to filter out cases that are not valid for
a given `ModuleConfig`
- adds `DiffEngine::name()`
- constructs each `DiffEngine` using a `ModuleConfig`, eliminating
`DiffIgnoreError` completely
- prints an execution rate to the `differential_meta` target
Still TODO:
- `get_exported_function_signatures` could be re-written in terms of the
Wasmtime API instead `wasmparser`
- the fuzzer crashes eventually, we think due to the signal handler
interference between OCaml and Wasmtime
- the spec interpreter has several cases that we skip for now but could
be fuzzed with further work
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* fix: avoid SIGSEGV by explicitly initializing OCaml runtime first
* review: use Wasmtime's API to retrieve exported functions
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
In #4671, the meta-differential fuzz target was finding errors when
running certain Wasm modules (specifically `shr_s` in that case).
@conrad-watt diagnosed the issue as a missing reversal in the operands
passed to the spec interpreter. This change fixes#4671 and adds an
additional unit test to keep it fixed.
This commit builds on bytecodealliance/wasm-tools#690 to add support to
testing of the component model to execute functions when running
`*.wast` files. This support is all built on #4442 as functions are
invoked through a "dynamic" API. Right now the testing and integration
is fairly crude but I'm hoping that we can try to improve it over time
as necessary. For now this should provide a hopefully more convenient
syntax for unit tests and the like.
* Bump versions of wasm-tools crates
Note that this leaves new features in the component model, outer type
aliases for core wasm types, unimplemented for now.
* Move to crates.io-based versions of tools
* Update tracing-core to a version which doesn't depend on lazy-static.
* Update crossbeam-utils to a version that doesn't depend on lazy-static.
* Update crossbeam-epoch to a version that doesn't depend on lazy-static.
* Update clap to a version that doesn't depend on lazy-static.
* Convert Wasmtime's own use of lazy_static to once_cell.
* Make `GDB_REGISTRATION`'s comment a doc comment.
* Fix compilation on Windows.
The documentation for the `wasm-spec-interpreter` was not up-to-date,
causing some confusion on non-Ubuntu machines. This change adds the
correct dependencies to install and includes the `libgmp` path for
Fedora by default (i.e., `/lib64`).
* Upgrade all crates to the Rust 2021 edition
I've personally started using the new format strings for things like
`panic!("some message {foo}")` or similar and have been upgrading crates
on a case-by-case basis, but I think it probably makes more sense to go
ahead and blanket upgrade everything so 2021 features are always
available.
* Fix compile of the C API
* Fix a warning
* Fix another warning
This seems to have intended to allow overrides but the specific Makefile
syntax used didn't actually allow overrides, so update that to allow env
vars from the outside world to override the variable (needed locally on
AArch64 I'm building on which has a different path to libgmp)
This commit updates the build script which clones the spec interpreter
for fuzzing to specifically pin at a hardcoded revision. This is
intended at improving reproducibility if we hit any issues while fuzzing
to ensure that the same wasmtime revision is always using the same spec
interpreter revision.
This commit removes the `differential_spec` fuzz target for now,
although this removal is intended to be temporary. We have #3251 to
track re-enabling the spec interpreter in a way that it won't time out,
and additionally the spec interpreter is also failing to build with
ocaml on oss-fuzz so that will also need to be investigated when
re-enabling.
In #3186, we found an issue that requires patching the spec interpreter
for now. Our plan is to have a `fuzzing` branch in our spec-repo mirror
that lets us make these fixes locally before they are upstreamed.
This PR updates the build script for the spec-interpreter wrapper
crate to clone this particular `fuzzing` branch instead of the main
branch.
The WebAssembly spec interpreter is written in OCaml and the new crate
uses `ocaml-interop` along with a small OCaml wrapper to interpret Wasm
modules in-process. The build process for this crate is currently
Linux-specific: it requires several OCaml packages (e.g. `apt install -y
ocaml-nox ocamlbuild`) as well as `make`, `cp`, and `ar`.
