Commit Graph

71 Commits

Author SHA1 Message Date
Andrew Brown
651a321f1a fuzz: add differential_spec fuzzing target
This new target compares the outputs of executing the first exported
function of a Wasm module in Wasmtime and in the official Wasm spec
interpreter (using the `wasm-spec-interpreter` crate). This is an
initial step towards more fully-featured fuzzing (e.g. compare memories,
add `v128`, add references, add other proposals, etc.)
2021-08-10 11:56:07 -07:00
Andrew Brown
f3955fa62a refactor: rename DifferentialWasmiModuleConfig to SingleFunctionModuleConfig
Since we plan to reuse this configuration, we rename it and ensure it
has at least 1 type (this resulted in invalid modules).
2021-08-10 11:56:07 -07:00
Alex Crichton
480dff21e8 fuzz: Disable more features for spectests fuzzer (#3159)
The previous commit to eanble multi-memory and simd leaked into the
spectest fuzzer, but to pass the spec tests we can't enable these features.
2021-08-06 16:27:42 -05:00
Nick Fitzgerald
e5ef1455a3 Merge pull request #3157 from alexcrichton/centralize-error-handling
fuzz: Centralize handling instantiation errors
2021-08-06 10:38:48 -07:00
Alex Crichton
45896e0533 Decrease memory limit in fuzzing to 1gb (#3155)
This should keep us under the default 2gb limit when fuzzing
2021-08-06 12:28:49 -05:00
Alex Crichton
3bdf6c7a48 fuzz: Centralize handling instantiation errors
At the same time remove some string matching in favor of checking for
oom explicitly.
2021-08-06 07:47:28 -07:00
Alex Crichton
bb85366a3b Enable simd fuzzing on oss-fuzz (#3152)
* Enable simd fuzzing on oss-fuzz

This commit generally enables the simd feature while fuzzing, which
should affect almost all fuzzers. For fuzzers that just throw random
data at the wall and see what sticks, this means that they'll now be
able to throw simd-shaped data at the wall and have it stick. For
wasm-smith-based fuzzers this commit also updates wasm-smith to 0.6.0
which allows further configuring the `SwarmConfig` after generation,
notably allowing `instantiate-swarm` to generate modules using simd
using `wasm-smith`. This should much more reliably feed simd-related
things into the fuzzers.

Finally, this commit updates wasmtime to avoid usage of the general
`wasm_smith::Module` generator to instead use a Wasmtime-specific custom
default configuration which enables various features we have
implemented.

* Allow dummy table creation to fail

Tables might creation for imports may exceed the memory limit on the
store, which we'll want to gracefully recover from and not fail the
fuzzers.
2021-08-05 16:24:42 -05:00
Alex Crichton
214c5f862d fuzz: Implement finer memory limits per-store (#3149)
* fuzz: Implement finer memory limits per-store

This commit implements a custom resource limiter for fuzzing. Locally I
was seeing a lot of ooms while fuzzing and I believe it was generally
caused from not actually having any runtime limits for wasm modules. I'm
actually surprised that this hasn't come up more on oss-fuzz more in
reality, but with a custom store limiter I think this'll get the job
done where we have an easier knob to turn for controlling the memory
usage of fuzz-generated modules.

For now I figure a 2gb limit should be good enough for limiting fuzzer
execution. Additionally the "out of resources" check if instantiation
fails now looks for the `oom` flag to be set instead of pattern matching
on some error messages about resources.

* Fix tests
2021-08-05 15:07:33 -05:00
Alex Crichton
ff1ae6e10c Flag another error as ok to hit when fuzzing (#3092)
We've got a lot of fuzz failures right now of modules instantiating
memories of 65536 pages, which we specifically disallow since the
representation of limits within Wasmtime don't support full 4GB
memories. This is ok, however, and it's not a fuzz failure that we're
interested in, so this commit allows strings of that error to pass
through the fuzzer.
2021-07-16 14:37:27 -05:00
Alex Crichton
1047c4e156 Fix fuzzers requesting 4gb memories (#3029)
Wasmtime was updated to reject creation of memories exactly 4gb in size
in #3013, but the fuzzers still had the assumption that any request to
create a host object for a particular wasm type would succeed.
Unfortunately now, though, a request to create a 4gb memory fails. This
is an expected failure, though, so the fix here was to catch the error
and allow it.
2021-06-24 14:53:18 -05:00
Pat Hickey
8b4bdf92e2 make ResourceLimiter operate on Store data; add hooks for entering and exiting native code (#2952)
* wasmtime_runtime: move ResourceLimiter defaults into this crate

In preparation of changing wasmtime::ResourceLimiter to be a re-export
of this definition, because translating between two traits was causing
problems elsewhere.

* wasmtime: make ResourceLimiter a re-export of wasmtime_runtime::ResourceLimiter

* refactor Store internals to support ResourceLimiter as part of store's data

* add hooks for entering and exiting native code to Store

* wasmtime-wast, fuzz: changes to adapt ResourceLimiter API

* fix tests

* wrap calls into wasm with entering/exiting exit hooks as well

* the most trivial test found a bug, lets write some more

* store: mark some methods as #[inline] on Store, StoreInner, StoreInnerMost

Co-authored-By: Alex Crichton <alex@alexcrichton.com>

* improve tests for the entering/exiting native hooks

Co-authored-by: Alex Crichton <alex@alexcrichton.com>
2021-06-08 09:37:00 -05:00
Alex Crichton
7a1b7cdf92 Implement RFC 11: Redesigning Wasmtime's APIs (#2897)
Implement Wasmtime's new API as designed by RFC 11. This is quite a large commit which has had lots of discussion externally, so for more information it's best to read the RFC thread and the PR thread.
2021-06-03 09:10:53 -05:00
Peter Huene
f12b4c467c Add resource limiting to the Wasmtime API. (#2736)
* Add resource limiting to the Wasmtime API.

This commit adds a `ResourceLimiter` trait to the Wasmtime API.

When used in conjunction with `Store::new_with_limiter`, this can be used to
monitor and prevent WebAssembly code from growing linear memories and tables.

This is particularly useful when hosts need to take into account host resource
usage to determine if WebAssembly code can consume more resources.

A simple `StaticResourceLimiter` is also included with these changes that will
simply limit the size of linear memories or tables for all instances created in
the store based on static values.

* Code review feedback.

* Implemented `StoreLimits` and `StoreLimitsBuilder`.
* Moved `max_instances`, `max_memories`, `max_tables` out of `Config` and into
  `StoreLimits`.
* Moved storage of the limiter in the runtime into `Memory` and `Table`.
* Made `InstanceAllocationRequest` use a reference to the limiter.
* Updated docs.
* Made `ResourceLimiterProxy` generic to remove a level of indirection.
* Fixed the limiter not being used for `wasmtime::Memory` and
  `wasmtime::Table`.

* Code review feedback and bug fix.

* `Memory::new` now returns `Result<Self>` so that an error can be returned if
  the initial requested memory exceeds any limits placed on the store.

* Changed an `Arc` to `Rc` as the `Arc` wasn't necessary.

* Removed `Store` from the `ResourceLimiter` callbacks. Custom resource limiter
  implementations are free to capture any context they want, so no need to
  unnecessarily store a weak reference to `Store` from the proxy type.

* Fixed a bug in the pooling instance allocator where an instance would be
  leaked from the pool. Previously, this would only have happened if the OS was
  unable to make the necessary linear memory available for the instance. With
  these changes, however, the instance might not be created due to limits
  placed on the store. We now properly deallocate the instance on error.

* Added more tests, including one that covers the fix mentioned above.

* Code review feedback.

* Add another memory to `test_pooling_allocator_initial_limits_exceeded` to
  ensure a partially created instance is successfully deallocated.
* Update some doc comments for better documentation of `Store` and
  `ResourceLimiter`.
2021-04-19 09:19:20 -05:00
Alex Crichton
6748dfa43a Disable module linking in spectest fuzzing
Module linking implicitly enables multiple tables but that isn't
supported by the spec tests today.
2021-03-31 07:20:22 -07:00
Alex Crichton
550c774c1d fuzz: Allow incompatible import types in instantiation (#2778)
Yesterday fuzzing was switched to using a `Linker` to improve coverage
when using module linking since we can fake instance imports with
definitions of each individual item. Using a `Linker`, however, means
that we can't necessarily instantiate all modules, such as

    (module
      (import "" "" (memory (;0;) 0 1))
      (import "" "" (memory (;1;) 2)))

As a result this just allows these sorts of "incompatible import type"
errors when fuzzing to not trigger crashes.
2021-03-26 14:38:34 -05:00
Alex Crichton
516a97b3f3 A few more small fuzzing fixes (#2770)
* Increase allowances for values when fuzzing

The wasm-smith limits for generating modules are a bit higher than what
we specify, so sync those up to avoid getting too many false positives
with limits getting blown.

* Ensure fuzzing `*.wat` files are in sync

I keep looking at `*.wat` files that are actually stale, so remove stale
files if we write out a `*.wasm` file and can't disassemble it.

* Enable shadowing in dummy_linker

Fixes an issues where the same name is imported twice and we generated
two values for that. We don't mind the error here, we just want to
ignore the shadowing errors.
2021-03-25 18:44:04 -05:00
Alex Crichton
81c44033d6 Disable module-linking in differential fuzzing (#2769)
Currently this exposes a bug where modules broken by module linking
cause failures in the fuzzer, but we want to fuzz those modules since
module linking isn't enabled when generating these modules.
2021-03-25 13:56:34 -05:00
Alex Crichton
9476581ae6 Instantiate fewer instances when fuzzing (#2768)
This commit fixes an issue where when module linking was enabled for
fuzzing (which it is) import types of modules show as imports of
instances. In an attempt to satisfy the dummy values of such imports the
fuzzing integration would create instances for each import. This would,
however, count towards instance limits and isn't always desired.

This commit refactors the creation of dummy import values to decompose
imports of instances into imports of each individual item. This should
retain the pre-module-linking behavior of dummy imports for various
fuzzers.
2021-03-25 13:35:38 -05:00
Peter Huene
54c07d8f16 Implement shared host functions. (#2625)
* Implement defining host functions at the Config level.

This commit introduces defining host functions at the `Config` rather than with
`Func` tied to a `Store`.

The intention here is to enable a host to define all of the functions once
with a `Config` and then use a `Linker` (or directly with
`Store::get_host_func`) to use the functions when instantiating a module.

This should help improve the performance of use cases where a `Store` is
short-lived and redefining the functions at every module instantiation is a
noticeable performance hit.

This commit adds `add_to_config` to the code generation for Wasmtime's `Wasi`
type.

The new method adds the WASI functions to the given config as host functions.

This commit adds context functions to `Store`: `get` to get a context of a
particular type and `set` to set the context on the store.

For safety, `set` cannot replace an existing context value of the same type.

`Wasi::set_context` was added to set the WASI context for a `Store` when using
`Wasi::add_to_config`.

* Add `Config::define_host_func_async`.

* Make config "async" rather than store.

This commit moves the concept of "async-ness" to `Config` rather than `Store`.

Note: this is a breaking API change for anyone that's already adopted the new
async support in Wasmtime.

Now `Config::new_async` is used to create an "async" config and any `Store`
associated with that config is inherently "async".

This is needed for async shared host functions to have some sanity check during their
execution (async host functions, like "async" `Func`, need to be called with
the "async" variants).

* Update async function tests to smoke async shared host functions.

This commit updates the async function tests to also smoke the shared host
functions, plus `Func::wrap0_async`.

This also changes the "wrap async" method names on `Config` to
`wrap$N_host_func_async` to slightly better match what is on `Func`.

* Move the instance allocator into `Engine`.

This commit moves the instantiated instance allocator from `Config` into
`Engine`.

This makes certain settings in `Config` no longer order-dependent, which is how
`Config` should ideally be.

This also removes the confusing concept of the "default" instance allocator,
instead opting to construct the on-demand instance allocator when needed.

This does alter the semantics of the instance allocator as now each `Engine`
gets its own instance allocator rather than sharing a single one between all
engines created from a configuration.

* Make `Engine::new` return `Result`.

This is a breaking API change for anyone using `Engine::new`.

As creating the pooling instance allocator may fail (likely cause is not enough
memory for the provided limits), instead of panicking when creating an
`Engine`, `Engine::new` now returns a `Result`.

* Remove `Config::new_async`.

This commit removes `Config::new_async` in favor of treating "async support" as
any other setting on `Config`.

The setting is `Config::async_support`.

* Remove order dependency when defining async host functions in `Config`.

This commit removes the order dependency where async support must be enabled on
the `Config` prior to defining async host functions.

The check is now delayed to when an `Engine` is created from the config.

* Update WASI example to use shared `Wasi::add_to_config`.

This commit updates the WASI example to use `Wasi::add_to_config`.

As only a single store and instance are used in the example, it has no semantic
difference from the previous example, but the intention is to steer users
towards defining WASI on the config and only using `Wasi::add_to_linker` when
more explicit scoping of the WASI context is required.
2021-03-11 10:14:03 -06:00
Alex Crichton
de27fbe20f Tweak C API for fuel
* Ensure `store` is in the function names
* Don't abort the process on `add_fuel` when fuel isn't configured
* Allow learning about failure in both `add_fuel` and `fuel_consumed`
2021-02-09 07:19:22 -08:00
Alex Crichton
0e41861662 Implement limiting WebAssembly execution with fuel (#2611)
* Consume fuel during function execution

This commit adds codegen infrastructure necessary to instrument wasm
code to consume fuel as it executes. Currently nothing is really done
with the fuel, but that'll come in later commits.

The focus of this commit is to implement the codegen infrastructure
necessary to consume fuel and account for fuel consumed correctly.

* Periodically check remaining fuel in wasm JIT code

This commit enables wasm code to periodically check to see if fuel has
run out. When fuel runs out an intrinsic is called which can do what it
needs to do in the result of fuel running out. For now a trap is thrown
to have at least some semantics in synchronous stores, but another
planned use for this feature is for asynchronous stores to periodically
yield back to the host based on fuel running out.

Checks for remaining fuel happen in the same locations as interrupt
checks, which is to say the start of the function as well as loop
headers.

* Improve codegen by caching `*const VMInterrupts`

The location of the shared interrupt value and fuel value is through a
double-indirection on the vmctx (load through the vmctx and then load
through that pointer). The second pointer in this chain, however, never
changes, so we can alter codegen to account for this and remove some
extraneous load instructions and hopefully reduce some register
pressure even maybe.

* Add tests fuel can abort infinite loops

* More fuzzing with fuel

Use fuel to time out modules in addition to time, using fuzz input to
figure out which.

* Update docs on trapping instructions

* Fix doc links

* Fix a fuzz test

* Change setting fuel to adding fuel

* Fix a doc link

* Squelch some rustdoc warnings
2021-01-29 08:57:17 -06:00
Nick Fitzgerald
be1ac15beb Merge pull request #2617 from alexcrichton/limit-tables-and-memeories
Add knobs to limit memories/tables in a `Store`
2021-01-28 10:20:33 -08:00
Alex Crichton
dccaa64962 Add knobs to limit memories/tables in a Store
Fuzzing has turned up that module linking can create large amounts of
tables and memories in addition to instances. For example if N instances
are allowed and M tables are allowed per-instance, then currently
wasmtime allows MxN tables (which is quite a lot). This is causing some
wasm-smith-generated modules to exceed resource limits while fuzzing!

This commits adds corresponding `max_tables` and `max_memories`
functions to sit alongside the `max_instances` configuration.
Additionally fuzzing now by default configures all of these to a
somewhat low value to avoid too much resource usage while fuzzing.
2021-01-28 08:47:00 -08:00
Alex Crichton
cb65c755c5 Disable module-linking in plain instantiate fuzzers
We already cover module linking with the `instantiate-swarm` target and
otherwise enabling module linking is preventing otherwise-valid modules
from being compiled because of the breaking change in the module linking
proposal with respect to imports.
2021-01-28 07:55:04 -08:00
David Haynes
02260b7cd0 2499: First pass on TableOps fuzzer generator wasm_encoder migration (#2501)
* 2499: First pass on TableOps fuzzer generator wasm_encoder migration

- wasm binary generated via sections and smushed together into a module
- test: compare generated wat against expected wat
- note: doesn't work
  - Grouped instructions not implemented
  - Vec<u8> to wat String not implemented

* 2499: Add typesection, abstract instruction puts, and update test

- TableOp.insert now will interact with a function object directly
- add types for generated function
- expected test string now reflects expected generated code

* 2499: Mark unused index as _i

* 2499: Function insertion is in proper stack order, and fix off by 1
      index

- imported functions must be typed
- instructions operate on a stack ie. define values as instructions
  before using

* 2499: Apply suggestions from code review

- typo fixing
- oracle ingests binary bytes itself

Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>

* 2499: Code cleanup + renaming vars

- busywork, nothing to see here

Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
2020-12-17 15:47:18 -06:00
Alex Crichton
2d7c2fba5f Update wasm-smith (#2509)
Brings in a few bug fixes for generating module-linking modules
2020-12-14 13:39:38 -06:00
Alex Crichton
42adeba65d Fix fuzzer expectation about valid modules
Recent changes to fuzzers made expectations more strict about handling
errors while fuzzing, but this erroneously changed a module compilation
step to always assume that the input wasm is valid. Instead a flag is
now passed through indicating whether the wasm blob is known valid or
invalid, and only if compilation fails and it's known valid do we panic.
2020-12-14 08:31:46 -08:00
Alex Crichton
25000afe69 Enable fuzzing the module linking implementation
This commit updates all the wasm-tools crates that we use and enables
fuzzing of the module linking proposal in our various fuzz targets. This
also refactors some of the dummy value generation logic to not be
fallible and to always succeed, the thinking being that we don't want to
accidentally hide errors while fuzzing. Additionally instantiation is
only allowed to fail with a `Trap`, other failure reasons are unwrapped.
2020-12-11 08:36:52 -08:00
Alex Crichton
e09b9400f8 Restrict threads spawned during fuzzing (#2485)
I was having limited success fuzzing locally because apparently the
fuzzer was spawning too many threads. Looking into it that indeed
appears to be the case! The threads which time out runtime of wasm only
exit after the sleep has completely finished, meaning that if we execute
a ton of wasm that exits quickly each run will generate a sleeping thread.

This commit fixes the issue by using some synchronization to ensure the
sleeping thread exits when our fuzzed run also exits.
2020-12-09 10:34:37 -06:00
Chris Fallin
bbdea06e2d Add differential fuzzing against wasmi (a Wasm interpreter).
This PR adds a new fuzz target, `differential_wasmi`, that runs a
Cranelift-based Wasm backend alongside a simple third-party Wasm
interpeter crate (`wasmi`).  The fuzzing runs the first function in a
given module to completion on each side, and then diffs the return value
and linear memory contents.

This strategy should provide end-to-end coverage including both the Wasm
translation to CLIF (which has seen some subtle and scary bugs at
times), the lowering from CLIF to VCode, the register allocation, and
the final code emission.

This PR also adds a feature `experimental_x64` to the fuzzing crate (and
the chain of dependencies down to `cranelift-codegen`) so that we can
fuzz the new x86-64 backend as well as the current one.
2020-12-02 14:52:44 -08:00
Alex Crichton
35327ed4d7 Print a message in log_wat while fuzzing
Similar to `log_wasm`, just indicates that a file was written.
2020-11-02 06:52:01 -08:00
Alex Crichton
b73b831892 Replace binaryen -ttf based fuzzing with wasm-smith (#2336)
This commit removes the binaryen support for fuzzing from wasmtime,
instead switching over to `wasm-smith`. In general it's great to have
what fuzzing we can, but our binaryen support suffers from a few issues:

* The Rust crate, binaryen-sys, seems largely unmaintained at this
  point. While we could likely take ownership and/or send PRs to update
  the crate it seems like the maintenance is largely on us at this point.

* Currently the binaryen-sys crate doesn't support fuzzing anything
  beyond MVP wasm, but we're interested at least in features like bulk
  memory and reference types. Additionally we'll also be interested in
  features like module-linking. New features would require either
  implementation work in binaryen or the binaryen-sys crate to support.

* We have 4-5 fuzz-bugs right now related to timeouts simply in
  generating a module for wasmtime to fuzz. One investigation along
  these lines in the past revealed a bug in binaryen itself, and in any
  case these bugs would otherwise need to get investigated, reported,
  and possibly fixed ourselves in upstream binaryen.

Overall I'm not sure at this point if maintaining binaryen fuzzing is
worth it with the advent of `wasm-smith` which has similar goals for
wasm module generation, but is much more readily maintainable on our
end.

Additonally in this commit I've added a fuzzer for wasm-smith's
`SwarmConfig`-based fuzzer which should expand the coverage of tested
modules.

Closes #2163
2020-10-29 10:02:59 -05:00
Alex Crichton
cdf158cd50 Fix enabling interrupts in fuzzers
Modifying the `Config` after the gneine has been created has no effect,
so be sure to flag the configuration as interruptible earlier.
2020-10-09 10:09:27 -07:00
Alex Crichton
38428e1fbb Expand modules instantiated in instantiate-wasm-smith
This commit uses the new `MaybeInvalidModule` type in `wasm-smith` to
try to explore more points in the fuzz target space in the
`instantiate-maybe-invalid` fuzz target. The goal here is to use the raw
fuzz input as the body of a function to stress the validator/decoder a
bit more, and try to get inputs we might not otherwise generate.
2020-09-08 14:29:27 -07:00
Alex Crichton
5893317a8e Fix execution of spectest fuzzer
We've enabled bulk memory and reference types by default now which means
that wasmtime in its default settings no longer passes the spec test
suite (due to changes in error messages in initialization), so when
we're running the spec test fuzzer be sure to disable reference types
and bulk memory since that's required to pass.
2020-08-10 06:55:36 -07:00
Nick Fitzgerald
98e899f6b3 fuzz: Add a fuzz target for table.{get,set} operations
This new fuzz target exercises sequences of `table.get`s, `table.set`s, and
GCs.

It already found a couple bugs:

* Some leaks due to ref count cycles between stores and host-defined functions
  closing over those stores.

* If there are no live references for a PC, Cranelift can avoid emiting an
  associated stack map. This was running afoul of a debug assertion.
2020-06-30 12:00:57 -07:00
Alex Crichton
5fa4d36b0d Disable Cranelift debug verifier when fuzzing (#1851)
* Add CLI flags for internal cranelift options

This commit adds two flags to the `wasmtime` CLI:

* `--enable-cranelift-debug-verifier`
* `--enable-cranelift-nan-canonicalization`

These previously weren't exposed from the command line but have been
useful to me at least for reproducing slowdowns found during fuzzing on
the CLI.

* Disable Cranelift debug verifier when fuzzing

This commit disables Cranelift's debug verifier for our fuzz targets.
We've gotten a good number of timeouts on OSS-Fuzz and some I've
recently had some discussion over at google/oss-fuzz#3944 about this
issue and what we can do. The result of that discussion was that there
are two primary ways we can speed up our fuzzers:

* One is independent of Wasmtime, which is to tweak the flags used to
  compile code. The conclusion was that one flag was passed to LLVM
  which significantly increased runtime for very little benefit. This
  has now been disabled in rust-fuzz/cargo-fuzz#229.

* The other way is to reduce the amount of debug checks we run while
  fuzzing wasmtime itself. To put this in perspective, a test case which
  took ~100ms to instantiate was taking 50 *seconds* to instantiate in
  the fuzz target. This 500x slowdown was caused by a ton of
  multiplicative factors, but two major contributors were NaN
  canonicalization and cranelift's debug verifier. I suspect the NaN
  canonicalization itself isn't too pricy but when paired with the debug
  verifier in float-heavy code it can create lots of IR to verify.

This commit is specifically tackling this second point in an attempt to
avoid slowing down our fuzzers too much. The intent here is that we'll
disable the cranelift debug verifier for now but leave all other checks
enabled. If the debug verifier gets a speed boost we can try re-enabling
it, but otherwise it seems like for now it's otherwise not catching any
bugs and creating lots of noise about timeouts that aren't relevant.

It's not great that we have to turn off internal checks since that's
what fuzzing is supposed to trigger, but given the timeout on OSS-Fuzz
and the multiplicative effects of all the slowdowns we have when
fuzzing, I'm not sure we can afford the massive slowdown of the debug verifier.
2020-06-10 12:50:21 -05:00
Yury Delendik
15c68f2cc1 Disconnects Store state fields from Compiler (#1761)
*  Moves CodeMemory, VMInterrupts and SignatureRegistry from Compiler
*  CompiledModule holds CodeMemory and GdbJitImageRegistration
*  Store keeps track of its JIT code
*  Makes "jit_int.rs" stuff Send+Sync
*  Adds the threads example.
2020-06-02 13:44:39 -05:00
Nick Fitzgerald
f28b3738ee Rename anyref to externref across the board 2020-05-20 11:58:55 -07:00
Alex Crichton
d88a147b4d Add some debugging assistance to spectest oracle
Basically just log what's happening so if you're running the fuzzer you
can see what test is being run.
2020-04-30 12:25:06 -07:00
Alex Crichton
c9a0ba81a0 Implement interrupting wasm code, reimplement stack overflow (#1490)
* Implement interrupting wasm code, reimplement stack overflow

This commit is a relatively large change for wasmtime with two main
goals:

* Primarily this enables interrupting executing wasm code with a trap,
  preventing infinite loops in wasm code. Note that resumption of the
  wasm code is not a goal of this commit.

* Additionally this commit reimplements how we handle stack overflow to
  ensure that host functions always have a reasonable amount of stack to
  run on. This fixes an issue where we might longjmp out of a host
  function, skipping destructors.

Lots of various odds and ends end up falling out in this commit once the
two goals above were implemented. The strategy for implementing this was
also lifted from Spidermonkey and existing functionality inside of
Cranelift. I've tried to write up thorough documentation of how this all
works in `crates/environ/src/cranelift.rs` where gnarly-ish bits are.

A brief summary of how this works is that each function and each loop
header now checks to see if they're interrupted. Interrupts and the
stack overflow check are actually folded into one now, where function
headers check to see if they've run out of stack and the sentinel value
used to indicate an interrupt, checked in loop headers, tricks functions
into thinking they're out of stack. An interrupt is basically just
writing a value to a location which is read by JIT code.

When interrupts are delivered and what triggers them has been left up to
embedders of the `wasmtime` crate. The `wasmtime::Store` type has a
method to acquire an `InterruptHandle`, where `InterruptHandle` is a
`Send` and `Sync` type which can travel to other threads (or perhaps
even a signal handler) to get notified from. It's intended that this
provides a good degree of flexibility when interrupting wasm code. Note
though that this does have a large caveat where interrupts don't work
when you're interrupting host code, so if you've got a host import
blocking for a long time an interrupt won't actually be received until
the wasm starts running again.

Some fallout included from this change is:

* Unix signal handlers are no longer registered with `SA_ONSTACK`.
  Instead they run on the native stack the thread was already using.
  This is possible since stack overflow isn't handled by hitting the
  guard page, but rather it's explicitly checked for in wasm now. Native
  stack overflow will continue to abort the process as usual.

* Unix sigaltstack management is now no longer necessary since we don't
  use it any more.

* Windows no longer has any need to reset guard pages since we no longer
  try to recover from faults on guard pages.

* On all targets probestack intrinsics are disabled since we use a
  different mechanism for catching stack overflow.

* The C API has been updated with interrupts handles. An example has
  also been added which shows off how to interrupt a module.

Closes #139
Closes #860
Closes #900

* Update comment about magical interrupt value

* Store stack limit as a global value, not a closure

* Run rustfmt

* Handle review comments

* Add a comment about SA_ONSTACK

* Use `usize` for type of `INTERRUPTED`

* Parse human-readable durations

* Bring back sigaltstack handling

Allows libstd to print out stack overflow on failure still.

* Add parsing and emission of stack limit-via-preamble

* Fix new example for new apis

* Fix host segfault test in release mode

* Fix new doc example
2020-04-21 11:03:28 -07:00
Dan Gohman
9364eb1d98 Refactor (#1524)
* Compute instance exports on demand.

Instead having instances eagerly compute a Vec of Externs, and bumping
the refcount for each Extern, compute Externs on demand.

This also enables `Instance::get_export` to avoid doing a linear search.

This also means that the closure returned by `get0` and friends now
holds an `InstanceHandle` to dynamically hold the instance live rather
than being scoped to a lifetime.

* Compute module imports and exports on demand too.

And compute Extern::ty on demand too.

* Add a utility function for computing an ExternType.

* Add a utility function for looking up a function's signature.

* Add a utility function for computing the ValType of a Global.

* Rename wasmtime_environ::Export to EntityIndex.

This helps differentiate it from other Export types in the tree, and
describes what it is.

* Fix a typo in a comment.

* Simplify module imports and exports.

* Make `Instance::exports` return the export names.

This significantly simplifies the public API, as it's relatively common
to need the names, and this avoids the need to do a zip with
`Module::exports`.

This also changes `ImportType` and `ExportType` to have public members
instead of private members and accessors, as I find that simplifies the
usage particularly in cases where there are temporary instances.

* Remove `Instance::module`.

This doesn't quite remove `Instance`'s `module` member, it gets a step
closer.

* Use a InstanceHandle utility function.

* Don't consume self in the `Func::get*` methods.

Instead, just create a closure containing the instance handle and the
export for them to call.

* Use `ExactSizeIterator` to avoid needing separate `num_*` methods.

* Rename `Extern::func()` etc. to `into_func()` etc.

* Revise examples to avoid using `nth`.

* Add convenience methods to instance for getting specific extern types.

* Use the convenience functions in more tests and examples.

* Avoid cloning strings for `ImportType` and `ExportType`.

* Remove more obviated clone() calls.

* Simplify `Func`'s closure state.

* Make wasmtime::Export's fields private.

This makes them more consistent with ExportType.

* Fix compilation error.

* Make a lifetime parameter explicit, and use better lifetime names.

Instead of 'me, use 'instance and 'module to make it clear what the
lifetime is.

* More lifetime cleanups.
2020-04-20 15:55:33 -05:00
Alex Crichton
6dde222992 Add a spec test fuzzer for Config (#1509)
* Add a spec test fuzzer for Config

This commit adds a new fuzzer which is intended to run on oss-fuzz. This
fuzzer creates and arbitrary `Config` which *should* pass spec tests and
then asserts that it does so. The goal here is to weed out any
accidental bugs in global configuration which could cause
non-spec-compliant behavior.

* Move implementation to `fuzzing` crate
2020-04-15 08:29:12 -05:00
Alex Crichton
bd374fd6fc Add Wasmtime-specific C API functions to return errors (#1467)
* Add Wasmtime-specific C API functions to return errors

This commit adds new `wasmtime_*` symbols to the C API, many of which
mirror the existing counterparts in the `wasm.h` header. These APIs are
enhanced in a number of respects:

* Detailed error information is now available through a
  `wasmtime_error_t`. Currently this only exposes one function which is
  to extract a string version of the error.

* There is a distinction now between traps and errors during
  instantiation and function calling. Traps only happen if wasm traps,
  and errors can happen for things like runtime type errors when
  interacting with the API.

* APIs have improved safety with respect to embedders where the lengths
  of arrays are now taken as explicit parameters rather than assumed
  from other parameters.

* Handle trap updates

* Update C examples

* Fix memory.c compile on MSVC

* Update test assertions

* Refactor C slightly

* Bare-bones .NET update

* Remove bogus nul handling
2020-04-06 15:13:06 -05:00
teapotd
2180e9ce16 fuzzing: Enable NaN canonicalization (#1334)
* Method to enable NaN canonicalization in Config

* Use fuzz_default_config in DifferentialConfig

* Enable NaN canonicalization for fuzzing
2020-03-31 09:22:08 -05:00
Alex Crichton
210bfddfa9 Fix unused imports in oracles 2020-03-17 12:02:09 -07:00
Alex Crichton
5f47068eb1 take ttf in differential 2020-03-17 09:52:17 -07:00
Alex Crichton
23bc79f66d rustfmt 2020-03-17 09:51:59 -07:00
Alex Crichton
b0cf8c021f Turn off binaryen in fuzzing by default
... but turn it back on in CI by default. The `binaryen-sys` crate
builds binaryen from source, which is a drag on CI for a few reasons:

* This is quite large and takes a good deal of time to build
* The debug build directory for binaryen is 4GB large

In an effort to both save time and disk space on the builders this
commit adds a `binaryen` feature to the `wasmtime-fuzz` crate. This
feature is enabled specifically when running the fuzzers on CI, but it
is disabled during the typical `cargo test --all` command. This means
that the test builders should save an extra 4G of space and be a bit
speedier now that they don't build a giant wad of C++.

We'll need to update the OSS-fuzz integration to enable the `binaryen`
feature when executing `cargo fuzz build`, and I'll do that once this
gets closer to landing.
2020-03-17 09:51:59 -07:00
Nick Fitzgerald
2ead747f48 Enable bulk memory in the fuzzers (#1277) 2020-03-11 08:02:19 -05:00