Commit Graph

89 Commits

Author SHA1 Message Date
Alex Crichton
7ce46043dc Add guard pages to the front of linear memories (#2977)
* Add guard pages to the front of linear memories

This commit implements a safety feature for Wasmtime to place guard
pages before the allocation of all linear memories. Guard pages placed
after linear memories are typically present for performance (at least)
because it can help elide bounds checks. Guard pages before a linear
memory, however, are never strictly needed for performance or features.
The intention of a preceding guard page is to help insulate against bugs
in Cranelift or other code generators, such as CVE-2021-32629.

This commit adds a `Config::guard_before_linear_memory` configuration
option, defaulting to `true`, which indicates whether guard pages should
be present both before linear memories as well as afterwards. Guard
regions continue to be controlled by
`{static,dynamic}_memory_guard_size` methods.

The implementation here affects both on-demand allocated memories as
well as the pooling allocator for memories. For on-demand memories this
adjusts the size of the allocation as well as adjusts the calculations
for the base pointer of the wasm memory. For the pooling allocator this
will place a singular extra guard region at the very start of the
allocation for memories. Since linear memories in the pooling allocator
are contiguous every memory already had a preceding guard region in
memory, it was just the previous memory's guard region afterwards. Only
the first memory needed this extra guard.

I've attempted to write some tests to help test all this, but this is
all somewhat tricky to test because the settings are pretty far away
from the actual behavior. I think, though, that the tests added here
should help cover various use cases and help us have confidence in
tweaking the various `Config` settings beyond their defaults.

Note that this also contains a semantic change where
`InstanceLimits::memory_reservation_size` has been removed. Instead this
field is now inferred from the `static_memory_maximum_size` and guard
size settings. This should hopefully remove some duplication in these
settings, canonicalizing on the guard-size/static-size settings as the
way to control memory sizes and virtual reservations.

* Update config docs

* Fix a typo

* Fix benchmark

* Fix wasmtime-runtime tests

* Fix some more tests

* Try to fix uffd failing test

* Review items

* Tweak 32-bit defaults

Makes the pooling allocator a bit more reasonable by default on 32-bit
with these settings.
2021-06-18 09:57:08 -05:00
Pat Hickey
8b4bdf92e2 make ResourceLimiter operate on Store data; add hooks for entering and exiting native code (#2952)
* wasmtime_runtime: move ResourceLimiter defaults into this crate

In preparation of changing wasmtime::ResourceLimiter to be a re-export
of this definition, because translating between two traits was causing
problems elsewhere.

* wasmtime: make ResourceLimiter a re-export of wasmtime_runtime::ResourceLimiter

* refactor Store internals to support ResourceLimiter as part of store's data

* add hooks for entering and exiting native code to Store

* wasmtime-wast, fuzz: changes to adapt ResourceLimiter API

* fix tests

* wrap calls into wasm with entering/exiting exit hooks as well

* the most trivial test found a bug, lets write some more

* store: mark some methods as #[inline] on Store, StoreInner, StoreInnerMost

Co-authored-By: Alex Crichton <alex@alexcrichton.com>

* improve tests for the entering/exiting native hooks

Co-authored-by: Alex Crichton <alex@alexcrichton.com>
2021-06-08 09:37:00 -05:00
Alex Crichton
7a1b7cdf92 Implement RFC 11: Redesigning Wasmtime's APIs (#2897)
Implement Wasmtime's new API as designed by RFC 11. This is quite a large commit which has had lots of discussion externally, so for more information it's best to read the RFC thread and the PR thread.
2021-06-03 09:10:53 -05:00
Peter Huene
f12b4c467c Add resource limiting to the Wasmtime API. (#2736)
* Add resource limiting to the Wasmtime API.

This commit adds a `ResourceLimiter` trait to the Wasmtime API.

When used in conjunction with `Store::new_with_limiter`, this can be used to
monitor and prevent WebAssembly code from growing linear memories and tables.

This is particularly useful when hosts need to take into account host resource
usage to determine if WebAssembly code can consume more resources.

A simple `StaticResourceLimiter` is also included with these changes that will
simply limit the size of linear memories or tables for all instances created in
the store based on static values.

* Code review feedback.

* Implemented `StoreLimits` and `StoreLimitsBuilder`.
* Moved `max_instances`, `max_memories`, `max_tables` out of `Config` and into
  `StoreLimits`.
* Moved storage of the limiter in the runtime into `Memory` and `Table`.
* Made `InstanceAllocationRequest` use a reference to the limiter.
* Updated docs.
* Made `ResourceLimiterProxy` generic to remove a level of indirection.
* Fixed the limiter not being used for `wasmtime::Memory` and
  `wasmtime::Table`.

* Code review feedback and bug fix.

* `Memory::new` now returns `Result<Self>` so that an error can be returned if
  the initial requested memory exceeds any limits placed on the store.

* Changed an `Arc` to `Rc` as the `Arc` wasn't necessary.

* Removed `Store` from the `ResourceLimiter` callbacks. Custom resource limiter
  implementations are free to capture any context they want, so no need to
  unnecessarily store a weak reference to `Store` from the proxy type.

* Fixed a bug in the pooling instance allocator where an instance would be
  leaked from the pool. Previously, this would only have happened if the OS was
  unable to make the necessary linear memory available for the instance. With
  these changes, however, the instance might not be created due to limits
  placed on the store. We now properly deallocate the instance on error.

* Added more tests, including one that covers the fix mentioned above.

* Code review feedback.

* Add another memory to `test_pooling_allocator_initial_limits_exceeded` to
  ensure a partially created instance is successfully deallocated.
* Update some doc comments for better documentation of `Store` and
  `ResourceLimiter`.
2021-04-19 09:19:20 -05:00
Alex Crichton
6748dfa43a Disable module linking in spectest fuzzing
Module linking implicitly enables multiple tables but that isn't
supported by the spec tests today.
2021-03-31 07:20:22 -07:00
Alex Crichton
550c774c1d fuzz: Allow incompatible import types in instantiation (#2778)
Yesterday fuzzing was switched to using a `Linker` to improve coverage
when using module linking since we can fake instance imports with
definitions of each individual item. Using a `Linker`, however, means
that we can't necessarily instantiate all modules, such as

    (module
      (import "" "" (memory (;0;) 0 1))
      (import "" "" (memory (;1;) 2)))

As a result this just allows these sorts of "incompatible import type"
errors when fuzzing to not trigger crashes.
2021-03-26 14:38:34 -05:00
Alex Crichton
516a97b3f3 A few more small fuzzing fixes (#2770)
* Increase allowances for values when fuzzing

The wasm-smith limits for generating modules are a bit higher than what
we specify, so sync those up to avoid getting too many false positives
with limits getting blown.

* Ensure fuzzing `*.wat` files are in sync

I keep looking at `*.wat` files that are actually stale, so remove stale
files if we write out a `*.wasm` file and can't disassemble it.

* Enable shadowing in dummy_linker

Fixes an issues where the same name is imported twice and we generated
two values for that. We don't mind the error here, we just want to
ignore the shadowing errors.
2021-03-25 18:44:04 -05:00
Alex Crichton
81c44033d6 Disable module-linking in differential fuzzing (#2769)
Currently this exposes a bug where modules broken by module linking
cause failures in the fuzzer, but we want to fuzz those modules since
module linking isn't enabled when generating these modules.
2021-03-25 13:56:34 -05:00
Alex Crichton
9476581ae6 Instantiate fewer instances when fuzzing (#2768)
This commit fixes an issue where when module linking was enabled for
fuzzing (which it is) import types of modules show as imports of
instances. In an attempt to satisfy the dummy values of such imports the
fuzzing integration would create instances for each import. This would,
however, count towards instance limits and isn't always desired.

This commit refactors the creation of dummy import values to decompose
imports of instances into imports of each individual item. This should
retain the pre-module-linking behavior of dummy imports for various
fuzzers.
2021-03-25 13:35:38 -05:00
Peter Huene
54c07d8f16 Implement shared host functions. (#2625)
* Implement defining host functions at the Config level.

This commit introduces defining host functions at the `Config` rather than with
`Func` tied to a `Store`.

The intention here is to enable a host to define all of the functions once
with a `Config` and then use a `Linker` (or directly with
`Store::get_host_func`) to use the functions when instantiating a module.

This should help improve the performance of use cases where a `Store` is
short-lived and redefining the functions at every module instantiation is a
noticeable performance hit.

This commit adds `add_to_config` to the code generation for Wasmtime's `Wasi`
type.

The new method adds the WASI functions to the given config as host functions.

This commit adds context functions to `Store`: `get` to get a context of a
particular type and `set` to set the context on the store.

For safety, `set` cannot replace an existing context value of the same type.

`Wasi::set_context` was added to set the WASI context for a `Store` when using
`Wasi::add_to_config`.

* Add `Config::define_host_func_async`.

* Make config "async" rather than store.

This commit moves the concept of "async-ness" to `Config` rather than `Store`.

Note: this is a breaking API change for anyone that's already adopted the new
async support in Wasmtime.

Now `Config::new_async` is used to create an "async" config and any `Store`
associated with that config is inherently "async".

This is needed for async shared host functions to have some sanity check during their
execution (async host functions, like "async" `Func`, need to be called with
the "async" variants).

* Update async function tests to smoke async shared host functions.

This commit updates the async function tests to also smoke the shared host
functions, plus `Func::wrap0_async`.

This also changes the "wrap async" method names on `Config` to
`wrap$N_host_func_async` to slightly better match what is on `Func`.

* Move the instance allocator into `Engine`.

This commit moves the instantiated instance allocator from `Config` into
`Engine`.

This makes certain settings in `Config` no longer order-dependent, which is how
`Config` should ideally be.

This also removes the confusing concept of the "default" instance allocator,
instead opting to construct the on-demand instance allocator when needed.

This does alter the semantics of the instance allocator as now each `Engine`
gets its own instance allocator rather than sharing a single one between all
engines created from a configuration.

* Make `Engine::new` return `Result`.

This is a breaking API change for anyone using `Engine::new`.

As creating the pooling instance allocator may fail (likely cause is not enough
memory for the provided limits), instead of panicking when creating an
`Engine`, `Engine::new` now returns a `Result`.

* Remove `Config::new_async`.

This commit removes `Config::new_async` in favor of treating "async support" as
any other setting on `Config`.

The setting is `Config::async_support`.

* Remove order dependency when defining async host functions in `Config`.

This commit removes the order dependency where async support must be enabled on
the `Config` prior to defining async host functions.

The check is now delayed to when an `Engine` is created from the config.

* Update WASI example to use shared `Wasi::add_to_config`.

This commit updates the WASI example to use `Wasi::add_to_config`.

As only a single store and instance are used in the example, it has no semantic
difference from the previous example, but the intention is to steer users
towards defining WASI on the config and only using `Wasi::add_to_linker` when
more explicit scoping of the WASI context is required.
2021-03-11 10:14:03 -06:00
Nick Fitzgerald
824ce7bf89 deps: Update Arbitrary to 1.0; libfuzzer-sys to 0.4.0; wasm-smith to 0.4.0 2021-02-25 15:34:02 -08:00
Alex Crichton
de27fbe20f Tweak C API for fuel
* Ensure `store` is in the function names
* Don't abort the process on `add_fuel` when fuel isn't configured
* Allow learning about failure in both `add_fuel` and `fuel_consumed`
2021-02-09 07:19:22 -08:00
Alex Crichton
0e41861662 Implement limiting WebAssembly execution with fuel (#2611)
* Consume fuel during function execution

This commit adds codegen infrastructure necessary to instrument wasm
code to consume fuel as it executes. Currently nothing is really done
with the fuel, but that'll come in later commits.

The focus of this commit is to implement the codegen infrastructure
necessary to consume fuel and account for fuel consumed correctly.

* Periodically check remaining fuel in wasm JIT code

This commit enables wasm code to periodically check to see if fuel has
run out. When fuel runs out an intrinsic is called which can do what it
needs to do in the result of fuel running out. For now a trap is thrown
to have at least some semantics in synchronous stores, but another
planned use for this feature is for asynchronous stores to periodically
yield back to the host based on fuel running out.

Checks for remaining fuel happen in the same locations as interrupt
checks, which is to say the start of the function as well as loop
headers.

* Improve codegen by caching `*const VMInterrupts`

The location of the shared interrupt value and fuel value is through a
double-indirection on the vmctx (load through the vmctx and then load
through that pointer). The second pointer in this chain, however, never
changes, so we can alter codegen to account for this and remove some
extraneous load instructions and hopefully reduce some register
pressure even maybe.

* Add tests fuel can abort infinite loops

* More fuzzing with fuel

Use fuel to time out modules in addition to time, using fuzz input to
figure out which.

* Update docs on trapping instructions

* Fix doc links

* Fix a fuzz test

* Change setting fuel to adding fuel

* Fix a doc link

* Squelch some rustdoc warnings
2021-01-29 08:57:17 -06:00
Nick Fitzgerald
be1ac15beb Merge pull request #2617 from alexcrichton/limit-tables-and-memeories
Add knobs to limit memories/tables in a `Store`
2021-01-28 10:20:33 -08:00
Alex Crichton
dccaa64962 Add knobs to limit memories/tables in a Store
Fuzzing has turned up that module linking can create large amounts of
tables and memories in addition to instances. For example if N instances
are allowed and M tables are allowed per-instance, then currently
wasmtime allows MxN tables (which is quite a lot). This is causing some
wasm-smith-generated modules to exceed resource limits while fuzzing!

This commits adds corresponding `max_tables` and `max_memories`
functions to sit alongside the `max_instances` configuration.
Additionally fuzzing now by default configures all of these to a
somewhat low value to avoid too much resource usage while fuzzing.
2021-01-28 08:47:00 -08:00
Alex Crichton
cb65c755c5 Disable module-linking in plain instantiate fuzzers
We already cover module linking with the `instantiate-swarm` target and
otherwise enabling module linking is preventing otherwise-valid modules
from being compiled because of the breaking change in the module linking
proposal with respect to imports.
2021-01-28 07:55:04 -08:00
Nick Fitzgerald
c7c6e76f9b fuzzing: Add tests for dummy import generation (#2604) 2021-01-26 09:11:24 -06:00
David Haynes
02260b7cd0 2499: First pass on TableOps fuzzer generator wasm_encoder migration (#2501)
* 2499: First pass on TableOps fuzzer generator wasm_encoder migration

- wasm binary generated via sections and smushed together into a module
- test: compare generated wat against expected wat
- note: doesn't work
  - Grouped instructions not implemented
  - Vec<u8> to wat String not implemented

* 2499: Add typesection, abstract instruction puts, and update test

- TableOp.insert now will interact with a function object directly
- add types for generated function
- expected test string now reflects expected generated code

* 2499: Mark unused index as _i

* 2499: Function insertion is in proper stack order, and fix off by 1
      index

- imported functions must be typed
- instructions operate on a stack ie. define values as instructions
  before using

* 2499: Apply suggestions from code review

- typo fixing
- oracle ingests binary bytes itself

Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>

* 2499: Code cleanup + renaming vars

- busywork, nothing to see here

Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
2020-12-17 15:47:18 -06:00
Alex Crichton
2d7c2fba5f Update wasm-smith (#2509)
Brings in a few bug fixes for generating module-linking modules
2020-12-14 13:39:38 -06:00
Alex Crichton
42adeba65d Fix fuzzer expectation about valid modules
Recent changes to fuzzers made expectations more strict about handling
errors while fuzzing, but this erroneously changed a module compilation
step to always assume that the input wasm is valid. Instead a flag is
now passed through indicating whether the wasm blob is known valid or
invalid, and only if compilation fails and it's known valid do we panic.
2020-12-14 08:31:46 -08:00
Alex Crichton
25000afe69 Enable fuzzing the module linking implementation
This commit updates all the wasm-tools crates that we use and enables
fuzzing of the module linking proposal in our various fuzz targets. This
also refactors some of the dummy value generation logic to not be
fallible and to always succeed, the thinking being that we don't want to
accidentally hide errors while fuzzing. Additionally instantiation is
only allowed to fail with a `Trap`, other failure reasons are unwrapped.
2020-12-11 08:36:52 -08:00
Alex Crichton
e09b9400f8 Restrict threads spawned during fuzzing (#2485)
I was having limited success fuzzing locally because apparently the
fuzzer was spawning too many threads. Looking into it that indeed
appears to be the case! The threads which time out runtime of wasm only
exit after the sleep has completely finished, meaning that if we execute
a ton of wasm that exits quickly each run will generate a sleeping thread.

This commit fixes the issue by using some synchronization to ensure the
sleeping thread exits when our fuzzed run also exits.
2020-12-09 10:34:37 -06:00
Chris Fallin
bbdea06e2d Add differential fuzzing against wasmi (a Wasm interpreter).
This PR adds a new fuzz target, `differential_wasmi`, that runs a
Cranelift-based Wasm backend alongside a simple third-party Wasm
interpeter crate (`wasmi`).  The fuzzing runs the first function in a
given module to completion on each side, and then diffs the return value
and linear memory contents.

This strategy should provide end-to-end coverage including both the Wasm
translation to CLIF (which has seen some subtle and scary bugs at
times), the lowering from CLIF to VCode, the register allocation, and
the final code emission.

This PR also adds a feature `experimental_x64` to the fuzzing crate (and
the chain of dependencies down to `cranelift-codegen`) so that we can
fuzz the new x86-64 backend as well as the current one.
2020-12-02 14:52:44 -08:00
Alex Crichton
8dd091219a Update wasm-tools dependencies
Brings in fixes for some assorted wast issues.
2020-11-09 08:50:03 -08:00
Alex Crichton
73cda83548 Propagate module-linking types to wasmtime (#2115)
This commit adds lots of plumbing to get the type section from the
module linking proposal plumbed all the way through to the `wasmtime`
crate and the `wasmtime-c-api` crate. This isn't all that useful right
now because Wasmtime doesn't support imported/exported
modules/instances, but this is all necessary groundwork to getting that
exported at some point. I've added some light tests but I suspect the
bulk of the testing will come in a future commit.

One major change in this commit is that `SignatureIndex` no longer
follows type type index space in a wasm module. Instead a new
`TypeIndex` type is used to track that. Function signatures, still
indexed by `SignatureIndex`, are then packed together tightly.
2020-11-06 14:48:09 -06:00
Alex Crichton
a277cf5ee4 Store WasmFuncType in FuncType (#2365)
This commit updates `wasmtime::FuncType` to exactly store an internal
`WasmFuncType` from the cranelift crates. This allows us to remove a
translation layer when we are given a `FuncType` and want to get an
internal cranelift type out as a result.

The other major change from this commit was changing the constructor and
accessors of `FuncType` to be iterator-based instead of exposing
implementation details.
2020-11-05 08:49:03 -06:00
Alex Crichton
35327ed4d7 Print a message in log_wat while fuzzing
Similar to `log_wasm`, just indicates that a file was written.
2020-11-02 06:52:01 -08:00
Alex Crichton
b73b831892 Replace binaryen -ttf based fuzzing with wasm-smith (#2336)
This commit removes the binaryen support for fuzzing from wasmtime,
instead switching over to `wasm-smith`. In general it's great to have
what fuzzing we can, but our binaryen support suffers from a few issues:

* The Rust crate, binaryen-sys, seems largely unmaintained at this
  point. While we could likely take ownership and/or send PRs to update
  the crate it seems like the maintenance is largely on us at this point.

* Currently the binaryen-sys crate doesn't support fuzzing anything
  beyond MVP wasm, but we're interested at least in features like bulk
  memory and reference types. Additionally we'll also be interested in
  features like module-linking. New features would require either
  implementation work in binaryen or the binaryen-sys crate to support.

* We have 4-5 fuzz-bugs right now related to timeouts simply in
  generating a module for wasmtime to fuzz. One investigation along
  these lines in the past revealed a bug in binaryen itself, and in any
  case these bugs would otherwise need to get investigated, reported,
  and possibly fixed ourselves in upstream binaryen.

Overall I'm not sure at this point if maintaining binaryen fuzzing is
worth it with the advent of `wasm-smith` which has similar goals for
wasm module generation, but is much more readily maintainable on our
end.

Additonally in this commit I've added a fuzzer for wasm-smith's
`SwarmConfig`-based fuzzer which should expand the coverage of tested
modules.

Closes #2163
2020-10-29 10:02:59 -05:00
Alex Crichton
cdf158cd50 Fix enabling interrupts in fuzzers
Modifying the `Config` after the gneine has been created has no effect,
so be sure to flag the configuration as interruptible earlier.
2020-10-09 10:09:27 -07:00
Alex Crichton
2c6841041d Validate modules while translating (#2059)
* Validate modules while translating

This commit is a change to cranelift-wasm to validate each function body
as it is translated. Additionally top-level module translation functions
will perform module validation. This commit builds on changes in
wasmparser to perform module validation interwtwined with parsing and
translation. This will be necessary for future wasm features such as
module linking where the type behind a function index, for example, can
be far away in another module. Additionally this also brings a nice
benefit where parsing the binary only happens once (instead of having an
up-front serial validation step) and validation can happen in parallel
for each function.

Most of the changes in this commit are plumbing to make sure everything
lines up right. The major functional change here is that module
compilation should be faster by validating in parallel (or skipping
function validation entirely in the case of a cache hit). Otherwise from
a user-facing perspective nothing should be that different.

This commit does mean that cranelift's translation now inherently
validates the input wasm module. This means that the Spidermonkey
integration of cranelift-wasm will also be validating the function as
it's being translated with cranelift. The associated PR for wasmparser
(bytecodealliance/wasmparser#62) provides the necessary tools to create
a `FuncValidator` for Gecko, but this is something I'll want careful
review for before landing!

* Read function operators until EOF

This way we can let the validator take care of any issues with
mismatched `end` instructions and/or trailing operators/bytes.
2020-10-05 11:02:01 -05:00
Alex Crichton
b189321d61 Actually add instantiate-maybe-invalid fuzz target (#2190)
Forgot to add it to the manifest so it didn't actually get built!
2020-09-09 12:09:04 -05:00
Alex Crichton
38428e1fbb Expand modules instantiated in instantiate-wasm-smith
This commit uses the new `MaybeInvalidModule` type in `wasm-smith` to
try to explore more points in the fuzz target space in the
`instantiate-maybe-invalid` fuzz target. The goal here is to use the raw
fuzz input as the body of a function to stress the validator/decoder a
bit more, and try to get inputs we might not otherwise generate.
2020-09-08 14:29:27 -07:00
Alex Crichton
5893317a8e Fix execution of spectest fuzzer
We've enabled bulk memory and reference types by default now which means
that wasmtime in its default settings no longer passes the spec test
suite (due to changes in error messages in initialization), so when
we're running the spec test fuzzer be sure to disable reference types
and bulk memory since that's required to pass.
2020-08-10 06:55:36 -07:00
Alex Crichton
1000f21338 Update wasmparser to 0.59.0 (#2013)
This commit is intended to update wasmparser to 0.59.0. This primarily
includes bytecodealliance/wasm-tools#40 which is a large update to how
parsing and validation works. The impact on Wasmtime is pretty small at
this time, but over time I'd like to refactor the internals here to lean
more heavily on that upstream wasmparser refactoring.

For now, though, the intention is to get on the train of wasmparser's
latest `main` branch to ensure we get bug fixes and such.

As part of this update a few other crates and such were updated. This is
primarily to handle the new encoding of `ref.is_null` where the type is
not part of the instruction encoding any more.
2020-07-13 16:22:41 -05:00
Nick Fitzgerald
9b56203732 fuzzing: Enable reference types by default
Part of #929
2020-07-06 09:18:52 -07:00
Nick Fitzgerald
98e899f6b3 fuzz: Add a fuzz target for table.{get,set} operations
This new fuzz target exercises sequences of `table.get`s, `table.set`s, and
GCs.

It already found a couple bugs:

* Some leaks due to ref count cycles between stores and host-defined functions
  closing over those stores.

* If there are no live references for a PC, Cranelift can avoid emiting an
  associated stack map. This was running afoul of a debug assertion.
2020-06-30 12:00:57 -07:00
Alex Crichton
5fa4d36b0d Disable Cranelift debug verifier when fuzzing (#1851)
* Add CLI flags for internal cranelift options

This commit adds two flags to the `wasmtime` CLI:

* `--enable-cranelift-debug-verifier`
* `--enable-cranelift-nan-canonicalization`

These previously weren't exposed from the command line but have been
useful to me at least for reproducing slowdowns found during fuzzing on
the CLI.

* Disable Cranelift debug verifier when fuzzing

This commit disables Cranelift's debug verifier for our fuzz targets.
We've gotten a good number of timeouts on OSS-Fuzz and some I've
recently had some discussion over at google/oss-fuzz#3944 about this
issue and what we can do. The result of that discussion was that there
are two primary ways we can speed up our fuzzers:

* One is independent of Wasmtime, which is to tweak the flags used to
  compile code. The conclusion was that one flag was passed to LLVM
  which significantly increased runtime for very little benefit. This
  has now been disabled in rust-fuzz/cargo-fuzz#229.

* The other way is to reduce the amount of debug checks we run while
  fuzzing wasmtime itself. To put this in perspective, a test case which
  took ~100ms to instantiate was taking 50 *seconds* to instantiate in
  the fuzz target. This 500x slowdown was caused by a ton of
  multiplicative factors, but two major contributors were NaN
  canonicalization and cranelift's debug verifier. I suspect the NaN
  canonicalization itself isn't too pricy but when paired with the debug
  verifier in float-heavy code it can create lots of IR to verify.

This commit is specifically tackling this second point in an attempt to
avoid slowing down our fuzzers too much. The intent here is that we'll
disable the cranelift debug verifier for now but leave all other checks
enabled. If the debug verifier gets a speed boost we can try re-enabling
it, but otherwise it seems like for now it's otherwise not catching any
bugs and creating lots of noise about timeouts that aren't relevant.

It's not great that we have to turn off internal checks since that's
what fuzzing is supposed to trigger, but given the timeout on OSS-Fuzz
and the multiplicative effects of all the slowdowns we have when
fuzzing, I'm not sure we can afford the massive slowdown of the debug verifier.
2020-06-10 12:50:21 -05:00
Yury Delendik
15c68f2cc1 Disconnects Store state fields from Compiler (#1761)
*  Moves CodeMemory, VMInterrupts and SignatureRegistry from Compiler
*  CompiledModule holds CodeMemory and GdbJitImageRegistration
*  Store keeps track of its JIT code
*  Makes "jit_int.rs" stuff Send+Sync
*  Adds the threads example.
2020-06-02 13:44:39 -05:00
Nick Fitzgerald
f28b3738ee Rename anyref to externref across the board 2020-05-20 11:58:55 -07:00
Alex Crichton
57fb1c69c5 Enable the multi-value proposal by default (#1667)
This was merged into the wasm spec upstream in WebAssembly/spec#1145, so
let's follow the spec and enable it by default here as well!
2020-05-06 12:37:29 -05:00
Alex Crichton
d88a147b4d Add some debugging assistance to spectest oracle
Basically just log what's happening so if you're running the fuzzer you
can see what test is being run.
2020-04-30 12:25:06 -07:00
Alex Crichton
363cd2d20f Expose memory-related options in Config (#1513)
* Expose memory-related options in `Config`

This commit was initially motivated by looking more into #1501, but it
ended up balooning a bit after finding a few issues. The high-level
items in this commit are:

* New configuration options via `wasmtime::Config` are exposed to
  configure the tunable limits of how memories are allocated and such.
* The `MemoryCreator` trait has been updated to accurately reflect the
  required allocation characteristics that JIT code expects.
* A bug has been fixed in the cranelift wasm code generation where if no
  guard page was present bounds checks weren't accurately performed.

The new `Config` methods allow tuning the memory allocation
characteristics of wasmtime. Currently 64-bit platforms will reserve 6GB
chunks of memory for each linear memory, but by tweaking various config
options you can change how this is allocate, perhaps at the cost of
slower JIT code since it needs more bounds checks. The methods are
intended to be pretty thoroughly documented as to the effect they have
on the JIT code and what values you may wish to select. These new
methods have been added to the spectest fuzzer to ensure that various
configuration values for these methods don't affect correctness.

The `MemoryCreator` trait previously only allocated memories with a
`MemoryType`, but this didn't actually reflect the guarantees that JIT
code expected. JIT code is generated with an assumption about the
minimum size of the guard region, as well as whether memory is static or
dynamic (whether the base pointer can be relocated). These properties
must be upheld by custom allocation engines for JIT code to perform
correctly, so extra parameters have been added to
`MemoryCreator::new_memory` to reflect this.

Finally the fuzzing with `Config` turned up an issue where if no guard
pages present the wasm code wouldn't correctly bounds-check memory
accesses. The issue here was that with a guard page we only need to
bounds-check the first byte of access, but without a guard page we need
to bounds-check the last byte of access. This meant that the code
generation needed to account for the size of the memory operation
(load/store) and use this as the offset-to-check in the no-guard-page
scenario. I've attempted to make the various comments in cranelift a bit
more exhaustive too to hopefully make it a bit clearer for future
readers!

Closes #1501

* Review comments

* Update a comment
2020-04-29 17:10:00 -07:00
Alex Crichton
c9a0ba81a0 Implement interrupting wasm code, reimplement stack overflow (#1490)
* Implement interrupting wasm code, reimplement stack overflow

This commit is a relatively large change for wasmtime with two main
goals:

* Primarily this enables interrupting executing wasm code with a trap,
  preventing infinite loops in wasm code. Note that resumption of the
  wasm code is not a goal of this commit.

* Additionally this commit reimplements how we handle stack overflow to
  ensure that host functions always have a reasonable amount of stack to
  run on. This fixes an issue where we might longjmp out of a host
  function, skipping destructors.

Lots of various odds and ends end up falling out in this commit once the
two goals above were implemented. The strategy for implementing this was
also lifted from Spidermonkey and existing functionality inside of
Cranelift. I've tried to write up thorough documentation of how this all
works in `crates/environ/src/cranelift.rs` where gnarly-ish bits are.

A brief summary of how this works is that each function and each loop
header now checks to see if they're interrupted. Interrupts and the
stack overflow check are actually folded into one now, where function
headers check to see if they've run out of stack and the sentinel value
used to indicate an interrupt, checked in loop headers, tricks functions
into thinking they're out of stack. An interrupt is basically just
writing a value to a location which is read by JIT code.

When interrupts are delivered and what triggers them has been left up to
embedders of the `wasmtime` crate. The `wasmtime::Store` type has a
method to acquire an `InterruptHandle`, where `InterruptHandle` is a
`Send` and `Sync` type which can travel to other threads (or perhaps
even a signal handler) to get notified from. It's intended that this
provides a good degree of flexibility when interrupting wasm code. Note
though that this does have a large caveat where interrupts don't work
when you're interrupting host code, so if you've got a host import
blocking for a long time an interrupt won't actually be received until
the wasm starts running again.

Some fallout included from this change is:

* Unix signal handlers are no longer registered with `SA_ONSTACK`.
  Instead they run on the native stack the thread was already using.
  This is possible since stack overflow isn't handled by hitting the
  guard page, but rather it's explicitly checked for in wasm now. Native
  stack overflow will continue to abort the process as usual.

* Unix sigaltstack management is now no longer necessary since we don't
  use it any more.

* Windows no longer has any need to reset guard pages since we no longer
  try to recover from faults on guard pages.

* On all targets probestack intrinsics are disabled since we use a
  different mechanism for catching stack overflow.

* The C API has been updated with interrupts handles. An example has
  also been added which shows off how to interrupt a module.

Closes #139
Closes #860
Closes #900

* Update comment about magical interrupt value

* Store stack limit as a global value, not a closure

* Run rustfmt

* Handle review comments

* Add a comment about SA_ONSTACK

* Use `usize` for type of `INTERRUPTED`

* Parse human-readable durations

* Bring back sigaltstack handling

Allows libstd to print out stack overflow on failure still.

* Add parsing and emission of stack limit-via-preamble

* Fix new example for new apis

* Fix host segfault test in release mode

* Fix new doc example
2020-04-21 11:03:28 -07:00
Dan Gohman
9364eb1d98 Refactor (#1524)
* Compute instance exports on demand.

Instead having instances eagerly compute a Vec of Externs, and bumping
the refcount for each Extern, compute Externs on demand.

This also enables `Instance::get_export` to avoid doing a linear search.

This also means that the closure returned by `get0` and friends now
holds an `InstanceHandle` to dynamically hold the instance live rather
than being scoped to a lifetime.

* Compute module imports and exports on demand too.

And compute Extern::ty on demand too.

* Add a utility function for computing an ExternType.

* Add a utility function for looking up a function's signature.

* Add a utility function for computing the ValType of a Global.

* Rename wasmtime_environ::Export to EntityIndex.

This helps differentiate it from other Export types in the tree, and
describes what it is.

* Fix a typo in a comment.

* Simplify module imports and exports.

* Make `Instance::exports` return the export names.

This significantly simplifies the public API, as it's relatively common
to need the names, and this avoids the need to do a zip with
`Module::exports`.

This also changes `ImportType` and `ExportType` to have public members
instead of private members and accessors, as I find that simplifies the
usage particularly in cases where there are temporary instances.

* Remove `Instance::module`.

This doesn't quite remove `Instance`'s `module` member, it gets a step
closer.

* Use a InstanceHandle utility function.

* Don't consume self in the `Func::get*` methods.

Instead, just create a closure containing the instance handle and the
export for them to call.

* Use `ExactSizeIterator` to avoid needing separate `num_*` methods.

* Rename `Extern::func()` etc. to `into_func()` etc.

* Revise examples to avoid using `nth`.

* Add convenience methods to instance for getting specific extern types.

* Use the convenience functions in more tests and examples.

* Avoid cloning strings for `ImportType` and `ExportType`.

* Remove more obviated clone() calls.

* Simplify `Func`'s closure state.

* Make wasmtime::Export's fields private.

This makes them more consistent with ExportType.

* Fix compilation error.

* Make a lifetime parameter explicit, and use better lifetime names.

Instead of 'me, use 'instance and 'module to make it clear what the
lifetime is.

* More lifetime cleanups.
2020-04-20 15:55:33 -05:00
Alex Crichton
4c82da440a Move most wasmtime tests into one test suite (#1544)
* Move most wasmtime tests into one test suite

This commit moves most wasmtime tests into a single test suite which
gets compiled into one executable instead of having lots of test
executables. The goal here is to reduce disk space on CI, and this
should be achieved by having fewer executables which means fewer copies
of `libwasmtime.rlib` linked across binaries on the system. More
importantly though this means that DWARF debug information should only
be in one executable rather than duplicated across many.

* Share more build caches

Globally set `RUSTFLAGS` to `-Dwarnings` instead of individually so all
build steps share the same value.

* Allow some dead code in cranelift-codegen

Prevents having to fix all warnings for all possible feature
combinations, only the main ones which come up.

* Update some debug file paths
2020-04-17 17:22:12 -05:00
Alex Crichton
6dde222992 Add a spec test fuzzer for Config (#1509)
* Add a spec test fuzzer for Config

This commit adds a new fuzzer which is intended to run on oss-fuzz. This
fuzzer creates and arbitrary `Config` which *should* pass spec tests and
then asserts that it does so. The goal here is to weed out any
accidental bugs in global configuration which could cause
non-spec-compliant behavior.

* Move implementation to `fuzzing` crate
2020-04-15 08:29:12 -05:00
Alex Crichton
bd374fd6fc Add Wasmtime-specific C API functions to return errors (#1467)
* Add Wasmtime-specific C API functions to return errors

This commit adds new `wasmtime_*` symbols to the C API, many of which
mirror the existing counterparts in the `wasm.h` header. These APIs are
enhanced in a number of respects:

* Detailed error information is now available through a
  `wasmtime_error_t`. Currently this only exposes one function which is
  to extract a string version of the error.

* There is a distinction now between traps and errors during
  instantiation and function calling. Traps only happen if wasm traps,
  and errors can happen for things like runtime type errors when
  interacting with the API.

* APIs have improved safety with respect to embedders where the lengths
  of arrays are now taken as explicit parameters rather than assumed
  from other parameters.

* Handle trap updates

* Update C examples

* Fix memory.c compile on MSVC

* Update test assertions

* Refactor C slightly

* Bare-bones .NET update

* Remove bogus nul handling
2020-04-06 15:13:06 -05:00
teapotd
2180e9ce16 fuzzing: Enable NaN canonicalization (#1334)
* Method to enable NaN canonicalization in Config

* Use fuzz_default_config in DifferentialConfig

* Enable NaN canonicalization for fuzzing
2020-03-31 09:22:08 -05:00
Alex Crichton
afd980b4f6 Refactor the internals of Func to remove layers of indirection (#1363)
* Remove `WrappedCallable` indirection

At this point `Func` has evolved quite a bit since inception and the
`WrappedCallable` trait I don't believe is needed any longer. This
should help clean up a few entry points by having fewer traits in play.

* Remove the `Callable` trait

This commit removes the `wasmtime::Callable` trait, changing the
signature of `Func::new` to take an appropriately typed `Fn`.
Additionally the function now always takes `&Caller` like `Func::wrap`
optionally can, to empower `Func::new` to have the same capabilities of
`Func::wrap`.

* Add a test for an already-fixed issue

Closes #849

* rustfmt

* Update more locations for `Callable`

* rustfmt

* Remove a stray leading borrow

* Review feedback

* Remove unneeded `wasmtime_call_trampoline` shim
2020-03-19 14:21:45 -05:00
Alex Crichton
210bfddfa9 Fix unused imports in oracles 2020-03-17 12:02:09 -07:00