Commit Graph

467 Commits

Author SHA1 Message Date
Chris Fallin
f227699536 Make ISLE part of the root Cargo workspace. 2021-11-11 16:00:12 -08:00
Pat Hickey
6428ac80d0 cargo update -p userfaultfd to 0.4.2 (#3505)
which has bugfixes to work correctly on linux 5.11 and above, which is
required for github's ubuntu-latest builder as of this morning
2021-11-03 16:50:40 -05:00
Benjamin Bouvier
c952969389 Remove unused dependencies (#3490)
* Remove unused dependencies in Cranelift

* add serde to the current workspace

* remove more unused dependencies in wasmtime?
2021-11-02 12:08:30 -05:00
Alex Crichton
9a27fdad86 Update v8 used during fuzzing (#3493)
This commit updates the crate name from `rusty_v8` to `v8` as well since
the upstream bindings have sinced moved. I originally wanted to do this
to see if a fix for one of our fuzz bugs was pulled in but I don't think
the fix has been pulled in yet. Despite that it seems reasonable to go
ahead and update.
2021-11-01 09:18:11 -05:00
wasmtime-publish
c1a6a0523d Release Wasmtime 0.31.0 (#3489)
* Bump Wasmtime to 0.31.0

[automatically-tag-and-release-this-commit]

* Update 0.31.0 release notes

Co-authored-by: Wasmtime Publish <wasmtime-publish@users.noreply.github.com>
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
2021-10-29 09:09:35 -05:00
Pat Hickey
7962fe3f43 update tracing-subscriber (#3473)
which gets rid of the transitive dep on `chrono` that was affected by
`RUSTSEC-2020-0159`.
2021-10-26 09:06:45 -05:00
Alex Crichton
490d49a768 Adjust dependency directives between crates (#3420)
* Adjust dependency directives between crates

This commit is a preparation for the release process for Wasmtime. The
specific changes here are to delineate which crates are "public", and
all version requirements on non-public crates will now be done with
`=A.B.C` version requirements instead of today's `A.B.C` version
requirements.

The purpose for doing this is to assist with patch releases that might
happen in the future. Patch releases of wasmtime are already required to
not break the APIs of "public" crates, but no such guarantee is given
about "internal" crates. This means that a patch release runs the risk,
for example, of breaking an internal API. In doing so though we would
also need to release a new major version of the internal crate, but we
wouldn't have a great hole in the number scheme of major versions to do
so. By using `=A.B.C` requirements for internal crates it means we can
safely ignore strict semver-compatibility between releases of internal
crates for patch releases, since the only consumers of the crate will be
the corresponding patch release of the `wasmtime` crate itself (or other
public crates).

The `publish.rs` script has been updated with a check to verify that
dependencies on internal crates are all specified with an `=`
dependency, and dependnecies on all public crates are without a `=`
dependency. This will hopefully make it so we don't have to worry about
what to use where, we just let CI tell us what to do. Using this
modification all version dependency declarations have been updated.

Note that some crates were adjusted to simply remove their `version`
requirement in cases such as the crate wasn't published anyway (`publish
= false` was specified) or it's in the `dev-dependencies` section which
doesn't need version specifiers for path dependencies.

* Switch to normal sever deps for cranelift dependencies

These crates will now all be considered "public" where in patch releases
they will be guaranteed to not have breaking changes.
2021-10-26 09:06:03 -05:00
Pat Hickey
18a355e092 give sychronous ResourceLimiter an async alternative 2021-10-21 12:10:03 -07:00
Chris Fallin
e9921574d7 Update to regalloc.rs 0.0.32.
It appears that some allocation heuristics have changed slightly since
0.0.31, so some of the golden-output filetests are updated as well.
Ideally we would rely more on runtests rather than golden-compilation
tests; but for now this is sufficient. (I'm not sure exactly what in
regalloc.rs changed to alter these heuristics; it's actually been almost
a year since the 0.0.31 release with several refactorings and tweaks
merged since then.)

Fixes #3441.
2021-10-20 15:28:42 -07:00
Alex Crichton
fb585fde40 Update the wast crate dependency (#3464)
Pulls in a few minor fixes for stack overflows with module linking as
well as some updates to other various wasm proposals.
2021-10-20 11:25:52 -05:00
Alex Crichton
e2a724ce18 Update the object crate to 0.27.0 (#3465)
Mostly just keeping us up to date with changes there since we somewhat
heavily rely on it now.
2021-10-20 10:52:06 -05:00
Alex Crichton
9c6884e28d Update the spec reference testsuite submodule (#3450)
* Update the spec reference testsuite submodule

This commit brings in recent updates to the spec test suite. Most of the
changes here were already fixed in `wasmparser` with some tweaks to
esoteric modules, but Wasmtime also gets a bug fix where where import
matching for the size of tables/memories is based on the current runtime
size of the table/memory rather than the original type of the
table/memory. This means that during type matching the actual value is
consulted for its size rather than using the minimum size listed in its
type.

* Fix now-missing directories in build script
2021-10-13 16:14:12 -05:00
Pat Hickey
8554d69e4b update userfaultfd to 0.4.1 (#3442)
which updates nix to 0.23.0, getting rid of the benign RUSTSEC-2021-0119
in our dep tree
2021-10-11 13:06:54 -05:00
Alex Crichton
713ce07d35 Add some debug logging for timing in module compiles (#3417)
* Add some debug logging for timing in module compiles

This is sometimes helpful when debugging slow compiles from fuzz bugs or
similar.

* Fix total duration calculation to not double-count
2021-10-11 12:50:15 -05:00
Frank Denis
0ee1689e2f Update wasi-crypto crate
This pulls more recent versions of its dependencies.
2021-10-10 21:34:43 +02:00
Dan Gohman
fa108d9a86 Remove the rsix dependency in cranelift-native. (#3395)
Revert the part of 47490b4383 which
changed cranelift-native to use rsix. It's just one call, and this lets
Cranelift users that don't otherwise depend on rsix to avoid it.
2021-09-30 06:11:29 -07:00
Alex Crichton
1ee2af0098 Remove the lightbeam backend (#3390)
This commit removes the Lightbeam backend from Wasmtime as per [RFC 14].
This backend hasn't received maintenance in quite some time, and as [RFC
14] indicates this doesn't meet the threshold for keeping the code
in-tree, so this commit removes it.

A fast "baseline" compiler may still be added in the future. The
addition of such a backend should be in line with [RFC 14], though, with
the principles we now have for stable releases of Wasmtime. I'll close
out Lightbeam-related issues once this is merged.

[RFC 14]: https://github.com/bytecodealliance/rfcs/pull/14
2021-09-27 12:27:19 -05:00
Alex Crichton
98831fe4e2 Update zeroize_derive to fix a rustsec warning (#3389)
Should hopefully appease CI
2021-09-24 15:07:16 -05:00
Dan Gohman
47490b4383 Use rsix to make system calls in Wasmtime. (#3355)
* Use rsix to make system calls in Wasmtime.

`rsix` is a system call wrapper crate that we use in `wasi-common`,
which can provide the following advantages in the rest of Wasmtime:

 - It eliminates some `unsafe` blocks in Wasmtime's code. There's
   still an `unsafe` block in the library, but this way, the `unsafe`
   is factored out and clearly scoped.

 - And, it makes error handling more consistent, factoring out code for
   checking return values and `io::Error::last_os_error()`, and code that
   does `errno::set_errno(0)`.

This doesn't cover *all* system calls; `rsix` doesn't implement
signal-handling APIs, and this doesn't cover calls made through `std` or
crates like `userfaultfd`, `rand`, and `region`.
2021-09-17 15:28:56 -07:00
Nick Fitzgerald
a1f4b46f64 Bump Wasmtime to version 0.30.0; cranelift to 0.77.0 2021-09-17 10:33:50 -07:00
Dan Gohman
4d86f0ca10 Update to cap-std 0.19.0 and rsix 0.22.4. (#3331)
This pulls in the s390x fix needed by #3330.

Also a small `rsix` API update; `PollFdVec` has been removed in favor of
just using `Vec<PollFd>`.
2021-09-11 12:28:30 -05:00
Alex Crichton
c73673559b Avoid vector allocations in wasm->host calls (#3294)
This commit improves the runtime support for wasm-to-host invocations
for functions created with `Func::new` or `wasmtime_func_new` in the C
API. Previously a `Vec` (sometimes a `SmallVec`) would be dynamically
allocated on each host call to store the arguments that are coming from
wasm and going to the host. In the case of the `wasmtime` crate we need
to decode the `u128`-stored values, and in the case of the C API we need
to decode the `Val` into the C API's `wasmtime_val_t`.

The technique used in this commit is to store a singular `Vec<T>` inside
the "store", be it the literal `Store<T>` or within the `T` in the case
of the C API, which can be reused across wasm->host calls. This means
that we're unlikely to actually perform dynamic memory allocation and
instead we should hit a faster path where the `Vec` always has enough
capacity.

Note that this is just a mild improvement for `Func::new`-based
functions. It's still the case that `Func::wrap` is much faster, but
unfortunately the C API doesn't have access to `Func::wrap`, so the main
motivation here is accelerating the C API.
2021-09-03 15:14:21 -05:00
Dan Gohman
197aec9a08 Update io-lifetimes, cap-std, and rsix (#3269)
- Fixes for compiling on OpenBSD

 - io-lifetimes 0.3.0 has an option (io_lifetimes_use_std, which is off
   by default) for testing the `io_safety` feature in Rust nightly.
2021-08-31 13:02:37 -07:00
Alex Crichton
4376cf2609 Add differential fuzzing against V8 (#3264)
* Add differential fuzzing against V8

This commit adds a differential fuzzing target to Wasmtime along the
lines of the wasmi and spec interpreters we already have, but with V8
instead. The intention here is that wasmi is unlikely to receive updates
over time (e.g. for SIMD), and the spec interpreter is not suitable for
fuzzing against in general due to its performance characteristics. The
hope is that V8 is indeed appropriate to fuzz against because it's
naturally receiving updates and it also is expected to have good
performance.

Here the `rusty_v8` crate is used which provides bindings to V8 as well
as precompiled binaries by default. This matches exactly the use case we
need and at least for now I think the `rusty_v8` crate will be
maintained by the Deno folks as they continue to develop it. If it
becomes an issue though maintaining we can evaluate other options to
have differential fuzzing against.

For now this commit enables the SIMD and bulk-memory feature of
fuzz-target-generation which should enable them to get
differentially-fuzzed with V8 in addition to the compilation fuzzing
we're already getting.

* Use weak linkage for GDB jit helpers

This should help us deduplicate our symbol with other JIT runtimes, if
any. For now this leans on some C helpers to define the weak linkage
since Rust doesn't support that on stable yet.

* Don't use rusty_v8 on MinGW

They don't have precompiled libraries there.

* Fix msvc build

* Comment about execution
2021-08-31 09:34:55 -05:00
Alex Crichton
eb251deca9 Remove scroll dependency from wasmtime-jit (#3260)
Similar functionality to `scroll` is provided with the `object` crate
and doesn't have a `*_derive` crate to go with it. This commit updates
the jitdump linux support to use `object` instead of `scroll` to achieve
the needs of writing structs-as-bytes onto disk.
2021-08-30 13:26:07 -05:00
Alex Crichton
a978c7e384 Update wasm-smith (#3267)
Brings in a fix for a fuzz-bug found on oss-fuzz.
2021-08-30 11:48:50 -05:00
Alex Crichton
c73be1f13a Use an mmap-friendly serialization format (#3257)
* Use an mmap-friendly serialization format

This commit reimplements the main serialization format for Wasmtime's
precompiled artifacts. Previously they were generally a binary blob of
`bincode`-encoded metadata prefixed with some versioning information.
The downside of this format, though, is that loading a precompiled
artifact required pushing all information through `bincode`. This is
inefficient when some data, such as trap/address tables, are rarely
accessed.

The new format added in this commit is one which is designed to be
`mmap`-friendly. This means that the relevant parts of the precompiled
artifact are already page-aligned for updating permissions of pieces
here and there. Additionally the artifact is optimized so that if data
is rarely read then we can delay reading it until necessary.

The new artifact format for serialized modules is an ELF file. This is
not a public API guarantee, so it cannot be relied upon. In the meantime
though this is quite useful for exploring precompiled modules with
standard tooling like `objdump`. The ELF file is already constructed as
part of module compilation, and this is the main contents of the
serialized artifact.

THere is some extra information, though, not encoded in each module's
individual ELF file such as type information. This information continues
to be `bincode`-encoded, but it's intended to be much smaller and much
faster to deserialize. This extra information is appended to the end of
the ELF file. This means that the original ELF file is still a valid ELF
file, we just get to have extra bits at the end. More information on the
new format can be found in the module docs of the serialization module
of Wasmtime.

Another refatoring implemented as part of this commit is to deserialize
and store object files directly in `mmap`-backed storage. This avoids
the need to copy bytes after the artifact is loaded into memory for each
compiled module, and in a future commit it opens up the door to avoiding
copying the text section into a `CodeMemory`. For now, though, the main
change is that copies are not necessary when loading from a precompiled
compilation artifact once the artifact is itself in mmap-based memory.

To assist with managing `mmap`-based memory a new `MmapVec` type was
added to `wasmtime_jit` which acts as a form of `Vec<T>` backed by a
`wasmtime_runtime::Mmap`. This type notably supports `drain(..N)` to
slice the buffer into disjoint regions that are all separately owned,
such as having a separately owned window into one artifact for all
object files contained within.

Finally this commit implements a small refactoring in `wasmtime-cache`
to use the standard artifact format for cache entries rather than a
bincode-encoded version. This required some more hooks for
serializing/deserializing but otherwise the crate still performs as
before.

* Review comments
2021-08-30 09:19:20 -05:00
Alex Crichton
d12f1d77e6 Convert compilation artifacts to just bytes (#3239)
* Convert compilation artifacts to just bytes

This commit strips the `CompilationArtifacts` type down to simply a list
of bytes. This moves all extra metadata elsewhere to live within the
list of bytes itself as `bincode`-encoded information.

Small affordance is made to avoid an in-process
serialize-then-deserialize round-trip for use cases like `Module::new`,
but otherwise this is mostly just moving some data around.

* Rename data section to `.rodata.wasm`
2021-08-26 21:17:02 -05:00
Alex Crichton
d74cc33856 Merge wasmtime-jit and wasmtime-profiling (#3247)
* Merge `wasmtime-jit` and `wasmtime-profiling`

This commit merges the `wasmtime-profiling` crate into the
`wasmtime-jit` crate. It wasn't really buying a ton being a separate
crate and an upcoming refactoring I'd like to do is to remove the
`FinishedFunctions` structure. To enable the profilers to work as they
used to this commit changes them to pass `CompiledModule` as the
argument, but this only works if the profiling trait can see the
`CompiledModule` type.

* Fix a length calculation
2021-08-26 16:22:11 -05:00
Alex Crichton
7d05ebe7ff Move wasm data/debuginfo into the ELF compilation image (#3235)
* Move wasm data/debuginfo into the ELF compilation image

This commit moves existing allocations of `Box<[u8]>` stored separately
from compilation's final ELF image into the ELF image itself. The goal
of this commit is to reduce the amount of data which `bincode` will need
to process in the future. DWARF debugging information and wasm data
segments can be quite large, and they're relatively rarely read, so
there's typically no need to copy them around. Instead by moving them
into the ELF image this opens up the opportunity in the future to
eliminate copies and use data directly as-found in the image itself.

For information accessed possibly-multiple times, such as the wasm data
ranges, the indexes of the data within the ELF image are computed when
a `CompiledModule` is created. These indexes are then used to directly
index into the image without having to root around in the ELF file each
time they're accessed.

One other change located here is that the symbolication context
previously cloned the debug information into it to adhere to the
`'static` lifetime safely, but this isn't actually ever used in
`wasmtime` right now so the unsafety around this has been removed and
instead borrowed data is returned (no more clones, yay!).

* Fix lightbeam
2021-08-25 09:03:07 -05:00
Alex Crichton
dd58d8d61f Update some wasm-tools deps
Fixes a few fuzz bugs found on CI
2021-08-23 15:41:33 -07:00
Alex Crichton
3e65b46779 Update crossbeam-deque to appease cargo-deny (#3227)
Hopefully unblocks some CI
2021-08-23 14:35:31 -05:00
Alex Crichton
ddfadaeb38 Add a cranelift compile-time feature to wasmtime (#3206)
* Remove unnecessary into_iter/map

Forgotten from a previous refactoring, this variable was already of the
right type!

* Move `wasmtime_jit::Compiler` into `wasmtime`

This `Compiler` struct is mostly a historical artifact at this point and
wasn't necessarily pulling much weight any more. This organization also
doesn't lend itself super well to compiling out `cranelift` when the
`Compiler` here is used for both parallel iteration configuration
settings as well as compilation.

The movement into `wasmtime` is relatively small, with
`Module::build_artifacts` being the main function added here which is a
merging of the previous functions removed from the `wasmtime-jit` crate.

* Add a `cranelift` compile-time feature to `wasmtime`

This commit concludes the saga of refactoring Wasmtime and making
Cranelift an optional dependency by adding a new Cargo feature to the
`wasmtime` crate called `cranelift`, which is enabled by default.

This feature is implemented by having a new cfg for `wasmtime` itself,
`cfg(compiler)`, which is used wherever compilation is necessary. This
bubbles up to disable APIs such as `Module::new`, `Func::new`,
`Engine::precompile_module`, and a number of `Config` methods affecting
compiler configuration. Checks are added to CI that when built in this
mode Wasmtime continues to successfully build. It's hoped that although
this is effectively "sprinkle `#[cfg]` until things compile" this won't
be too too bad to maintain over time since it's also an use case we're
interested in supporting.

With `cranelift` disabled the only way to create a `Module` is with the
`Module::deserialize` method, which requires some form of precompiled
artifact.

Two consequences of this change are:

* `Module::serialize` is also disabled in this mode. The reason for this
  is that serialized modules contain ISA/shared flags encoded in them
  which were used to produce the compiled code. There's no storage for
  this if compilation is disabled. This could probably be re-enabled in
  the future if necessary, but it may not end up being all that necessary.

* Deserialized modules are not checked to ensure that their ISA/shared
  flags are compatible with the host CPU. This is actually already the
  case, though, with normal modules. We'll likely want to fix this in
  the future using a shared implementation for both these locations.

Documentation should be updated to indicate that `cranelift` can be
disabled, although it's not really the most prominent documentation
because this is expected to be a somewhat niche use case (albeit
important, just not too common).

* Always enable cranelift for the C API

* Fix doc example builds

* Fix check tests on GitHub Actions
2021-08-18 16:47:47 -05:00
Alex Crichton
87c33c2969 Remove wasmtime-environ's dependency on cranelift-codegen (#3199)
* Move `CompiledFunction` into wasmtime-cranelift

This commit moves the `wasmtime_environ::CompiledFunction` type into the
`wasmtime-cranelift` crate. This type has lots of Cranelift-specific
pieces of compilation and doesn't need to be generated by all Wasmtime
compilers. This replaces the usage in the `Compiler` trait with a
`Box<Any>` type that each compiler can select. Each compiler must still
produce a `FunctionInfo`, however, which is shared information we'll
deserialize for each module.

The `wasmtime-debug` crate is also folded into the `wasmtime-cranelift`
crate as a result of this commit. One possibility was to move the
`CompiledFunction` commit into its own crate and have `wasmtime-debug`
depend on that, but since `wasmtime-debug` is Cranelift-specific at this
time it didn't seem like it was too too necessary to keep it separate.
If `wasmtime-debug` supports other backends in the future we can
recreate a new crate, perhaps with it refactored to not depend on
Cranelift.

* Move wasmtime_environ::reference_type

This now belongs in wasmtime-cranelift and nowhere else

* Remove `Type` reexport in wasmtime-environ

One less dependency on `cranelift-codegen`!

* Remove `types` reexport from `wasmtime-environ`

Less cranelift!

* Remove `SourceLoc` from wasmtime-environ

Change the `srcloc`, `start_srcloc`, and `end_srcloc` fields to a custom
`FilePos` type instead of `ir::SourceLoc`. These are only used in a few
places so there's not much to lose from an extra abstraction for these
leaf use cases outside of cranelift.

* Remove wasmtime-environ's dep on cranelift's `StackMap`

This commit "clones" the `StackMap` data structure in to
`wasmtime-environ` to have an independent representation that that
chosen by Cranelift. This allows Wasmtime to decouple this runtime
dependency of stack map information and let the two evolve
independently, if necessary.

An alternative would be to refactor cranelift's implementation into a
separate crate and have wasmtime depend on that but it seemed a bit like
overkill to do so and easier to clone just a few lines for this.

* Define code offsets in wasmtime-environ with `u32`

Don't use Cranelift's `binemit::CodeOffset` alias to define this field
type since the `wasmtime-environ` crate will be losing the
`cranelift-codegen` dependency soon.

* Commit to using `cranelift-entity` in Wasmtime

This commit removes the reexport of `cranelift-entity` from the
`wasmtime-environ` crate and instead directly depends on the
`cranelift-entity` crate in all referencing crates. The original reason
for the reexport was to make cranelift version bumps easier since it's
less versions to change, but nowadays we have a script to do that.
Otherwise this encourages crates to use whatever they want from
`cranelift-entity` since  we'll always depend on the whole crate.

It's expected that the `cranelift-entity` crate will continue to be a
lean crate in dependencies and suitable for use at both runtime and
compile time. Consequently there's no need to avoid its usage in
Wasmtime at runtime, since "remove Cranelift at compile time" is
primarily about the `cranelift-codegen` crate.

* Remove most uses of `cranelift-codegen` in `wasmtime-environ`

There's only one final use remaining, which is the reexport of
`TrapCode`, which will get handled later.

* Limit the glob-reexport of `cranelift_wasm`

This commit removes the glob reexport of `cranelift-wasm` from the
`wasmtime-environ` crate. This is intended to explicitly define what
we're reexporting and is a transitionary step to curtail the amount of
dependencies taken on `cranelift-wasm` throughout the codebase. For
example some functions used by debuginfo mapping are better imported
directly from the crate since they're Cranelift-specific. Note that
this is intended to be a temporary state affairs, soon this reexport
will be gone entirely.

Additionally this commit reduces imports from `cranelift_wasm` and also
primarily imports from `crate::wasm` within `wasmtime-environ` to get a
better sense of what's imported from where and what will need to be
shared.

* Extract types from cranelift-wasm to cranelift-wasm-types

This commit creates a new crate called `cranelift-wasm-types` and
extracts type definitions from the `cranelift-wasm` crate into this new
crate. The purpose of this crate is to be a shared definition of wasm
types that can be shared both by compilers (like Cranelift) as well as
wasm runtimes (e.g. Wasmtime). This new `cranelift-wasm-types` crate
doesn't depend on `cranelift-codegen` and is the final step in severing
the unconditional dependency from Wasmtime to `cranelift-codegen`.

The final refactoring in this commit is to then reexport this crate from
`wasmtime-environ`, delete the `cranelift-codegen` dependency, and then
update all `use` paths to point to these new types.

The main change of substance here is that the `TrapCode` enum is
mirrored from Cranelift into this `cranelift-wasm-types` crate. While
this unfortunately results in three definitions (one more which is
non-exhaustive in Wasmtime itself) it's hopefully not too onerous and
ideally something we can patch up in the future.

* Get lightbeam compiling

* Remove unnecessary dependency

* Fix compile with uffd

* Update publish script

* Fix more uffd tests

* Rename cranelift-wasm-types to wasmtime-types

This reflects the purpose a bit more where it's types specifically
intended for Wasmtime and its support.

* Fix publish script
2021-08-18 13:14:52 -05:00
Dan Gohman
fde767fedc Update to cap-std 0.17.0. (#3198)
This completes the posish->rsix rename, and contains a number of other
minor cleanups, including avoiding the `cstr` dependency.
2021-08-17 16:08:03 -07:00
Alex Crichton
e8aa7bb53b Reimplement how unwind information is stored (#3180)
* Reimplement how unwind information is stored

This commit is a major refactoring of how unwind information is stored
after compilation of a function has finished. Previously we would store
the raw `UnwindInfo` as a result of compilation and this would get
serialized/deserialized alongside the rest of the ELF object that
compilation creates. Whenever functions were registered with
`CodeMemory` this would also result in registering unwinding information
dynamically at runtime, which in the case of Unix, for example, would
dynamically created FDE/CIE entries on-the-fly.

Eventually I'd like to support compiling Wasmtime without Cranelift, but
this means that `UnwindInfo` wouldn't be easily available to decode into
and create unwinding information from. To solve this I've changed the
ELF object created to have the unwinding information encoded into it
ahead-of-time so loading code into memory no longer needs to create
unwinding tables. This change has two different implementations for
Windows/Unix:

* On Windows the implementation was much easier. The unwinding
  information on Windows is already stored after the function itself in
  the text section. This was actually slightly duplicated in object
  building and in code memory allocation. Now the object building
  continues to do the same, recording unwinding information after
  functions, and code memory no longer manually tracks this.
  Additionally Wasmtime will emit a special custom section in the object
  file with unwinding information which is the list of
  `RUNTIME_FUNCTION` structures that `RtlAddFunctionTable` expects. This
  means that the object file has all the information precompiled into it
  and registration at runtime is simply passing a few pointers around to
  the runtime.

* Unix was a little bit more difficult than Windows. Today a `.eh_frame`
  section is created on-the-fly with offsets in FDEs specified as the
  absolute address that functions are loaded at. This absolute
  address hindered the ability to precompile the FDE into the object
  file itself. I've switched how addresses are encoded, though, to using
  `DW_EH_PE_pcrel` which means that FDE addresses are now specified
  relative to the FDE itself. This means that we can maintain a fixed
  offset between the `.eh_frame` loaded in memory and the beginning of
  code memory. When doing so this enables precompiling the `.eh_frame`
  section into the object file and at runtime when loading an object no
  further construction of unwinding information is needed.

The overall result of this commit is that unwinding information is no
longer stored in its cranelift-data-structure form on disk. This means
that this unwinding information format is only present during
compilation, which will make it that much easier to compile out
cranelift in the future.

This commit also significantly refactors `CodeMemory` since the way
unwinding information is handled is not much different from before.
Previously `CodeMemory` was suitable for incrementally adding more and
more functions to it, but nowadays a `CodeMemory` either lives per
module (in which case all functions are known up front) or it's created
once-per-`Func::new` with two trampolines. In both cases we know all
functions up front so the functionality of incrementally adding more and
more segments is no longer needed. This commit removes the ability to
add a function-at-a-time in `CodeMemory` and instead it can now only
load objects in their entirety. A small helper function is added to
build a small object file for trampolines in `Func::new` to handle
allocation there.

Finally, this commit also folds the `wasmtime-obj` crate directly into
the `wasmtime-cranelift` crate and its builder structure to be more
amenable to this strategy of managing unwinding tables.

It is not intentional to have any real functional change as a result of
this commit. This might accelerate loading a module from cache slightly
since less work is needed to manage the unwinding information, but
that's just a side benefit from the main goal of this commit which is to
remove the dependence on cranelift unwinding information being available
at runtime.

* Remove isa reexport from wasmtime-environ

* Trim down reexports of `cranelift-codegen`

Remove everything non-essential so that only the bits which will need to
be refactored out of cranelift remain.

* Fix debug tests

* Review comments
2021-08-17 17:14:18 -05:00
Alex Crichton
0642e62f16 Use wasm-smith to canonicalize NaN in differential fuzzing (#3195)
* Update wasm-smith to 0.7.0

* Canonicalize NaN with wasm-smith for differential fuzzing

This then also enables floating point executing in wasmi in addition to
the spec interpreter. With NaN canonicalization at the wasm level this
means that we should be producing deterministic results between Wasmtime
and these alternative implementations.
2021-08-17 11:42:22 -05:00
Alex Crichton
0313e30d76 Remove dependency on TargetIsa from Wasmtime crates (#3178)
This commit started off by deleting the `cranelift_codegen::settings`
reexport in the `wasmtime-environ` crate and then basically played
whack-a-mole until everything compiled again. The main result of this is
that the `wasmtime-*` family of crates have generally less of a
dependency on the `TargetIsa` trait and type from Cranelift. While the
dependency isn't entirely severed yet this is at least a significant
start.

This commit is intended to be largely refactorings, no functional
changes are intended here. The refactorings are:

* A `CompilerBuilder` trait has been added to `wasmtime_environ` which
  server as an abstraction used to create compilers and configure them
  in a uniform fashion. The `wasmtime::Config` type now uses this
  instead of cranelift-specific settings. The `wasmtime-jit` crate
  exports the ability to create a compiler builder from a
  `CompilationStrategy`, which only works for Cranelift right now. In a
  cranelift-less build of Wasmtime this is expected to return a trait
  object that fails all requests to compile.

* The `Compiler` trait in the `wasmtime_environ` crate has been souped
  up with a number of methods that Wasmtime and other crates needed.

* The `wasmtime-debug` crate is now moved entirely behind the
  `wasmtime-cranelift` crate.

* The `wasmtime-cranelift` crate is now only depended on by the
  `wasmtime-jit` crate.

* Wasm types in `cranelift-wasm` no longer contain their IR type,
  instead they only contain the `WasmType`. This is required to get
  everything to align correctly but will also be required in a future
  refactoring where the types used by `cranelift-wasm` will be extracted
  to a separate crate.

* I moved around a fair bit of code in `wasmtime-cranelift`.

* Some gdb-specific jit-specific code has moved from `wasmtime-debug` to
  `wasmtime-jit`.
2021-08-16 09:55:39 -05:00
Alex Crichton
e9f33fc618 Move all trampoline compilation to wasmtime-cranelift (#3176)
* Move all trampoline compilation to `wasmtime-cranelift`

This commit moves compilation of all the trampolines used in wasmtime
behind the `Compiler` trait object to live in `wasmtime-cranelift`. The
long-term goal of this is to enable depending on cranelift *only* from
the `wasmtime-cranelift` crate, so by moving these dependencies we
should make that a little more flexible.

* Fix windows build
2021-08-12 16:58:21 -05:00
Andrew Brown
a7f592a026 Add a crate to interface with the WebAssembly spec interpreter
The WebAssembly spec interpreter is written in OCaml and the new crate
uses `ocaml-interop` along with a small OCaml wrapper to interpret Wasm
modules in-process. The build process for this crate is currently
Linux-specific: it requires several OCaml packages (e.g. `apt install -y
ocaml-nox ocamlbuild`) as well as `make`, `cp`, and `ar`.
2021-08-10 11:56:07 -07:00
Alex Crichton
bb85366a3b Enable simd fuzzing on oss-fuzz (#3152)
* Enable simd fuzzing on oss-fuzz

This commit generally enables the simd feature while fuzzing, which
should affect almost all fuzzers. For fuzzers that just throw random
data at the wall and see what sticks, this means that they'll now be
able to throw simd-shaped data at the wall and have it stick. For
wasm-smith-based fuzzers this commit also updates wasm-smith to 0.6.0
which allows further configuring the `SwarmConfig` after generation,
notably allowing `instantiate-swarm` to generate modules using simd
using `wasm-smith`. This should much more reliably feed simd-related
things into the fuzzers.

Finally, this commit updates wasmtime to avoid usage of the general
`wasm_smith::Module` generator to instead use a Wasmtime-specific custom
default configuration which enables various features we have
implemented.

* Allow dummy table creation to fail

Tables might creation for imports may exceed the memory limit on the
store, which we'll want to gracefully recover from and not fail the
fuzzers.
2021-08-05 16:24:42 -05:00
Alex Crichton
a33caec9be Bump the wasm-tools crates (#3139)
* Bump the wasm-tools crates

Pulls in some updates here and there, mostly for updating crates to the
latest version to prepare for later memory64 work.

* Update lightbeam
2021-08-04 09:53:47 -05:00
Chris Fallin
a13a777230 Bump to Wasmtime v0.29.0 and Cranelift 0.76.0. 2021-08-02 11:24:09 -07:00
Nick Fitzgerald
10eead18c8 Update object to 0.26.0 2021-07-26 12:10:41 -07:00
Nick Fitzgerald
514bbb20b4 Update backtrace to 0.3.61 2021-07-26 12:05:44 -07:00
Nick Fitzgerald
3d76cbdf34 Update gimli to 0.25; addr2line to 0.16 2021-07-26 11:04:53 -07:00
Qiu Wenbo
f628d06118 Upgrade capstone to v0.9 2021-07-19 17:14:28 +08:00
Pat Hickey
4fa4a72328 wiggle: expand test suite
sync test: show the dummy executor will trap (rather than panic) when a
future inside it pends.

async test: show that the executor is hooked up to a future that pends
for a trivial amount of time.

this adds tokio to the dev-dependencies of wiggle, it shouldn't end up
increasing the build burden for the project as a whole since its already
a dev-dependency.
2021-07-16 09:32:43 -07:00
Dan Gohman
6a5a295019 Port wasi-common from unsafe-io to io-lifetimes (#3049)
* Port wasi-common to io-lifetimes.

This ports wasi-common from unsafe-io to io-lifetimes.

Ambient authority is now indicated via calls to `ambient_authority()`
from the ambient-authority crate, rather than using `unsafe` blocks.

The `GetSetFdFlags::set_fd_flags` function is now split into two phases,
to simplify lifetimes in implementations which need to close and re-open
the underlying file.

* Use posish for errno values instead of libc.

This eliminates one of the few remaining direct libc dependencies.

* Port to posish::io::poll.

Use posish::io::poll instead of calling libc directly. This factors out
more code from Wasmtime, and eliminates the need to manipulate raw file
descriptors directly.

And, this eliminates the last remaining direct dependency on libc in
wasi-common.

* Port wasi-c-api to io-lifetimes.

* Update to posish 0.16.0.

* Embeded NULs in filenames now get `EINVAL` instead of `EILSEQ`.

* Accept either `EILSEQ` or `EINVAL` for embedded NULs.

* Bump the nightly toolchain to 2021-07-12.

This fixes build errors on the semver crate, which as of this writing
builds with latest nightly and stable but not 2021-04-11, the old pinned
version.

* Have cap-std-sync re-export ambient_authority so that users get the same version.
2021-07-14 15:39:09 -07:00
Benjamin Bouvier
b0ce0c8f5d Upgrade Tokio to 1.8.1
This contains a fix for a vulnerability reported as RUSTSEC-2021-0072.
2021-07-08 12:13:53 +02:00