Commit Graph

348 Commits

Author SHA1 Message Date
Johnnie Birch
2190fb68fd Update version for fs-set-times to 3.1 2021-03-04 11:36:31 -08:00
Andrew Brown
44e76fe9c0 Update spec tests (#2690)
* Update wasm-tools crates

* Update Wasm SIMD spec tests

* Invert 'experimental_x64_should_panic' logic

By doing this, it is easier to see which spec tests currently panic. The new tests correspond to recently-added instructions.

* Fix: ignore new spec tests for all backends
2021-03-01 16:39:20 -06:00
Alex Crichton
7795a230f2 Implement support for async functions in Wasmtime (#2434)
* Implement support for `async` functions in Wasmtime

This is an implementation of [RFC 2] in Wasmtime which is to support
`async`-defined host functions. At a high level support is added by
executing WebAssembly code that might invoke an asynchronous host
function on a separate native stack. When the host function's future is
not ready we switch back to the main native stack to continue execution.

There's a whole bunch of details in this commit, and it's a bit much to
go over them all here in this commit message. The most important changes
here are:

* A new `wasmtime-fiber` crate has been written to manage the low-level
  details of stack-switching. Unixes use `mmap` to allocate a stack and
  Windows uses the native fibers implementation. We'll surely want to
  refactor this to move stack allocation elsewhere in the future. Fibers
  are intended to be relatively general with a lot of type paremters to
  fling values back and forth across suspension points. The whole crate
  is a giant wad of `unsafe` unfortunately and involves handwritten
  assembly with custom dwarf CFI directives to boot. Definitely deserves
  a close eye in review!

* The `Store` type has two new methods -- `block_on` and `on_fiber`
  which bridge between the async and non-async worlds. Lots of unsafe
  fiddly bits here as we're trying to communicate context pointers
  between disparate portions of the code. Extra eyes and care in review
  is greatly appreciated.

* The APIs for binding `async` functions are unfortunately pretty ugly
  in `Func`. This is mostly due to language limitations and compiler
  bugs (I believe) in Rust. Instead of `Func::wrap` we have a
  `Func::wrapN_async` family of methods, and we've also got a whole
  bunch of `Func::getN_async` methods now too. It may be worth
  rethinking the API of `Func` to try to make the documentation page
  actually grok'able.

This isn't super heavily tested but the various test should suffice for
engaging hopefully nearly all the infrastructure in one form or another.
This is just the start though!

[RFC 2]: https://github.com/bytecodealliance/rfcs/pull/2

* Add wasmtime-fiber to publish script

* Save vector/float registers on ARM too.

* Fix a typo

* Update lock file

* Implement periodically yielding with fuel consumption

This commit implements APIs on `Store` to periodically yield execution
of futures through the consumption of fuel. When fuel runs out a
future's execution is yielded back to the caller, and then upon
resumption fuel is re-injected. The goal of this is to allow cooperative
multi-tasking with futures.

* Fix compile without async

* Save/restore the frame pointer in fiber switching

Turns out this is another caller-saved register!

* Simplify x86_64 fiber asm

Take a leaf out of aarch64's playbook and don't have extra memory to
load/store these arguments, instead leverage how `wasmtime_fiber_switch`
already loads a bunch of data into registers which we can then
immediately start using on a fiber's start without any extra memory
accesses.

* Add x86 support to wasmtime-fiber

* Add ARM32 support to fiber crate

* Make fiber build file probing more flexible

* Use CreateFiberEx on Windows

* Remove a stray no-longer-used trait declaration

* Don't reach into `Caller` internals

* Tweak async fuel to eventually run out.

With fuel it's probably best to not provide any way to inject infinite
fuel.

* Fix some typos

* Cleanup asm a bit

* Use a shared header file to deduplicate some directives
* Guarantee hidden visibility for functions
* Enable gc-sections on macOS x86_64
* Add `.type` annotations for ARM

* Update lock file

* Fix compile error

* Review comments
2021-02-26 16:19:56 -06:00
Nick Fitzgerald
824ce7bf89 deps: Update Arbitrary to 1.0; libfuzzer-sys to 0.4.0; wasm-smith to 0.4.0 2021-02-25 15:34:02 -08:00
Pat Hickey
4559537f24 latest tracing-subscriber fixes performance problem reported in #2662 2021-02-23 09:10:38 -08:00
Andrew Brown
96556ed700 Optionally compile wasmtime-bench-api with wasi-nn and wasi-crypto (#2677)
This adds the ability to add feature flags (e.g. `--features wasi-nn`) when compiling `wasmtime-bench-api` to allow benchmarking Wasmtime with WASI proposals included. Note that due to https://github.com/rust-lang/cargo/issues/5364, these features are only available:
 - in the `crates/bench-api` directory, e.g. `pushd crates/bench-api; cargo build --features wasi-crypto`
 - or from the top-level project directory using `-Zpackage-features`, e.g. `OPENVINO_INSTALL_DIR=/opt/intel/openvino cargo +nightly build -p wasmtime-bench-api -Zpackage-features --features wasi-nn`
2021-02-23 09:18:37 -06:00
Alex Crichton
98d3e6823f Update wasmparser/wat dependencies (#2675)
* Update wasmparser/wat dependencies

Bring in new opcodes and new instructions for SIMD

* Update module linking syntax
2021-02-22 11:56:34 -06:00
Alex Crichton
df9c725fa0 Update to the next version of the witx crate
This commit updates to the 0.9 version of the witx crate implemented in
WebAssembly/wasi#395. This new version drastically changes code
generation and how we interface with the crate. The intention is to
abstract the code generation aspects and allow code generators to
implement much more low-level instructions to enable more flexible APIs
in the future. Additionally a bunch of `*.witx` files were updated in
the WASI repository.

It's worth pointing out, however, that `wasi-common` does not change as
a result of this change. The shape of the APIs that we need to implement
are effectively the same and the only difference is that the shim
functions generated by wiggle are a bit different.
2021-02-18 14:45:20 -08:00
bjorn3
2fc964ea35 Add serde serialization support for the full clif ir 2021-02-18 11:27:02 +01:00
Dan Gohman
8d90ea0390 Bump version to 0.23.0
I used a specially modified version of the publish script to avoid
bumping the `witx` version.
2021-02-17 15:35:43 -08:00
Chris Fallin
e5267f056f Update rand dependency to 0.8.3. 2021-02-16 14:10:05 -08:00
Dan Gohman
d0e5d347b1 Update to rand_core 0.6.2.
See https://rustsec.org/advisories/RUSTSEC-2021-0023
2021-02-16 14:10:05 -08:00
Dan Gohman
b396f6dc6f Switch from memmap to memmap2.
See https://rustsec.org/advisories/RUSTSEC-2020-0077
2021-02-16 14:10:05 -08:00
Dan Gohman
d598883f3a Switch from term to termcolor.
See https://rustsec.org/advisories/RUSTSEC-2018-0015
2021-02-16 14:10:05 -08:00
Pat Hickey
21a1307c10 latest cap-std 0.13.3 fixes windows bug 2021-02-04 17:00:39 -08:00
Pat Hickey
35261453e9 wasi-common 0.13.2 2021-02-04 10:03:34 -08:00
Pat Hickey
72a8f9235d cap-primitives bump fixes windows bugs 2021-02-03 17:35:07 -08:00
Pat Hickey
e670c46862 system-interface 0.6 2021-02-03 16:07:05 -08:00
Pat Hickey
bc1992b9a8 cap-std 0.13 2021-02-03 15:46:03 -08:00
Pat Hickey
31145060b2 remove virtfs - it is not suitable for use 2021-02-03 15:04:02 -08:00
Pat Hickey
47a89d3cc2 cargo update. wanted the latest cap-primitives 2021-02-02 10:44:21 -08:00
Pat Hickey
ca657186f2 fix wasmtime doctest 2021-02-01 19:12:20 -08:00
Pat Hickey
0c4aec391e actually empty ready bytes of stdin 2021-02-01 14:43:32 -08:00
Pat Hickey
a4372c8c2e cap-std-sync: test opening a tempdir as ambient and then WasiDir 2021-02-01 13:15:39 -08:00
Pat Hickey
133344e880 woo it passes six tests even though its awful 2021-01-30 13:54:34 -08:00
Pat Hickey
fcecb3fea6 test-programs: test both cap-std-sync and virtfs backend 2021-01-30 13:39:18 -08:00
Pat Hickey
c12cd82fc0 careful now 2021-01-29 21:19:24 -08:00
Pat Hickey
d8f3a62b32 dont try to make a virtfs today. instead, delete virtfs example. 2021-01-29 17:27:05 -08:00
Pat Hickey
95ad13c82d wasi-common: break dep on system-interface by defining our own Advice enum 2021-01-29 17:03:08 -08:00
Pat Hickey
b48e7fcc5d integrate with c-api 2021-01-29 14:09:44 -08:00
Pat Hickey
d5fdd835ab port bench-api 2021-01-29 13:25:06 -08:00
Pat Hickey
11821e5bfd Merge remote-tracking branch 'origin/main' into pch/wasi_common_cap_std 2021-01-29 12:38:13 -08:00
Pat Hickey
e498888732 hook run command up to both wasi snapshots
sharing the same context! at long last
2021-01-29 12:29:30 -08:00
Alex Crichton
f1d922dc2d Update some wasm-tools dependencies
Mostly focused around some small bugfixes and improvements related to
module-linking fuzzing.
2021-01-29 10:04:36 -08:00
Alex Crichton
0e41861662 Implement limiting WebAssembly execution with fuel (#2611)
* Consume fuel during function execution

This commit adds codegen infrastructure necessary to instrument wasm
code to consume fuel as it executes. Currently nothing is really done
with the fuel, but that'll come in later commits.

The focus of this commit is to implement the codegen infrastructure
necessary to consume fuel and account for fuel consumed correctly.

* Periodically check remaining fuel in wasm JIT code

This commit enables wasm code to periodically check to see if fuel has
run out. When fuel runs out an intrinsic is called which can do what it
needs to do in the result of fuel running out. For now a trap is thrown
to have at least some semantics in synchronous stores, but another
planned use for this feature is for asynchronous stores to periodically
yield back to the host based on fuel running out.

Checks for remaining fuel happen in the same locations as interrupt
checks, which is to say the start of the function as well as loop
headers.

* Improve codegen by caching `*const VMInterrupts`

The location of the shared interrupt value and fuel value is through a
double-indirection on the vmctx (load through the vmctx and then load
through that pointer). The second pointer in this chain, however, never
changes, so we can alter codegen to account for this and remove some
extraneous load instructions and hopefully reduce some register
pressure even maybe.

* Add tests fuel can abort infinite loops

* More fuzzing with fuel

Use fuel to time out modules in addition to time, using fuzz input to
figure out which.

* Update docs on trapping instructions

* Fix doc links

* Fix a fuzz test

* Change setting fuel to adding fuel

* Fix a doc link

* Squelch some rustdoc warnings
2021-01-29 08:57:17 -06:00
Pat Hickey
9bd89abc0c rename everything c2 related to the "real" names 2021-01-28 15:34:03 -08:00
Pat Hickey
220509f3ed latest system-interface 2021-01-27 12:17:42 -08:00
Alex Crichton
7f840870c7 cranelift-native: Use libstd feature detection (#2607)
This commit switches cranelift-native to useing the
`is_x86_feature_detected!` macro in the standard library instead of the
`raw-cpuid` crate.
2021-01-26 16:42:11 -06:00
Chris Fallin
3c5416446c Fix cargo-deny issue with raw-cpuid advisory.
cargo-deny tells us that we should upgrade raw-cpuid to v9.0.0. This
new version also seems to lack the `nightly` feature (perhaps it has
been incorporated into the base functionality) so I had to remove this
feature selector to build.
2021-01-25 08:32:06 -08:00
Frank Denis
a0fad6065a Add support for the experimental wasi-crypto APIs (#2597)
* Add support for the experimental wasi-crypto APIs

The sole purpose of the implementation is to allow bindings and
application developers to test the proposed APIs.

Rust and AssemblyScript bindings are also available as examples.

Like `wasi-nn`, it is currently disabled by default, and requires
the `wasi-crypto` feature flag to be compiled in.

* Rename the wasi-crypto/spec submodule

* Add a path dependency into the submodule for wasi-crypto

* Tell the publish script to vendor wasi-crypto
2021-01-25 09:32:58 -06:00
Pat Hickey
2b70ea8b91 windows scheduler passes tests
unfortunately this does essentially nothing to convince me of its
correctness
2021-01-22 18:26:45 -08:00
Pat Hickey
19254bfd72 error mapping for windows! 2021-01-22 15:44:32 -08:00
Pat Hickey
86d369c72a inline poll from yanix 2021-01-21 18:17:49 -08:00
Pat Hickey
16b42a5707 fix 2021-01-21 14:14:05 -08:00
Pat Hickey
85c3b73b4e stdio compiles on windows! 2021-01-21 13:51:18 -08:00
Alex Crichton
4a351ab7fe Update a number of dependencies (#2594)
This commit goes through the dependencies that wasmtime has and updates
versions where possible. This notably brings in a wasmparser/wast update
which has some simd spec changes with new instructions. Otherwise most
of these are just routine updates.
2021-01-21 15:49:13 -06:00
Pat Hickey
fcd00f5de1 move more deps to cap-std-sync, define own SystemTimeSpec 2021-01-21 12:35:55 -08:00
Andronik Ordian
2d5037d84c cache: update zstd to 0.6, disable legacy feature (#2592) 2021-01-21 10:16:07 -06:00
Pat Hickey
f3e40e2fc4 restructure cap-std impls into their own crate 2021-01-20 19:09:15 -08:00
Pat Hickey
657024bd0d latest system-interface provides ReadReady::num_ready_bytes 2021-01-19 14:39:20 -08:00