453 Commits

Author SHA1 Message Date
Dan Gohman
ec6755512f Remove maintenance badges from the Cargo.toml files. (#6286)
Several of these badges were out of date, with some crates in wide production
use marked as "experimental". Insted of trying to keep them up to date, just
remove them, since they are [no longer displayed on crates.io].

[no longer displayed on crates.io]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-badges-section
2023-04-26 01:33:09 +00:00
Dan Gohman
c59bb8db39 Update several dependencies. (#6171)
This updates to rustix 0.37.13, which contains some features we can use to
implement more features in wasi-common for the wasi-sockets API. This also
pulls in several other updates to avoid having multiple versions of rustix.

This does introduce multiple versions of windows-sys, as the errno and tokio
crates are currently using 0.45 while rustix and other dependencies have
updated to 0.48; PRs updating these are already in flight so this will
hopefully be resolved soon.

It also includes cap-std 1.0.14, which disables the use of `openat2` and
`statx` on Android, fixing a bug where some Android devices crash the
process when those syscalls are executed.
2023-04-20 14:03:49 +00:00
Trevor Elliott
ef7af28ef0 Bump hashbrown to 0.13.2 (#6238) 2023-04-18 23:12:09 +00:00
Alex Crichton
91de5de049 Update wasm-tools crates (#6215)
While bringing in no major updates for Wasmtime I've taken this
opportunity to list myself for `cargo vet` with wildcard audits of this
family of crates. That means I shouldn't need to further add any more
entries in the future for updating these crates and additionally any
other organizations using these audits will automatically be able to
have audits for version that I publish.

While here I also ran `cargo vet prune` which was able to remove a
number of our exemptions.
2023-04-15 00:07:32 +00:00
Brendan Burns
2d34dbef4b Begin implementation of wasi-http (#5929)
* Integrate experimental HTTP into wasmtime.

* Reset Cargo.lock

* Switch to bail!, plumb options partially.

* Implement timeouts.

* Remove generated files & wasm, add Makefile

* Remove generated code textfile

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>

* Extract streams from request/response.

* Fix read for len < buffer length.

* Formatting.

* types impl: swap todos for traps

* streams_impl: idioms, and swap todos for traps

* component impl: idioms, swap all unwraps for traps, swap all todos for traps

* http impl: idiom

* Remove an unnecessary mut.

* Remove an unsupported function.

* Switch to the tokio runtime for the HTTP request.

* Add a rust example.

* Update to latest wit definition

* Remove example code.

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* Fix some warnings.

* bump new deps to latest releases (#3)

* Add tests for wasi-http to test-programs (#2)

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* bump new deps to latest releases

h2 0.3.16
http 0.2.9
mio 0.8.6
openssl 0.10.48
openssl-sys 0.9.83
tokio 1.26.0

---------

Co-authored-by: Brendan Burns <bburns@microsoft.com>

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* wasi-http: fix cargo.toml file and publish script to work together (#4)

unfortunately, the publish script doesn't use a proper toml parser (in
order to not have any dependencies), so the whitespace has to be the
trivial expected case.

then, add wasi-http to the list of crates to publish.

* Update crates/test-programs/build.rs

* Switch to rustls

* Cleanups.

* Merge switch to rustls.

* Formatting

* Remove libssl install

* Fix tests.

* Rename wasi-http -> wasmtime-wasi-http

* prtest:full

Conditionalize TLS on riscv64gc.

* prtest:full

Fix formatting, also disable tls on s390x

* prtest:full

Add a path parameter to wit-bindgen, remove symlink.

* prtest:full

Fix tests for places where SSL isn't supported.

* Update crates/wasi-http/Cargo.toml

---------

Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
Co-authored-by: Pat Hickey <phickey@fastly.com>
Co-authored-by: Pat Hickey <pat@moreproductive.org>
2023-04-05 20:33:03 +00:00
Remo Senekowitsch
7eb8914090 Chaos mode MVP: Skip branch optimization in MachBuffer (#6039)
* fuzz: Add chaos mode control plane

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: Skip branch optimization with chaos mode

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: Rename chaos engine -> control plane

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* chaos mode: refactoring ControlPlane to be passed through the call stack by reference

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Remo Senekowitsch <contact@remsle.dev>

* fuzz: annotate chaos todos

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: cleanup control plane

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: remove control plane from compiler context

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: move control plane into emit state

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: fix remaining compiler errors

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fix tests

* refactor emission state ctrl plane accessors

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* centralize conditional compilation of chaos mode

Also cleanup a few straggling dependencies on cranelift-control
that aren't needed anymore.

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* add cranelift-control to published crates

prtest:full

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* add cranelift-control to public crates

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

---------

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>
Co-authored-by: Remo Senekowitsch <contact@remsle.dev>
2023-04-05 19:28:46 +00:00
wasmtime-publish
bf741955f0 Bump Wasmtime to 9.0.0 (#6143)
Co-authored-by: Wasmtime Publish <wasmtime-publish@users.noreply.github.com>
2023-04-05 17:06:36 +00:00
Kevin Rizzo
3a92aa3d0a winch: Initial integration with wasmtime (#6119)
* Adding in trampoline compiling method for ISA

* Adding support for indirect call to memory address

* Refactoring frame to externalize defined locals, so it removes WASM depedencies in trampoline case

* Adding initial version of trampoline for testing

* Refactoring trampoline to be re-used by other architectures

* Initial wiring for winch with wasmtime

* Add a Wasmtime CLI option to select `winch`

This is effectively an option to select the `Strategy` enumeration.

* Implement `Compiler::compile_function` for Winch

Hook this into the `TargetIsa::compile_function` hook as well. Currently
this doesn't take into account `Tunables`, but that's left as a TODO for
later.

* Filling out Winch append_code method

* Adding back in changes from previous branch

Most of these are a WIP. It's missing trampolines for x64, but a basic
one exists for aarch64. It's missing the handling of arguments that
exist on the stack.

It currently imports `cranelift_wasm::WasmFuncType` since it's what's
passed to the `Compiler` trait. It's a bit awkward to use in the
`winch_codegen` crate since it mostly operates on `wasmparser` types.
I've had to hack in a conversion to get things working. Long term, I'm
not sure it's wise to rely on this type but it seems like it's easier on
the Cranelift side when creating the stub IR.

* Small API changes to make integration easier

* Adding in new FuncEnv, only a stub for now

* Removing unneeded parts of the old PoC, and refactoring trampoline code

* Moving FuncEnv into a separate file

* More comments for trampolines

* Adding in winch integration tests for first pass

* Using new addressing method to fix stack pointer error

* Adding test for stack arguments

* Only run tests on x86 for now, it's more complete for winch

* Add in missing documentation after rebase

* Updating based on feedback in draft PR

* Fixing formatting on doc comment for argv register

* Running formatting

* Lock updates, and turning on winch feature flags during tests

* Updating configuration with comments to no longer gate Strategy enum

* Using the winch-environ FuncEnv, but it required changing the sig

* Proper comment formatting

* Removing wasmtime-winch from dev-dependencies, adding the winch feature makes this not necessary

* Update doc attr to include winch check

* Adding winch feature to doc generation, which seems to fix the feature error in CI

* Add the `component-model` feature to the cargo doc invocation in CI

To match the metadata used by the docs.rs invocation when building docs.

* Add a comment clarifying the usage of `component-model` for docs.rs

* Correctly order wasmtime-winch and winch-environ in the publish script

* Ensure x86 test dependencies are included in cfg(target_arch)

* Further constrain Winch tests to x86_64 _and_ unix

---------

Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Co-authored-by: Saúl Cabrera <saulecabrera@gmail.com>
2023-04-05 00:32:40 +00:00
Saúl Cabrera
80bfb35072 winch: Introduce winch-environ (#6017)
This commit introduces the `winch-environ` crate. This crate's responsibility is
to provide a shared implementatation of the `winch_codegen::FuncEnv` trait,
which is Winch's function compilation environment, used to resolve module and
runtime specific information needed by the code generation, such as resolving
all the details about a callee in a WebAssembly module, or resolving specific
information from the `VMContext`.

As of this change, the implementation only includes the necessary pieces to
resolve a function callee in a WebAssembly module. The idea is to evolve the
`winch_codegen::FuncEnv` trait as we evolve Winch's code generation.
2023-03-14 19:59:15 +00:00
Nick Fitzgerald
9ed441e657 Introduce the wasmtime-explorer crate (#5975)
This implements Godbolt Compiler Explorer-like functionality for Wasmtime and
Cranelift. Given a Wasm module, it compiles the module to native code and then
writes a standalone HTML file that gives a split pane view between the WAT and
ASM disassemblies.
2023-03-11 00:33:06 +00:00
Nick Fitzgerald
5623f7280c Update wasmprinter and wasm-mutate deps (#5983)
* Bump wasm-mutate and wasmprinter deps

* Add wildcard audits for wasmprinter and wasm-mutate

* Add wildcard audit for bumpalo
2023-03-10 20:20:57 +00:00
Kevin Rizzo
013b35ff32 winch: Refactoring wasmtime compiler integration pieces to share more between Cranelift and Winch (#5944)
* Enable the native target by default in winch

Match cranelift-codegen's build script where if no architecture is
explicitly enabled then the host architecture is implicitly enabled.

* Refactor Cranelift's ISA builder to share more with Winch

This commit refactors the `Builder` type to have a type parameter
representing the finished ISA with Cranelift and Winch having their own
typedefs for `Builder` to represent their own builders. The intention is
to use this shared functionality to produce more shared code between the
two codegen backends.

* Moving compiler shared components to a separate crate

* Restore native flag inference in compiler building

This fixes an oversight from the previous commits to use
`cranelift-native` to infer flags for the native host when using default
settings with Wasmtime.

* Move `Compiler::page_size_align` into wasmtime-environ

The `cranelift-codegen` crate doesn't need this and winch wants the same
implementation, so shuffle it around so everyone has access to it.

* Fill out `Compiler::{flags, isa_flags}` for Winch

These are easy enough to plumb through with some shared code for
Wasmtime.

* Plumb the `is_branch_protection_enabled` flag for Winch

Just forwarding an isa-specific setting accessor.

* Moving executable creation to shared compiler crate

* Adding builder back in and removing from shared crate

* Refactoring the shared pieces for the `CompilerBuilder`

I decided to move a couple things around from Alex's initial changes.
Instead of having the shared builder do everything, I went back to
having each compiler have a distinct builder implementation. I
refactored most of the flag setting logic into a single shared location,
so we can still reduce the amount of code duplication.

With them being separate, we don't need to maintain things like
`LinkOpts` which Winch doesn't currently use. We also have an avenue to
error when certain flags are sent to Winch if we don't support them. I'm
hoping this will make things more maintainable as we build out Winch.

I'm still unsure about keeping everything shared in a single crate
(`cranelift_shared`). It's starting to feel like this crate is doing too
much, which makes it difficult to name. There does seem to be a need for
two distinct abstraction: creating the final executable and the handling
of shared/ISA flags when building the compiler. I could make them into
two separate crates, but there doesn't seem to be enough there yet to
justify it.

* Documentation updates, and renaming the finish method

* Adding back in a default temporarily to pass tests, and removing some unused imports

* Fixing winch tests with wrong method name

* Removing unused imports from codegen shared crate

* Apply documentation formatting updates

Co-authored-by: Saúl Cabrera <saulecabrera@gmail.com>

* Adding back in cranelift_native flag inferring

* Adding new shared crate to publish list

* Adding write feature to pass cargo check

---------

Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Co-authored-by: Saúl Cabrera <saulecabrera@gmail.com>
2023-03-08 15:07:13 +00:00
Qinxuan Chen
e2dcb19099 deps: update criterion dependency to v0.4.0 (#5935) 2023-03-07 15:21:48 +00:00
Alex Crichton
3c9fc3ec8c Update wasm-tools crates (#5945)
This notably updates `wasmparser` for updates to the relaxed-simd
proposal and an implementation of the function-references proposal.
Additionally there are some minor bug fixes being picked up for WIT and
the component model.
2023-03-06 23:47:34 +00:00
wasmtime-publish
354bc48015 Bump Wasmtime to 8.0.0 (#5932)
Co-authored-by: Wasmtime Publish <wasmtime-publish@users.noreply.github.com>
2023-03-06 15:08:16 +00:00
Andrew Brown
ad584f428a wasi-threads: run test suite (#5907)
* wasi-threads: run test suite

This change enables the running of the wasi-threads [test suite]. It
relies on a Wasmtime CLI binary being available and runs all `*.wasm`
and `*.wat` files present in the test suite directory. The results of
each execution are compared against a JSON spec file with the same base
name as the WebAssembly module. The spec file defines the expected exit
code, e.g.

This commit does not yet build any `*.c` or `*.s` files from the test
suite. That could be done later, perhaps upstream; in the meantime, this
work is still valuable as it lays the foundation for running other WASI
tests from the in-progress [wasi-testsuite] which share the same JSON
spec infrastructure.

[test suite]: https://github.com/WebAssembly/wasi-threads/tree/main/test/testsuite
[wasi-testsuite]: https://github.com/WebAssembly/wasi-testsuite

* review: move testsuite to top-level tests

* fix: remove now-unnecessary wasi-threads test

* fix: update testsuite submodule name

* fix: ignore tests on Windows

prtest:full

* fix: `cfg_attr` syntax

prtest:full
2023-03-04 21:50:15 +00:00
Sven Sauleau
0e9a48afd5 add basic coredump generation (#5868)
This change adds a basic coredump generation after a WebAssembly trap
was entered. The coredump includes rudimentary stack / process debugging
information.

A new CLI argument is added to enable coredump generation:
```
wasmtime --coredump-on-trap=/path/to/coredump/file module.wasm
```

See ./docs/examples-coredump.md for a working example.

Refs https://github.com/bytecodealliance/wasmtime/issues/5732
2023-02-28 20:27:52 +00:00
Jamey Sharp
539c42e590 Audit object crate update to 0.30.3 (#5827)
This audit is needed for #5619. I'm going ahead and updating Cargo.toml
and Cargo.lock at the same time because no source code changes are
required for this update.
2023-02-18 00:36:21 +00:00
Andrew Brown
c3c16eb207 wasi-threads: build the crate in the CLI application by default (#5782)
This change adds the `wasmtime-wasi-threads` crate as a default crate
for the CLI application. This is no change for embedders of Wasmtime:
they would still have to include `wasmtime-wasi-threads` manually.
Enabling the crate by default in the CLI application has several
benefits, e.g., that it is simpler to experiment with and that it will
be part of more test runs (and thus bugs can be discovered more
quickly). Users will still have to add
`--wasi-modules=experimental-wasi-threads` to enable wasi-threads on the
command line.
2023-02-16 09:37:11 -06:00
Alex Crichton
255fd6be0a Update world-selection in bindgen! macro (#5779)
* Update world-selection in `bindgen!` macro

Inspired by bytecodealliance/wit-bindgen#494 specifying a world or
document to bindgen is now optional as it's inferred if there's only one
`default world` in a package's documents.

* Add cargo-vet entry
2023-02-14 20:54:37 +00:00
Alphyr
cb150d37ce Update dependencies (#5513) 2023-02-14 19:45:15 +00:00
Alex Crichton
49613be393 Update wasm-tools crates (#5757)
* Update wasm-tools crates

Pulls in a new component binary format which should hopefully be the
last update for awhile.

* Update cargo vet configuration
2023-02-10 02:14:53 +00:00
Andrew Brown
edfa10d607 wasi-threads: an initial implementation (#5484)
This commit includes a set of changes that add initial support for `wasi-threads` to Wasmtime:

* feat: remove mutability from the WasiCtx Table

This patch adds interior mutability to the WasiCtx Table and the Table elements.

Major pain points:
* `File` only needs `RwLock<cap_std::fs::File>` to implement
  `File::set_fdflags()` on Windows, because of [1]
* Because `File` needs a `RwLock` and `RwLock*Guard` cannot
  be hold across an `.await`, The `async` from
  `async fn num_ready_bytes(&self)` had to be removed
* Because `File` needs a `RwLock` and `RwLock*Guard` cannot
  be dereferenced in `pollable`, the signature of
  `fn pollable(&self) -> Option<rustix::fd::BorrowedFd>`
  changed to `fn pollable(&self) -> Option<Arc<dyn AsFd + '_>>`

[1] da238e324e/src/fs/fd_flags.rs (L210-L217)

* wasi-threads: add an initial implementation

This change is a first step toward implementing `wasi-threads` in
Wasmtime. We may find that it has some missing pieces, but the core
functionality is there: when `wasi::thread_spawn` is called by a running
WebAssembly module, a function named `wasi_thread_start` is found in the
module's exports and called in a new instance. The shared memory of the
original instance is reused in the new instance.

This new WASI proposal is in its early stages and details are still
being hashed out in the [spec] and [wasi-libc] repositories. Due to its
experimental state, the `wasi-threads` functionality is hidden behind
both a compile-time and runtime flag: one must build with `--features
wasi-threads` but also run the Wasmtime CLI with `--wasm-features
threads` and `--wasi-modules experimental-wasi-threads`. One can
experiment with `wasi-threads` by running:

```console
$ cargo run --features wasi-threads -- \
    --wasm-features threads --wasi-modules experimental-wasi-threads \
    <a threads-enabled module>
```

Threads-enabled Wasm modules are not yet easy to build. Hopefully this
is resolved soon, but in the meantime see the use of
`THREAD_MODEL=posix` in the [wasi-libc] repository for some clues on
what is necessary. Wiggle complicates things by requiring the Wasm
memory to be exported with a certain name and `wasi-threads` also
expects that memory to be imported; this build-time obstacle can be
overcome with the `--import-memory --export-memory` flags only available
in the latest Clang tree. Due to all of this, the included tests are
written directly in WAT--run these with:

```console
$ cargo test --features wasi-threads -p wasmtime-cli -- cli_tests
```

[spec]: https://github.com/WebAssembly/wasi-threads
[wasi-libc]: https://github.com/WebAssembly/wasi-libc

This change does not protect the WASI implementations themselves from
concurrent access. This is already complete in previous commits or left
for future commits in certain cases (e.g., wasi-nn).

* wasi-threads: factor out process exit logic

As is being discussed [elsewhere], either calling `proc_exit` or
trapping in any thread should halt execution of all threads. The
Wasmtime CLI already has logic for adapting a WebAssembly error code to
a code expected in each OS. This change factors out this logic to a new
function, `maybe_exit_on_error`, for use within the `wasi-threads`
implementation.

This will work reasonably well for CLI users of Wasmtime +
`wasi-threads`, but embedders will want something better in the future:
when a `wasi-threads` threads fails, they may not want their application
to exit. Handling this is tricky, because it will require cancelling the
threads spawned by the `wasi-threads` implementation, something that is
not trivial to do in Rust. With this change, we defer that work until
later in order to provide a working implementation of `wasi-threads` for
experimentation.

[elsewhere]: https://github.com/WebAssembly/wasi-threads/pull/17

* review: work around `fd_fdstat_set_flags`

In order to make progress with wasi-threads, this change temporarily
works around limitations induced by `wasi-common`'s
`fd_fdstat_set_flags` to allow `&mut self` use in the implementation.
Eventual resolution is tracked in
https://github.com/bytecodealliance/wasmtime/issues/5643. This change
makes several related helper functions (e.g., `set_fdflags`) take `&mut
self` as well.

* test: use `wait`/`notify` to improve `threads.wat` test

Previously, the test simply executed in a loop for some hardcoded number
of iterations. This changes uses `wait` and `notify` and atomic
operations to keep track of when the spawned threads are done and join
on the main thread appropriately.

* various fixes and tweaks due to the PR review

---------

Signed-off-by: Harald Hoyer <harald@profian.com>
Co-authored-by: Harald Hoyer <harald@profian.com>
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
2023-02-07 13:43:02 -08:00
Pat Hickey
743a40a6c4 Cargo update cap-std family, and audit deps (#5710)
* update cap-std family and its deps, and audit them

* audit base64: append a safe-to-deploy entry

I mistakenly marked it safe-to-run not understanding that safe-to-deploy was required.

* update to fd-lock 3.0.10

eliminates duplicate dep on windows-sys
2023-02-06 10:16:19 -08:00
wasmtime-publish
482f541101 Bump Wasmtime to 7.0.0 (#5712)
Co-authored-by: Wasmtime Publish <wasmtime-publish@users.noreply.github.com>
2023-02-06 09:10:19 -06:00
Alex Crichton
a2a0a9ef5b Update to the latest wit-parser (#5694)
This notably pulls in support in WIT for types-in-worlds.
2023-02-02 19:21:01 +00:00
Jamey Sharp
915801551b Delete old cranelift-preopt crate (#5642)
Most of these optimizations are in the egraph `cprop.isle` rules now,
making a separate crate unnecessary.

Also I think the `udiv` optimizations here are straight-up wrong (doing
signed instead of unsigned division, and panicking instead of preserving
traps on division by zero) so I'm guessing this crate isn't seriously
used anywhere.

At the least, bjorn3 confirms that cg_clif doesn't use this, and I've
verified that Wasmtime doesn't either.

Closes #1090.
2023-01-26 21:32:33 +00:00
Alex Crichton
a7d0d00e57 Update wasm-tools crates (#5631)
Nothing major pulled in here, but wanted to update to the latest
versions which enable tail calls by default. When used in Wasmtime,
however, the feature is disabled without the possibility of being
enabled since it's not implemented.
2023-01-25 16:33:26 +00:00
Kevin Rizzo
da03ff47f1 winch: Adding support for integration tests (#5588)
* Adding in the foundations for Winch `filetests`

This commit adds two new crates into the Winch workspace:
`filetests` and `test-macros`. The intent is to mimic the
structure of Cranelift `filetests`, but in a simpler way.

* Updates to documentation

This commits adds a high level document to outline how to test Winch
through the `winch-tools` utility. It also updates some inline
documentation which gets propagated to the CLI.

* Updating test-macro to use a glob instead of only a flat directory
2023-01-19 07:34:48 -05:00
Dan Gohman
e260abfce7 Update to rustix 0.36.7. (#5590)
This fixes compilation on armv7-unknown-freebsd, as reported [here].

[here]: https://github.com/bytecodealliance/wasmtime/issues/5499#issuecomment-1383157702
2023-01-18 17:15:50 -08:00
Alex Crichton
d24c2fe48c Detect components in wasmtime compile more robustly (#5592)
The binary version of component is going to change over time so use a
more robust method than checking for a fixed 8 bytes.
2023-01-18 18:39:35 -06:00
Alex Crichton
247851234b Update WIT tooling used by Wasmtime (#5565)
* Update WIT tooling used by Wasmtime

This commit updates the WIT tooling, namely the wasm-tools family of
crates, with recent updates. Notably:

* bytecodealliance/wasm-tools#867
* bytecodealliance/wasm-tools#871

This updates index spaces in components and additionally bumps the
minimum required version of the component binary format to be consumed
by Wasmtime (because of the index space changes). Additionally WIT
tooling now fully supports `use`.

Note that WIT tooling doesn't, at this time, fully support packages and
depending on remotely defined WIT packages. Currently WIT still needs to
be vendored in the project. It's hoped that future work with `cargo
component` and possible integration here could make the story about
depending on remotely-defined WIT more ergonomic and streamlined.

* Fix `bindgen!` codegen tests

* Add a test for `use` paths an implement support

* Update to crates.io versions of wasm-tools

* Uncomment codegen tests
2023-01-18 15:37:03 +00:00
Saúl Cabrera
94b51cdb17 winch: Use cranelift-codegen x64 backend for emission. (#5581)
This change substitutes the string based emission mechanism with
cranelift-codegen's x64 backend.

This change _does not_:

* Introduce new functionality in terms of supported instructions.
* Change the semantics of the assembler/macroassembler in terms of the logic to
emit instructions.

The most notable differences between this change and the previous version are:

* Handling of shared flags and ISA-specific flags, which for now are left with
the default value.
* Simplification of instruction emission per operand size: previously the
assembler defined different methods depending on the operand size (e.g. `mov`
for 64 bits, and `movl` for 32 bits). This change updates such approach so that
each assembler method takes an operand size as a parameter, reducing duplication
and making the code more concise and easier to integrate with the x64's `Inst` enum.
* Introduction of a disassembler for testing purposes.

As of this change, Winch generates the following code for the following test
programs:

```wat
(module
  (export "main" (func $main))

  (func $main (result i32)
        (i32.const 10)
        (i32.const 20)
        i32.add
        ))
```

```asm
   0:	55                   	push	rbp
   1:	48 89 e5             	mov	rbp, rsp
   4:	b8 0a 00 00 00       	mov	eax, 0xa
   9:	83 c0 14             	add	eax, 0x14
   c:	5d                   	pop	rbp
   d:	c3                   	ret
```

```wat
(module
  (export "main" (func $main))

  (func $main (result i32)
        (local $foo i32)
    (local $bar i32)
        (i32.const 10)
    (local.set $foo)
        (i32.const 20)
    (local.set $bar)

        (local.get $foo)
        (local.get $bar)
        i32.add
        ))
```

```asm
   0:	55                   	push	rbp
   1:	48 89 e5             	mov	rbp, rsp
   4:	48 83 ec 08          	sub	rsp, 8
   8:	48 c7 04 24 00 00 00 00	mov	qword ptr [rsp], 0
  10:	b8 0a 00 00 00       	mov	eax, 0xa
  15:	89 44 24 04          	mov	dword ptr [rsp + 4], eax
  19:	b8 14 00 00 00       	mov	eax, 0x14
  1e:	89 04 24             	mov	dword ptr [rsp], eax
  21:	8b 04 24             	mov	eax, dword ptr [rsp]
  24:	8b 4c 24 04          	mov	ecx, dword ptr [rsp + 4]
  28:	01 c1                	add	ecx, eax
  2a:	48 89 c8             	mov	rax, rcx
  2d:	48 83 c4 08          	add	rsp, 8
  31:	5d                   	pop	rbp
  32:	c3                   	ret
```

```wat
(module
  (export "main" (func $main))

  (func $main (param i32) (param i32) (result i32)
        (local.get 0)
        (local.get 1)
        i32.add
        ))
```

```asm
   0:	55                   	push	rbp
   1:	48 89 e5             	mov	rbp, rsp
   4:	48 83 ec 08          	sub	rsp, 8
   8:	89 7c 24 04          	mov	dword ptr [rsp + 4], edi
   c:	89 34 24             	mov	dword ptr [rsp], esi
   f:	8b 04 24             	mov	eax, dword ptr [rsp]
  12:	8b 4c 24 04          	mov	ecx, dword ptr [rsp + 4]
  16:	01 c1                	add	ecx, eax
  18:	48 89 c8             	mov	rax, rcx
  1b:	48 83 c4 08          	add	rsp, 8
  1f:	5d                   	pop	rbp
  20:	c3                   	ret
```
2023-01-18 06:58:13 -05:00
wasmtime-publish
7bfbec1b57 Bump Wasmtime to 6.0.0 (#5521)
Co-authored-by: Wasmtime Publish <wasmtime-publish@users.noreply.github.com>
2023-01-05 09:46:01 -06:00
Alex Crichton
d9fdbfd50e Use the sym operator for inline assembly (#5459)
* Use the `sym` operator for inline assembly

Avoids extra `#[no_mangle]` functions and undue symbols being exposed
from Wasmtime. This is a newly stabilized feature in Rust 1.66.0. I've
also added a `rust-version` entry to the `wasmtime` crate to try to head
off possible reports in the future about odd error messages or usage of
unstable features if the rustc version is too old.

* Fix a s390x warning

* Add `rust-version` annotation to Wasmtime crate

As the other main entrypoint for embedders.
2022-12-16 20:12:24 +00:00
Alex Crichton
3861f667a2 Update some wasm-tools crates (#5422)
Notably this pulls in
https://github.com/bytecodealliance/wasm-tools/pull/862 which should fix
some fuzz bugs on oss-fuzz.
2022-12-12 18:34:29 -06:00
Nick Fitzgerald
f2e1eaa847 cranelift-filetest: Add support for Wasm-to-CLIF translation filetests (#5412)
This adds support for `.wat` tests in `cranelift-filetest`. The test runner
translates the WAT to Wasm and then uses `cranelift-wasm` to translate the Wasm
to CLIF.

These tests are always precise output tests. The test expectations can be
updated by running tests with the `CRANELIFT_TEST_BLESS=1` environment variable
set, similar to our compile precise output tests. The test's expected output is
contained in the last comment in the test file.

The tests allow for configuring the kinds of heaps used to implement Wasm linear
memory via TOML in a `;;!` comment at the start of the test.

To get ISA and Cranelift flags parsing available in the filetests crate, I had
to move the `parse_sets_and_triple` helper from the `cranelift-tools` binary
crate to the `cranelift-reader` crate, where I think it logically
fits.

Additionally, I had to make some more bits of `cranelift-wasm`'s dummy
environment `pub` so that I could properly wrap and compose it with the
environment used for the `.wat` tests. I don't think this is a big deal, but if
we eventually want to clean this stuff up, we can probably remove the dummy
environments completely, remove `translate_module`, and fold them into these new
test environments and test runner (since Wasmtime isn't using those things
anyways).
2022-12-12 19:31:29 +00:00
Martin Evans
8f23e5a66f --json option for wasmtime settings command (#5411)
* - Added `--json` flag to settings command
 - Refactored gathering of data into a `Settings` struct which can be used in both human/machine readable paths
 - Split out human readable output to another function, it produces exactly the same result as before

* Outputting JSON by hand formatting it. This approach has the advantage of not needing any extra dependencies (i.e.serde), but is obviously a bit ugly.

* Rewritten JSON serialization to use serde

* Commenting and formatting

* Applied rustfmt

* Reduced version of serde and serde_json to fix cargo vet errors

* Updated cargo.lock to fix cargo vet errors
2022-12-12 09:32:23 -06:00
Chris Fallin
f980defe17 egraph support: rewrite to work in terms of CLIF data structures. (#5382)
* egraph support: rewrite to work in terms of CLIF data structures.

This work rewrites the "egraph"-based optimization framework in
Cranelift to operate on aegraphs (acyclic egraphs) represented in the
CLIF itself rather than as a separate data structure to which and from
which we translate the CLIF.

The basic idea is to add a new kind of value, a "union", that is like an
alias but refers to two other values rather than one.  This allows us to
represent an eclass of enodes (values) as a tree. The union node allows
for a value to have *multiple representations*: either constituent value
could be used, and (in well-formed CLIF produced by correct
optimization rules) they must be equivalent.

Like the old egraph infrastructure, we take advantage of acyclicity and
eager rule application to do optimization in a single pass. Like before,
we integrate GVN (during the optimization pass) and LICM (during
elaboration).

Unlike the old egraph infrastructure, everything stays in the
DataFlowGraph. "Pure" enodes are represented as instructions that have
values attached, but that are not placed into the function layout. When
entering "egraph" form, we remove them from the layout while optimizing.
When leaving "egraph" form, during elaboration, we can place an
instruction back into the layout the first time we elaborate the enode;
if we elaborate it more than once, we clone the instruction.

The implementation performs two passes overall:

- One, a forward pass in RPO (to see defs before uses), that (i) removes
  "pure" instructions from the layout and (ii) optimizes as it goes. As
  before, we eagerly optimize, so we form the entire union of optimized
  forms of a value before we see any uses of that value. This lets us
  rewrite uses to use the most "up-to-date" form of the value and
  canonicalize and optimize that form.

  The eager rewriting and acyclic representation make each other work
  (we could not eagerly rewrite if there were cycles; and acyclicity
  does not miss optimization opportunities only because the first time
  we introduce a value, we immediately produce its "best" form). This
  design choice is also what allows us to avoid the "parent pointers"
  and fixpoint loop of traditional egraphs.

  This forward optimization pass keeps a scoped hashmap to "intern"
  nodes (thus performing GVN), and also interleaves on a per-instruction
  level with alias analysis. The interleaving with alias analysis allows
  alias analysis to see the most optimized form of each address (so it
  can see equivalences), and allows the next value to see any
  equivalences (reuses of loads or stored values) that alias analysis
  uncovers.

- Two, a forward pass in domtree preorder, that "elaborates" pure enodes
  back into the layout, possibly in multiple places if needed. This
  tracks the loop nest and hoists nodes as needed, performing LICM as it
  goes. Note that by doing this in forward order, we avoid the
  "fixpoint" that traditional LICM needs: we hoist a def before its
  uses, so when we place a node, we place it in the right place the
  first time rather than moving later.

This PR replaces the old (a)egraph implementation. It removes both the
cranelift-egraph crate and the logic in cranelift-codegen that uses it.

On `spidermonkey.wasm` running a simple recursive Fibonacci
microbenchmark, this work shows 5.5% compile-time reduction and 7.7%
runtime improvement (speedup).

Most of this implementation was done in (very productive) pair
programming sessions with Jamey Sharp, thus:

Co-authored-by: Jamey Sharp <jsharp@fastly.com>

* Review feedback.

* Review feedback.

* Review feedback.

* Bugfix: cprop rule: `(x + k1) - k2` becomes `x - (k2 - k1)`, not `x - (k1 - k2)`.

Co-authored-by: Jamey Sharp <jsharp@fastly.com>
2022-12-06 14:58:57 -08:00
Alex Crichton
2329ecc341 Add a wasmtime::component::bindgen! macro (#5317)
* Import Wasmtime support from the `wit-bindgen` repo

This commit imports the `wit-bindgen-gen-host-wasmtime-rust` crate from
the `wit-bindgen` repository into the upstream Wasmtime repository. I've
chosen to not import the full history here since the crate is relatively
small and doesn't have a ton of complexity. While the history of the
crate is quite long the current iteration of the crate's history is
relatively short so there's not a ton of import there anyway. The
thinking is that this can now continue to evolve in-tree.

* Refactor `wasmtime-component-macro` a bit

Make room for a `wit_bindgen` macro to slot in.

* Add initial support for a `bindgen` macro

* Add tests for `wasmtime::component::bindgen!`

* Improve error forgetting `async` feature

* Add end-to-end tests for bindgen

* Add an audit of `unicase`

* Add a license to the test-helpers crate

* Add vet entry for `pulldown-cmark`

* Update publish script with new crate

* Try to fix publish script

* Update audits

* Update lock file
2022-12-06 13:06:00 -06:00
Alex Crichton
4933762d81 Add release notes for 3.0.1 and update some versions (#5364)
* Add release notes for 3.0.1

* Update some version directives for crates in Wasmtime

* Mark anything with `publish = false` as version 0.0.0
* Mark the icache coherence crate with the same version as Wasmtime

* Fix manifest directives
2022-12-06 01:26:39 +00:00
wasmtime-publish
a28d4d3c89 Bump Wasmtime to 5.0.0 (#5372)
Co-authored-by: Wasmtime Publish <wasmtime-publish@users.noreply.github.com>
2022-12-05 08:38:57 -06:00
Alex Crichton
86acb9a438 Use workspace inheritance for some more dependencies (#5349)
Deduplicate some dependency directives through `[workspace.dependencies]`
2022-11-29 22:32:56 +00:00
Dan Gohman
d6d3c49972 Update to cap-std 1.0, io-lifetimes 1.0. (#5330)
The main change here is that io-lifetimes 1.0 switches to use the I/O safety
feature in the standard library rather than providing its own copy.

This also updates to windows-sys 0.42.0 and rustix 0.36.
2022-11-28 15:31:18 -08:00
Alex Crichton
b305f251fb Update the wasm-tools family of crates (#5310)
Most of the changes here are the updates to the component model which
includes optional URL fields in imports/exports.
2022-11-21 21:37:16 +00:00
Harald Hoyer
c74706aa59 feat: implement memory.atomic.notify,wait32,wait64 (#5255)
* feat: implement memory.atomic.notify,wait32,wait64

Added the parking_spot crate, which provides the needed registry for the
operations.

Signed-off-by: Harald Hoyer <harald@profian.com>

* fix: change trap message for HeapMisaligned

The threads spec test wants "unaligned atomic"
instead of "misaligned memory access".

Signed-off-by: Harald Hoyer <harald@profian.com>

* tests: add test for atomic wait on non-shared memory

Signed-off-by: Harald Hoyer <harald@profian.com>

* tests: add tests/spec_testsuite/proposals/threads

without pooling and reference types.
Also "shared_memory" is added to the "spectest" interface.

Signed-off-by: Harald Hoyer <harald@profian.com>

* tests: add atomics_notify.wast

checking that notify with 0 waiters returns 0 on shared and non-shared
memory.

Signed-off-by: Harald Hoyer <harald@profian.com>

* tests: add tests for atomic wait on shared memory

- return 2 - timeout for 0
- return 2 - timeout for 1000ns
- return 1 - invalid value

Signed-off-by: Harald Hoyer <harald@profian.com>

* fixup! feat: implement memory.atomic.notify,wait32,wait64

Signed-off-by: Harald Hoyer <harald@profian.com>

* fixup! feat: implement memory.atomic.notify,wait32,wait64

Signed-off-by: Harald Hoyer <harald@profian.com>

Signed-off-by: Harald Hoyer <harald@profian.com>
2022-11-21 18:23:06 +00:00
Andrew Brown
8426904129 bench: benchmark several common WASI scenarios (#5274)
In order to properly understand the impact of providing thread-safe
implmentations of WASI contexts (#5235), we need benchmarks that measure
the current performance of WASI calls using Wiggle. This change adds
several common WASI scenarios as WAT files (see `benches/wasi/*.wat`)
and benchmarks them with `criterion`. Using `criterion`'s `iter_custom`,
the WAT file runs the desired number of benchmark iterations internally
and the total duration of the runs is divided to get the average time
for each loop iteration.

Why WAT? When compiling these benchmarks from Rust to `wasm32-wasi`, the
output files are large, contain other WASI imports than the desired
ones, and overall it is difficult to tell if we are measuring what we
expect. By hand-writing the WAT, it is (slightly) more clear what each
benchmark is doing.
2022-11-15 17:02:35 -08:00
Alex Crichton
0548952319 Update wasm-tools crates (#5248)
No major updates, just keeping up-to-date.
2022-11-10 21:23:20 +00:00
Alex Crichton
3535acbf3b Merge pull request from GHSA-wh6w-3828-g9qf
* Unconditionally use `MemoryImageSlot`

This commit removes the internal branching within the pooling instance
allocator to sometimes use a `MemoryImageSlot` and sometimes now.
Instead this is now unconditionally used in all situations on all
platforms. This fixes an issue where the state of a slot could get
corrupted if modules being instantiated switched from having images to
not having an image or vice versa.

The bulk of this commit is the removal of the `memory-init-cow`
compile-time feature in addition to adding Windows support to the
`cow.rs` file.

* Fix compile on Unix

* Add a stricter assertion for static memory bounds

Double-check that when a memory is allocated the configuration required
is satisfied by the pooling allocator.
2022-11-10 11:34:38 -06:00
wasmtime-publish
08ef518c95 Bump Wasmtime to 4.0.0 (#5209)
Co-authored-by: Wasmtime Publish <wasmtime-publish@users.noreply.github.com>
2022-11-06 13:32:34 -06:00