2583 Commits

Author SHA1 Message Date
Saúl Cabrera
5d833a766e cranelift: Add missing control plane parameter to compile_with_cache (#6289)
It seems that this fell through given that the incremental cache is
behind a cargo feature. I noticed this while building
`cranelift-codegen` via `cargo build --all-features`.

I decided to add a check in CI to hopefully prevent this in the future,
but I'm happy to remove it / update it if there's a better way or another way.
2023-04-26 15:37:35 +00:00
Dan Gohman
ec6755512f Remove maintenance badges from the Cargo.toml files. (#6286)
Several of these badges were out of date, with some crates in wide production
use marked as "experimental". Insted of trying to keep them up to date, just
remove them, since they are [no longer displayed on crates.io].

[no longer displayed on crates.io]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-badges-section
2023-04-26 01:33:09 +00:00
Jamey Sharp
c7b83e8ef9 wasmtime: remove drop(&mut ...) used to silence warnings (#6278)
The `Config` needs to be mutable while building a compiler, but in a
build configuration without a compiler, declaring it as `mut` produces a
warning since nothing else needs that.

I found the existing workaround for this warning confusing, so this PR
removes `mut` from the binding for `config` and instead re-binds the
variable in builds where we call `build_compiler`.
2023-04-25 14:15:53 +00:00
Saúl Cabrera
a1732b2906 winch(fuzz): Initial support for differential fuzzing (#6281)
* winch(fuzz): Initial support for differential fuzzing

This commit introduces initial support for differential fuzzing for Winch. In
order to fuzz winch, this change introduces the `winch` cargo feature. When the
`winch` cargo feature is enabled the differential fuzz target uses `wasmi` as
the differential engine and `wasm-smith` and `single-inst` as the module sources.

The intention behind this change is to have a *local* approach for fuzzing and
verifying programs generated by Winch and to have an initial implementation that
will allow us to eventually enable this change by default. Currently it's not
worth it to enable this change by default given all the filtering that needs to
happen to ensure that the generated modules are supported by Winch.

It's worth noting that the Wasm filtering code will be temporary, until Winch
reaches feature parity in terms of Wasm operators.

* Check build targets with the `winch` feature flag

* Rename fuzz target feature to `fuzz-winch`
2023-04-25 03:56:24 +00:00
Jamey Sharp
bd473dba31 wasmtime-runtime: Re-export backtrace Frame type (#6280)
This crate re-exports the `Backtrace` type at top-level from a nested
module. `Backtrace` in turn has `Frame` in its public API, which is not
re-exported anywhere. This is legal and external users can call methods
on `Frame`, but it doesn't appear in the rustdocs, making it
unnecessarily difficult to figure out how to use this API. Re-exporting
`Frame` fixes that, and also allows naming the type directly if needed.
2023-04-25 01:02:54 +00:00
Alex Crichton
4b703f9dce Fix differential fuzzing when Wasmtime hits an OOM (#6273)
OSS-Fuzz found a case where the `differential` fuzzer was failing and
the underlying cause was that Wasmtime was hitting an OOM while Wasmi
wasn't. This meant that the two modules were producing "different
results" since memories had differing lengths, but this isn't a failure
we're interested in. This commit updates the differential fuzzer to
discard the test case once the Wasmtime half reaches OOM.
2023-04-24 16:00:24 +00:00
Brendan Burns
3e06a0a2bd Minor cleanups. (#6270) 2023-04-24 15:50:53 +00:00
Brendan Burns
43ec4816f1 Make streams owned by request/response that they are tied to. (#6228)
* Make streams owned by request/response that they are tied to.

* Address comments, fix tests.

* Address comment.

* Update crates/wasi-http/src/streams_impl.rs

Co-authored-by: Pat Hickey <pat@moreproductive.org>

* Switch to BytesMut

---------

Co-authored-by: Pat Hickey <pat@moreproductive.org>
2023-04-21 18:17:06 +00:00
Jamey Sharp
efdfc361f8 Allow WASI to open directories without O_DIRECTORY (#6163)
* Allow WASI to open directories without O_DIRECTORY

The `O_DIRECTORY` flag is a request that open should fail if the named
path is not a directory. Opening a path which turns out to be a
directory is not supposed to fail if this flag is not specified.
However, wasi-common required callers to use it when opening
directories.

With this PR, we always open the path the same way whether or not the
`O_DIRECTORY` flag is specified. However, after opening it, we `stat` it
to check whether it turned out to be a directory, and determine which
operations the file descriptor should support accordingly. In addition,
we explicitly check whether the precondition defined by `O_DIRECTORY` is
satisfied.

Closes #4947 and closes #4967, which were earlier attempts at fixing the
same issue, but which had race conditions.

prtest:full

* Add tests from #4967/#4947

This test was authored by Roman Volosatovs <rvolosatovs@riseup.net> as
part of #4947.

* Tests: Close FDs before trying to unlink files

On Windows, when opening a path which might be a directory using
`CreateFile`, cap-primitives also removes the `FILE_SHARE_DELETE` mode.

That means that if we implement WASI's `path_open` such that it always
uses `CreateFile` on Windows, for both files and directories, then
holding an open file handle prevents deletion of that file.

So I'm changing these test programs to make sure they've closed the
handle before trying to delete the file.
2023-04-21 16:55:35 +00:00
Dan Gohman
c59bb8db39 Update several dependencies. (#6171)
This updates to rustix 0.37.13, which contains some features we can use to
implement more features in wasi-common for the wasi-sockets API. This also
pulls in several other updates to avoid having multiple versions of rustix.

This does introduce multiple versions of windows-sys, as the errno and tokio
crates are currently using 0.45 while rustix and other dependencies have
updated to 0.48; PRs updating these are already in flight so this will
hopefully be resolved soon.

It also includes cap-std 1.0.14, which disables the use of `openat2` and
`statx` on Android, fixing a bug where some Android devices crash the
process when those syscalls are executed.
2023-04-20 14:03:49 +00:00
Pat Hickey
8e76ec82be wasi-common: export StringArray type (#6244)
reusable elsewhere: @mtr is gonna use it in wasi-nn.
2023-04-19 20:13:25 +00:00
Alex Crichton
077d4755d9 Mark loads from VMCallerCheckedAnyfunc as readonly (#6243)
This commit marks the loads of `*mut VMContext` and the callee function
pointer as `readonly` in the context of indirect function calls and
additionally calls to imported functions (which are indirect). Once a
`VMCallerCheckedAnyfunc` is initialized it's never modified so it should
be valid to mark these as readonly and if called in a loop should be
hoistable outside of the loop.
2023-04-19 19:22:25 +00:00
Pat Hickey
0a062369a8 delete wasi truncation_rights test (#6236)
This test was not meaningfully executing, because wasi-common never
provides rights containing RIGHTS_PATH_FILESTAT_SET_SIZE - this flag is
not even defined in wasi-common/srd/dir.rs as one of the DirCaps flags.

Even when you get rid of that guard that skips the meat of the test,
path_open was being called with OFLAGS_TRUNC but without
RIGHTS_FD_WRITE, which boils down to an `open(2)` with OFLAGS_TRUNC set
and none of the access modes set, so it will always fail with EINVAL.

So, it doesn't look like this test ever would have meaningfully passed,
even in pre-wiggle-rewrite version of wasi-common it landed in back in
late 2019. Late 2019! before the pandemic! our eyes were so full of
stars and dreams of the future!

The behavior we really care about for truncation are taken care of
by the fd_filestat_set test, which shows fd_filestat_set_size works
correctly, and the file_truncation test, which shows that opening
with OFLAGS_TRUNC will truncate the file.
2023-04-19 16:40:42 +00:00
Lann
51ed20ab4d Add Engine::precompile_compatibility_hash (#5826)
This method returns a Hash, the output of which can be used to index
precompiled binaries from one Engine instance that can be deserialized
by another Engine instance.
2023-04-18 21:21:16 +00:00
Pat Hickey
24b607cf75 wasi-common: change behavior of path_readlink to truncate on too-small buffers (#6225)
this is the same behavior as exists in posix readlink(2)
2023-04-18 18:12:30 +00:00
Alex Crichton
299131ae2d Gracefully handle ENOTSUP for memfd_create (#6230)
* Gracefully handle ENOTSUP for `memfd_create`

Looks like in #6229 older kernels may not support memfd in which case we
can fall back to manual memory initialization gracefully.

Closes #6229

* Fix tests
2023-04-18 16:50:48 +00:00
Trevor Elliott
d4b771e5a3 Allow inline source and the path attribute in the host bindgen! macro (#6226)
* Allow inline source and the `path` attribute in the host bindgen! macro

* Report an error if `world` is passed with `interfaces`
2023-04-17 23:51:52 +00:00
Pat Hickey
9ee613a0b7 wasi-common: deprecate fd_allocate (#6217)
* wasi-common: remove allocate from WasiFile trait, always fail with NOTSUP

This operation from cloudabi is linux-specific, isn't even
supported across all linux filesystems, and has no support on macos
or windows. Rather than ship spotty support, it has been removed
from preview 2, and we are no longer supporting it in preview 1 as
well.

The preview 1 implementation will still check if fd is a file, and has
rights, just to reject those cases with the errors expected.

* wasi-tests: expect fd_allocate to always fail now.

rewrite the file_allocate test to just check for failure.

remove use of fd_allocate from fd_advise test, and remove test
configuration setting used for excluding use of fd_allocate on macos and
windows.
2023-04-17 21:35:28 +00:00
kevaundray
85118c8c26 Add clippy suggestions (#6203)
* add clippy suggestions

* revert &/ref change

* Update cranelift/isle/isle/src/parser.rs

Co-authored-by: Jamey Sharp <jamey@minilop.net>

---------

Co-authored-by: Jamey Sharp <jamey@minilop.net>
2023-04-17 15:53:34 +00:00
Alex Crichton
91de5de049 Update wasm-tools crates (#6215)
While bringing in no major updates for Wasmtime I've taken this
opportunity to list myself for `cargo vet` with wildcard audits of this
family of crates. That means I shouldn't need to further add any more
entries in the future for updating these crates and additionally any
other organizations using these audits will automatically be able to
have audits for version that I publish.

While here I also ran `cargo vet prune` which was able to remove a
number of our exemptions.
2023-04-15 00:07:32 +00:00
Alex Crichton
2d25db047f x64: Lower SIMD requirement to SSE4.1 from SSE4.2 (#6206)
Cranelift only has one instruction SIMD which depends on SSE4.2 so this
commit adds a lowering rule for `pcmpgtq` which doesn't use SSE4.2 and
enables lowering the baseline requirement for SIMD support from SSE4.2
to SSE4.1.

The `has_sse42` setting is no longer enabled by default for Cranelift.
Additionally `enable_simd` no longer requires `has_sse42` on x64.
Finally the fuzz-generator for Wasmtime codegen settings now enables
flipping the `has_sse42` setting instead of unconditionally setting it
to `true`.

The specific lowering for `pcmpgtq` is copied from LLVM's lowering of
this instruction.
2023-04-14 17:24:43 +00:00
matt rice
26f9ce02bc For clarity in examples avoid importing/exporting functions of the same name. (#6211) 2023-04-14 15:32:05 +00:00
Pat Hickey
cc1c14ac91 wasmtime component bindgen: when tracing is enabled, emit an event for args & results (#6209)
* wasmtime component bindgen: when tracing is enabled, emit an event for arguments and results

This is consistient with what wiggle does (see
https://github.com/bytecodealliance/wasmtime/blob/main/crates/wiggle/generate/src/funcs.rs#L266), with the exceptions that
1. wiggle has a facility for disabling tracing on a per-function basis,
a requirement which was driven by functions which pass secrets into wasm. this will be added to wasmtime-wit-bindgen at a later date.
2. wiggle doesn't actually emit an event when calling a function which
takes no arguments (see `&& func.params.len() > 0` in predicate), in
this case we emit an event with the body `"call"`, to ensure these calls
are observable.

* review feedback: add call and return messages to events

* consistiency: dont drop `guest` from `wit-bindgen guest export` in span
2023-04-13 22:58:53 +00:00
Trevor Elliott
9425a252bb Rework only_interfaces to the interfaces field (#6210)
* Rework `only_interfaces` to the `interfaces` field

* Fix the docs

* Remove only_interfaces test from the component-macro package
2023-04-13 22:02:56 +00:00
Brendan Burns
7b5819b55a Small follow up fix for http tests. (#6207) 2023-04-13 14:37:49 +00:00
T0b1-iOS
3956a6aa0f remove unsigned_add_overflow_condition (#6199) 2023-04-13 14:30:44 +00:00
Pat Hickey
d1d381e298 add a wasi-test to show a dir_fd always gets ERRNO_BADF on appropriate fd_ operations (#6197)
* add a wasi-test to show a dir_fd always gets ERRNO_BADF on appropriate fd_ operations.

This is a conformance test for the current behavior of preview 1 in wasi-common. It is debatable whether this is the right errno, I think for most of these ERRNO_ISDIR would be more descriptive, but this is the behavior we have.

* Add comments to all the fd op failures explaining closest linux/posix behavior
2023-04-12 16:57:53 +00:00
Brendan Burns
3ff6e0fe03 Some improvements to the wasi-http client implementation of write. (#6161)
* Improve write implementation for streams

* Add trailers implementation for responses.

* Improve tests.

* Update tests.
2023-04-11 03:26:19 +00:00
Trevor Elliott
85f0c68008 Add only_interfaces and with to the bindgen! macro. (#6160)
* Add `only_interfaces` and `with` to the `bindgen!` macro.

* Add a version of the empty_error test for `only_interfaces` and `with`

* Review feedback

* Add docs
2023-04-10 23:28:52 +00:00
Alex Crichton
dbd000c1ce Change asm to __asm__ in helpers.c (#6188)
This is an attempt to fix #6177 since according to [this reference][1]
some modes of compilation require `__asm__` instead of `asm`.

[1]: https://en.cppreference.com/w/c/language/asm
2023-04-10 17:47:25 +00:00
Alex Crichton
83a8ca77cd Mirror default-owned change for guests in wit-bindgen (#6189)
This commit is a mirror of bytecodealliance/wit-bindgen#547 into the
`bindgen!` macro for Wasmtime. The new default is to generate only one
Rust type per WIT type input, regardless of if the representation can be
slightly more optimal in niche cases with more borrows. This should make
the macro easier to work with in the limit ideally.

Closes #6124
2023-04-10 16:27:13 +00:00
Dan Gohman
e84305b445 Add a WASI test for a creating an absolute-path symlink. (#6166)
Wasmtime disallows guests from using `path_symlink` to create
absolute-path symlinks, as they could confuse other code into
accessing resources on the host that the guest otherwise doesn't
have access to.

This patch adds a test for this behavior.
2023-04-08 06:13:50 +00:00
Chris Fallin
230e2135d6 Cranelift: remove non-egraphs optimization pipeline and use_egraphs option. (#6167)
* Cranelift: remove non-egraphs optimization pipeline and `use_egraphs` option.

This PR removes the LICM, GVN, and preopt passes, and associated support
pieces, from `cranelift-codegen`. Not to worry, we still have
optimizations: the egraph framework subsumes all of these, and has been
on by default since #5181.

A few decision points:

- Filetests for the legacy LICM, GVN and simple_preopt were removed too.
  As we built optimizations in the egraph framework we wrote new tests
  for the equivalent functionality, and many of the old tests were
  testing specific behaviors in the old implementations that may not be
  relevant anymore. However if folks prefer I could take a different
  approach here and try to port over all of the tests.

- The corresponding filetest modes (commands) were deleted too. The
  `test alias_analysis` mode remains, but no longer invokes a separate
  GVN first (since there is no separate GVN that will not also do alias
  analysis) so the tests were tweaked slightly to work with that. The
  egrpah testsuite also covers alias analysis.

- The `divconst_magic_numbers` module is removed since it's unused
  without `simple_preopt`, though this is the one remaining optimization
  we still need to build in the egraphs framework, pending #5908. The
  magic numbers will live forever in git history so removing this in the
  meantime is not a major issue IMHO.

- The `use_egraphs` setting itself was removed at both the Cranelift and
  Wasmtime levels. It has been marked deprecated for a few releases now
  (Wasmtime 6.0, 7.0, upcoming 8.0, and corresponding Cranelift
  versions) so I think this is probably OK. As an alternative if anyone
  feels strongly, we could leave the setting and make it a no-op.

* Update test outputs for remaining test differences.
2023-04-06 18:11:03 +00:00
Andrew Brown
5ba0d696b7 ci: unpin the wasi-nn tasks from an older Ubuntu (#6089)
* ci: unpin the wasi-nn tasks from an older Ubuntu

Previously, OpenVINO's lack of APT packages for Ubuntu 22.04 (`jammy`)
prevented us from upgrading the GitHub runner to use `ubuntu-latest`. I
updated the `install-openvino-action` to substitute in the `focal`
packages in this case (this is what the OpenVINO team considers the fix)
so this pin should no longer be necessary. Fixes #5408.

(Run all CI actions: prtest:full)

* vet: audit the openvino version bump
2023-04-06 15:44:16 +00:00
Brendan Burns
2d34dbef4b Begin implementation of wasi-http (#5929)
* Integrate experimental HTTP into wasmtime.

* Reset Cargo.lock

* Switch to bail!, plumb options partially.

* Implement timeouts.

* Remove generated files & wasm, add Makefile

* Remove generated code textfile

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>

* Extract streams from request/response.

* Fix read for len < buffer length.

* Formatting.

* types impl: swap todos for traps

* streams_impl: idioms, and swap todos for traps

* component impl: idioms, swap all unwraps for traps, swap all todos for traps

* http impl: idiom

* Remove an unnecessary mut.

* Remove an unsupported function.

* Switch to the tokio runtime for the HTTP request.

* Add a rust example.

* Update to latest wit definition

* Remove example code.

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* Fix some warnings.

* bump new deps to latest releases (#3)

* Add tests for wasi-http to test-programs (#2)

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* bump new deps to latest releases

h2 0.3.16
http 0.2.9
mio 0.8.6
openssl 0.10.48
openssl-sys 0.9.83
tokio 1.26.0

---------

Co-authored-by: Brendan Burns <bburns@microsoft.com>

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* wasi-http: fix cargo.toml file and publish script to work together (#4)

unfortunately, the publish script doesn't use a proper toml parser (in
order to not have any dependencies), so the whitespace has to be the
trivial expected case.

then, add wasi-http to the list of crates to publish.

* Update crates/test-programs/build.rs

* Switch to rustls

* Cleanups.

* Merge switch to rustls.

* Formatting

* Remove libssl install

* Fix tests.

* Rename wasi-http -> wasmtime-wasi-http

* prtest:full

Conditionalize TLS on riscv64gc.

* prtest:full

Fix formatting, also disable tls on s390x

* prtest:full

Add a path parameter to wit-bindgen, remove symlink.

* prtest:full

Fix tests for places where SSL isn't supported.

* Update crates/wasi-http/Cargo.toml

---------

Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
Co-authored-by: Pat Hickey <phickey@fastly.com>
Co-authored-by: Pat Hickey <pat@moreproductive.org>
2023-04-05 20:33:03 +00:00
Remo Senekowitsch
7eb8914090 Chaos mode MVP: Skip branch optimization in MachBuffer (#6039)
* fuzz: Add chaos mode control plane

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: Skip branch optimization with chaos mode

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: Rename chaos engine -> control plane

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* chaos mode: refactoring ControlPlane to be passed through the call stack by reference

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Remo Senekowitsch <contact@remsle.dev>

* fuzz: annotate chaos todos

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: cleanup control plane

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: remove control plane from compiler context

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: move control plane into emit state

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fuzz: fix remaining compiler errors

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* fix tests

* refactor emission state ctrl plane accessors

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* centralize conditional compilation of chaos mode

Also cleanup a few straggling dependencies on cranelift-control
that aren't needed anymore.

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* add cranelift-control to published crates

prtest:full

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

* add cranelift-control to public crates

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>

---------

Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com>
Co-authored-by: Moritz Waser <mzrw.dev@pm.me>
Co-authored-by: Remo Senekowitsch <contact@remsle.dev>
2023-04-05 19:28:46 +00:00
Alex Crichton
52e90532e0 Add a limits and trap-on-OOM options to the CLI (#6149)
* Add a limits and trap-on-OOM options to the CLI

This commit adds new options to the `wasmtime` CLI to control the
`Store::limiter` behavior at runtime. This enables artificially
restriction the memory usage of the wasm instance, for example.
Additionally a new option is added to `StoreLimits` to force a trap on
growth failure. This is intended to help quickly debug modules with
backtraces if OOM is happening, or even diagnosing if OOM is happening
in the first place.

* Fix compile of fuzzing oracle
2023-04-05 17:26:36 +00:00
wasmtime-publish
bf741955f0 Bump Wasmtime to 9.0.0 (#6143)
Co-authored-by: Wasmtime Publish <wasmtime-publish@users.noreply.github.com>
2023-04-05 17:06:36 +00:00
Alex Crichton
3275c45993 Use an unaligned write for resolving libcall relocations (#6147)
This commit changes resolution of libcall relocations from writing a
`usize` into a raw pointer to specifically performing an unaligned
write. The addresses of libcalls to write to are not guaranteed to be
aligned, so this could technically have caused issues on some platforms
perhaps.

This was discovered now that Rust nightly will panic on unaligned writes
to pointers, and fuzzing ran into this case when compiled with a more
recent Nightly build.
2023-04-05 15:28:18 +00:00
Kevin Rizzo
3a92aa3d0a winch: Initial integration with wasmtime (#6119)
* Adding in trampoline compiling method for ISA

* Adding support for indirect call to memory address

* Refactoring frame to externalize defined locals, so it removes WASM depedencies in trampoline case

* Adding initial version of trampoline for testing

* Refactoring trampoline to be re-used by other architectures

* Initial wiring for winch with wasmtime

* Add a Wasmtime CLI option to select `winch`

This is effectively an option to select the `Strategy` enumeration.

* Implement `Compiler::compile_function` for Winch

Hook this into the `TargetIsa::compile_function` hook as well. Currently
this doesn't take into account `Tunables`, but that's left as a TODO for
later.

* Filling out Winch append_code method

* Adding back in changes from previous branch

Most of these are a WIP. It's missing trampolines for x64, but a basic
one exists for aarch64. It's missing the handling of arguments that
exist on the stack.

It currently imports `cranelift_wasm::WasmFuncType` since it's what's
passed to the `Compiler` trait. It's a bit awkward to use in the
`winch_codegen` crate since it mostly operates on `wasmparser` types.
I've had to hack in a conversion to get things working. Long term, I'm
not sure it's wise to rely on this type but it seems like it's easier on
the Cranelift side when creating the stub IR.

* Small API changes to make integration easier

* Adding in new FuncEnv, only a stub for now

* Removing unneeded parts of the old PoC, and refactoring trampoline code

* Moving FuncEnv into a separate file

* More comments for trampolines

* Adding in winch integration tests for first pass

* Using new addressing method to fix stack pointer error

* Adding test for stack arguments

* Only run tests on x86 for now, it's more complete for winch

* Add in missing documentation after rebase

* Updating based on feedback in draft PR

* Fixing formatting on doc comment for argv register

* Running formatting

* Lock updates, and turning on winch feature flags during tests

* Updating configuration with comments to no longer gate Strategy enum

* Using the winch-environ FuncEnv, but it required changing the sig

* Proper comment formatting

* Removing wasmtime-winch from dev-dependencies, adding the winch feature makes this not necessary

* Update doc attr to include winch check

* Adding winch feature to doc generation, which seems to fix the feature error in CI

* Add the `component-model` feature to the cargo doc invocation in CI

To match the metadata used by the docs.rs invocation when building docs.

* Add a comment clarifying the usage of `component-model` for docs.rs

* Correctly order wasmtime-winch and winch-environ in the publish script

* Ensure x86 test dependencies are included in cfg(target_arch)

* Further constrain Winch tests to x86_64 _and_ unix

---------

Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Co-authored-by: Saúl Cabrera <saulecabrera@gmail.com>
2023-04-05 00:32:40 +00:00
Jimmy Bourassa
c35c047fc3 Derive Copy on wasmtime::ValType (#6138)
`ValType` is small enough that it can be copiable. Doing so means
Copy types can embed a `ValType` in them.

The need for this came up in this Wasmtime Ruby PR:
https://github.com/bytecodealliance/wasmtime-rb/pull/158
2023-04-04 16:49:00 +00:00
Roman Volosatovs
c10e804617 feat(wit-bindgen): relax Sized bound on Host traits (#6117)
This allows to use implementations as trait objects
(e.g. `Box<dyn myiface::Host>`)

Signed-off-by: Roman Volosatovs <roman@profian.com>
2023-03-29 18:30:54 +00:00
Peter Huene
73f42bf817 Fix export translation for components. (#6108)
* Fix export translation for components.

Exports in the component model cause a new index to be added to the index space
of the item being exported.

This commit updates component translation so that translation of component
export sections properly updates internal lists representing those index
spaces.

* Code review feedback.
2023-03-28 00:18:48 +00:00
Saúl Cabrera
a6925c21c5 wasmtime: Make StoreContextMut accessible in epoch deadline callback (#6075)
This commit changes the signature of the `Store::epoch_deadline_callback` to
take in `StoreContextMut` instead of a mutable reference to the store's data.

This is useful in cases in which the callback definition needs access to the
Store to be able to use other methods that take in `AsContext`/`AsContextMut`,
like for example `WasmtimeBacktrace::capture`
2023-03-23 14:39:36 +00:00
Benjamin Bouvier
6f4f30c840 Add support for generating perf maps for simple perf profiling (#6030)
* Add support for generating perf maps for simple perf profiling

* add missing enum entry in C code

* bugfix: use hexa when printing the code region's length too (thanks bjorn3!)

* sanitize file name + use bufwriter

* introduce --profile CLI flag for wasmtime

* Update doc and doc comments for new --profile option

* remove redundant FromStr import

* Apply review feedback: make_line receives a Write impl, report errors

* fix tests?

* better docs
2023-03-20 16:17:36 +00:00
Lann
cd1b19a289 Remove reference to store in Linker::instantiate_pre (#6057)
The `store` param was removed in #5683
2023-03-17 20:27:04 +00:00
Nick Fitzgerald
2e48babf23 cranelift-wasm: Add a bounds-checking optimization for dynamic memories and guard pages (#6031)
* cranelift-wasm: Add a bounds-checking optimization for dynamic memories and guard pages

This is a new special case for when we know that there are enough guard pages to
cover the memory access's offset and access size.

The precise should-we-trap condition is

    index + offset + access_size > bound

However, if we instead check only the partial condition

    index > bound

then the most out of bounds that the access can be, while that partial check
still succeeds, is `offset + access_size`.

However, when we have a guard region that is at least as large as `offset +
access_size`, we can rely on the virtual memory subsystem handling these
out-of-bounds errors at runtime. Therefore, the partial `index > bound` check is
sufficient for this heap configuration.

Additionally, this has the advantage that a series of Wasm loads that use the
same dynamic index operand but different static offset immediates -- which is a
common code pattern when accessing multiple fields in the same struct that is in
linear memory -- will all emit the same `index > bound` check, which we can GVN.

* cranelift: Add WAT tests for accessing dynamic memories with the same index but different offsets

The bounds check comparison is GVN'd but we still branch on values we should
know will always be true if we get this far in the code. This is actual `br_if`s
in the non-Spectre code and `select_spectre_guard`s that we should know will
always go a certain way if we have Spectre mitigations enabled.

Improving the non-Spectre case is pretty straightforward: walk the dominator
tree and remember which values we've already branched on at this point, and
therefore we can simplify any further conditional branches on those same values
into direct jumps.

Improving the Spectre case requires something that is morally the same, but has
a few snags:

* We don't have actual `br_if`s to determine whether the bounds checking
  condition succeeded or not. We need to instead reason about dominating
  `select_spectre_guard; {load, store}` instruction pairs.

* We have to be SUPER careful about reasoning "through" `select_spectre_guard`s.
  Our general rule is never to do that, since it could break the speculative
  execution sandboxing that the instruction is designed for.
2023-03-17 19:06:19 +00:00
Alex Crichton
28371bfd40 Validate faulting addresses are valid to fault on (#6028)
* Validate faulting addresses are valid to fault on

This commit adds a defense-in-depth measure to Wasmtime which is
intended to mitigate the impact of CVEs such as GHSA-ff4p-7xrq-q5r8.
Currently Wasmtime will catch `SIGSEGV` signals for WebAssembly code so
long as the instruction which faulted is an allow-listed instruction
(aka has a trap code listed for it). With the recent security issue,
however, the problem was that a wasm guest could exploit a compiler bug
to access memory outside of its sandbox. If the access was successful
there's no real way to detect that, but if the access was unsuccessful
then Wasmtime would happily swallow the `SIGSEGV` and report a nominal
trap. To embedders, this might look like nothing is going awry.

The new strategy implemented here in this commit is to attempt to be
more robust towards these sorts of failures. When a `SIGSEGV` is raised
the faulting pc is recorded but additionally the address of the
inaccessible location is also record. After the WebAssembly stack is
unwound and control returns to Wasmtime which has access to a `Store`
Wasmtime will now use this inaccessible faulting address to translate it
to a wasm address. This process should be guaranteed to succeed as
WebAssembly should only be able to access a well-defined region of
memory for all linear memories in a `Store`.

If no linear memory in a `Store` could contain the faulting address,
then Wasmtime now prints a scary message and aborts the process. The
purpose of this is to catch these sorts of bugs, make them very loud
errors, and hopefully mitigate impact. This would continue to not
mitigate the impact of a guest successfully loading data outside of its
sandbox, but if a guest was doing a sort of probing strategy trying to
find valid addresses then any invalid access would turn into a process
crash which would immediately be noticed by embedders.

While I was here I went ahead and additionally took a stab at #3120.
Traps due to `SIGSEGV` will now report the size of linear memory and the
address that was being accessed in addition to the bland "access out of
bounds" error. While this is still somewhat bland in the context of a
high level source language it's hopefully at least a little bit more
actionable for some. I'll note though that this isn't a guaranteed
contextual message since only the default configuration for Wasmtime
generates `SIGSEGV` on out-of-bounds memory accesses. Dynamically
bounds-checked configurations, for example, don't do this.

Testing-wise I unfortunately am not aware of a great way to test this.
The closet equivalent would be something like an `unsafe` method
`Config::allow_wasm_sandbox_escape`. In lieu of adding tests, though, I
can confirm that during development the crashing messages works just
fine as it took awhile on macOS to figure out where the faulting address
was recorded in the exception information which meant I had lots of
instances of recording an address of a trap not accessible from wasm.

* Fix tests

* Review comments

* Fix compile after refactor

* Fix compile on macOS

* Fix trap test for s390x

s390x rounds faulting addresses to 4k boundaries.
2023-03-17 14:52:54 +00:00
Alex Crichton
5ae8575296 x64: Take SIGFPE signals for divide traps (#6026)
* x64: Take SIGFPE signals for divide traps

Prior to this commit Wasmtime would configure `avoid_div_traps=true`
unconditionally for Cranelift. This, for the division-based
instructions, would change emitted code to explicitly trap on trap
conditions instead of letting the `div` x86 instruction trap.

There's no specific reason for Wasmtime, however, to specifically avoid
traps in the `div` instruction. This means that the extra generated
branches on x86 aren't necessary since the `div` and `idiv` instructions
already trap for similar conditions as wasm requires.

This commit instead disables the `avoid_div_traps` setting for
Wasmtime's usage of Cranelift. Subsequently the codegen rules were
updated slightly:

* When `avoid_div_traps=true`, traps are no longer emitted for `div`
  instructions.
* The `udiv`/`urem` instructions now list their trap as divide-by-zero
  instead of integer overflow.
* The lowering for `sdiv` was updated to still explicitly check for zero
  but the integer overflow case is deferred to the instruction itself.
* The lowering of `srem` no longer checks for zero and the listed trap
  for the `div` instruction is a divide-by-zero.

This means that the codegen for `udiv` and `urem` no longer have any
branches. The codegen for `sdiv` removes one branch but keeps the
zero-check to differentiate the two kinds of traps. The codegen for
`srem` removes one branch but keeps the -1 check since the semantics of
`srem` mismatch with the semantics of `idiv` with a -1 divisor
(specifically for INT_MIN).

This is unlikely to have really all that much of a speedup but was
something I noticed during #6008 which seemed like it'd be good to clean
up. Plus Wasmtime's signal handling was already set up to catch
`SIGFPE`, it was just never firing.

* Remove the `avoid_div_traps` cranelift setting

With no known users currently removing this should be possible and helps
simplify the x64 backend.

* x64: GC more support for avoid_div_traps

Remove the `validate_sdiv_divisor*` pseudo-instructions and clean up
some of the ISLE rules now that `div` is allowed to itself trap
unconditionally.

* x64: Store div trap code in instruction itself

* Keep divisors in registers, not in memory

Don't accidentally fold multiple traps together

* Handle EXC_ARITHMETIC on macos

* Update emit tests

* Update winch and tests
2023-03-16 00:18:45 +00:00
Benjamin Bouvier
2e6c7bf994 perf: Create a per-process JIT dump file (#6024) 2023-03-15 14:04:15 +00:00