Fix shadowing identified in #5322 for imul and swiden_high/swiden_low/uwiden_high/uwiden_low combinations in the x64 backend, and remove some redundant rules from the aarch64 dynamic neon ruleset. Additionally, add tests to the x64 backend showing that the imul specializations are firing.
The main change here is that io-lifetimes 1.0 switches to use the I/O safety
feature in the standard library rather than providing its own copy.
This also updates to windows-sys 0.42.0 and rustix 0.36.
Add assertions to the OperandCollector that show we're not using pinned vregs, and use reg_fixed_nonallocatable constraints when a real register is used with other constraint generation functions like reg_use etc.
* Clear affine slots when dropping a `Module`
This commit implements a resource usage optimization for Wasmtime with
the pooling instance allocator by ensuring that when a `Module` is
dropped its backing virtual memory mappings are all removed. Currently
when a `Module` is dropped it releases a strong reference to its
internal memory image but the memory image may stick around in
individual pooling instance allocator slots. When using the `Random`
allocation strategy, for example, this means that the memory images
could stick around for a long time.
While not a pressing issue this has resource usage implications for
Wasmtime. Namely removing a `Module` does not guarantee the memfd, if in
use for a memory image, is closed and deallocated within the kernel.
Unfortunately simply closing the memfd is not sufficient as well as the
mappings into the address space additionally all need to be removed for
the kernel to release the resources for the memfd. This means that to
release all kernel-level resources for a `Module` all slots which have
the memory image mapped in must have the slot reset.
This problem isn't particularly present when using the `NextAvailable`
allocation strategy since the number of lingering memfds is proportional
to the maximum concurrent size of wasm instances. With the `Random` and
`ReuseAffinity` strategies, however, it's much more prominent because
the number of lingering memfds can reach the total number of slots
available. This can appear as a leak of kernel-level memory which can
cause other system instability.
To fix this issue this commit adds necessary instrumentation to `Drop
for Module` to purge all references to the module in the pooling
instance allocator. All index allocation strategies now maintain
affinity tracking to ensure that regardless of the strategy in use a
module that is dropped will remove all its memory mappings. A new
allocation method was added to the index allocator for allocating an
index without setting affinity and only allocating affine slots. This is
used to iterate over all the affine slots without holding the global
index lock for an unnecessarily long time while mappings are removed.
* Review comments
This gets this package to build for `x86_64`, but does not include any
cache clearing for `aarch64-freebsd`. Which will probably build, but not work.
As far as I can tell membarriers are still in the process of being added.
See: https://reviews.freebsd.org/D32360
Since FreeBSD is mirroring the Linux membarrier syscall we may be able
to reuse the same implementation for both Linux and FreeBSD for AArch64.
There were several issues with ISLE's existing error reporting
implementation.
- When using Miette for more readable error reports, it would panic if
errors were reported from multiple files in the same run.
- Miette is pretty heavy-weight for what we're doing, with a lot of
dependencies.
- The `Error::Errors` enum variant led to normalization steps in many
places, to avoid using that variant to represent a single error.
This commit:
- replaces Miette with codespan-reporting
- gets rid of a bunch of cargo-vet exemptions
- replaces the `Error::Errors` variant with a new `Errors` type
- removes source info from `Error` variants so they're easy to construct
- adds source info only when formatting `Errors`
- formats `Errors` with a custom `Debug` impl
- shares common code between ISLE's callers, islec and cranelift-codegen
- includes a source snippet even with fancy-errors disabled
I tried to make this a series of smaller commits but I couldn't find any
good split points; everything was too entangled with everything else.
This commit refactors the internals of `wasmtime_runtime::SharedMemory`
a bit to expose the necessary functions to invoke from the
`wasmtime::SharedMemory` layer. Notably some items are moved out of the
`RwLock` from prior, such as the type and the `VMMemoryDefinition`.
Additionally the organization around the `atomic_*` methods has been
redone to ensure that the `wasmtime`-layer abstraction has a single
method to call into which everything else uses as well.
From the documentation of `CondVar::wait_timeout`:
> The semantics of this function are equivalent to wait except that the thread
> will be blocked for roughly no longer than `dur`. This method should not be
> used for precise timing due to anomalies such as preemption or platform
> differences that might not cause the maximum amount of time waited to be
> precisely `dur`.
Therefore, go to sleep again, if the thread has not slept long enough.
Signed-off-by: Harald Hoyer <harald@profian.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
* bench: add more WASI benchmarks
This follows up on #5274 to add several more scenarios with which to
benchmark WASI performance:
- `open-file.wat`: opens and closes a file
- `read-file.wat`: opens a file, reads 4K bytes from it, then closes it
- `read-dir.wat`: reads a directory's entries
Each benchmark is hand-crafted WAT to more clearly control what WASI
calls are made. As with #5274, these modules' sole entry point takes a
parameter indicating the number of iterations to run in order to use
`criterion`'s `iter_custom` feature.
* fix: reduce expected size of directory entries
* Cranelift: Define `heap_load` and `heap_store` instructions
* Cranelift: Implement interpreter support for `heap_load` and `heap_store`
* Cranelift: Add a suite runtests for `heap_{load,store}`
There are so many knobs we can twist for heaps and I wanted to exhaustively test
all of them, so I wrote a script to generate the tests. I've checked in the
script in case we want to make any changes in the future, but I don't think it
is worth adding this to CI to check that scripts are up to date or anything like
that.
* Review feedback
* feat: implement memory.atomic.notify,wait32,wait64
Added the parking_spot crate, which provides the needed registry for the
operations.
Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: change trap message for HeapMisaligned
The threads spec test wants "unaligned atomic"
instead of "misaligned memory access".
Signed-off-by: Harald Hoyer <harald@profian.com>
* tests: add test for atomic wait on non-shared memory
Signed-off-by: Harald Hoyer <harald@profian.com>
* tests: add tests/spec_testsuite/proposals/threads
without pooling and reference types.
Also "shared_memory" is added to the "spectest" interface.
Signed-off-by: Harald Hoyer <harald@profian.com>
* tests: add atomics_notify.wast
checking that notify with 0 waiters returns 0 on shared and non-shared
memory.
Signed-off-by: Harald Hoyer <harald@profian.com>
* tests: add tests for atomic wait on shared memory
- return 2 - timeout for 0
- return 2 - timeout for 1000ns
- return 1 - invalid value
Signed-off-by: Harald Hoyer <harald@profian.com>
* fixup! feat: implement memory.atomic.notify,wait32,wait64
Signed-off-by: Harald Hoyer <harald@profian.com>
* fixup! feat: implement memory.atomic.notify,wait32,wait64
Signed-off-by: Harald Hoyer <harald@profian.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
It seems they were mistakenly added to the `wasmtime_valunion` union whereas it is actually the `ValRaw` Rust type (represented by `wasmtime_val_raw`) that is affected by the change.
This commit replaces `wasmtime_environ::TrapCode` with `wasmtime::Trap`.
This is possible with past refactorings which slimmed down the `Trap`
definition in the `wasmtime` crate to a simple `enum`. This means that
there's one less place that all the various trap opcodes need to be
listed in Wasmtime.
In #5174 we decided it doesn't make sense for a rule to have a
bind-pattern at the root of its left-hand side. There's no Rust value
corresponding to the root value of such a term, because it actually
represents a function declaration with one or more arguments.
This commit takes that to its logical conclusion.
`sema::Rule` previously had an `lhs` field whose value must always be a
`Pattern::Term` variant, and anyone using that structure had to deal
with the possibility of finding the wrong variant there.
Now the relevant fields from that variant are stored directly in `Rule`
instead. Also, the (tiny!) portion of `translate_pattern` which applied
when the pattern was the root term is now inlined in `collect_rules`.
Because `translate_pattern` no longer has to special-case the root term,
we can delete its `rule_term` and `is_root` arguments. That brings it
down to a more manageable four arguments, which means many calls fit on
one line now.
This hopefully will speed things up slightly since ASan can add a chunk
to the compile time. Unsure if this will actually help, though, so let's
find out!
As it turns out, that distinction was not necessary for this
representation. Removing it eliminates some complexity around wrapping
expressions as bindings and vice versa. It also clears up some confusion
about which category to put certain constructs in (arguments and
extractors) by refusing to have different categories.
While I was writing this patch I also realized that `add_match_variant`
and `normalize_equivalence_classes` both need to do fundamentally the
same things with enum variants, so I refactored them to share code and
make their relationship clearer.
Finally, I reviewed all the comments in this file and fixed some places
where they could be more clear.
* convert wasi-common from defining its own error to using wiggle trappable error
* wasi-common impl crates: switch error strategy
* wasmtime-wasi: error is trappable, and no longer requires UserErrorConversion
* docs
* typo
* readdir: windows fixes
* fix windows scheduler errors
fun fact! the Send and Recv errors here that just had a `.context` on
them were previously not being captured in the downcasting either. They
need to be traps, and would have ended up that way by ommission, but
you'd never actually know that by reading the code!
* Shrink the size of SigData in Cranelift
* Update cranelift/codegen/src/machinst/abi.rs
Co-authored-by: Jamey Sharp <jamey@minilop.net>
* Change ret arg length to u16
* Add test
Co-authored-by: Jamey Sharp <jamey@minilop.net>
Avoid naming %rcx as written by the CoffTlsGetAddr pseudo-instruction in the x64 backend, and instead emit a fixed-def constraint for a fresh VReg and %rcx.
Remove some unnecessary moves in the x64 gen_memcpy implementation -- the call instruction that's generated will already constrain the args to those registers.
* Add a new "trappable" mode for wiggle to make an error type
start refactoring how errors are generated and configured
put a pin in this - you can now configure a generated error
but i need to go fix Names
Names is no longer a struct, rt is hardcoded to wiggle
rest of fixes to pass tests
its called a trappable error now
don't generate UserErrorConversion trait if empty
mention in macro docs
* undo omitting the user error conversion trait when empty
* Remove explicit `S` type parameters
This commit removes the explicit `S` type parameter on `Func::typed` and
`Instance::get_typed_func`. Historical versions of Rust required that
this be a type parameter but recent rustcs support a mixture of explicit
type parameters and `impl Trait`. This removes, at callsites, a
superfluous `, _` argument which otherwise never needs specification.
* Fix mdbook examples
In order to properly understand the impact of providing thread-safe
implmentations of WASI contexts (#5235), we need benchmarks that measure
the current performance of WASI calls using Wiggle. This change adds
several common WASI scenarios as WAT files (see `benches/wasi/*.wat`)
and benchmarks them with `criterion`. Using `criterion`'s `iter_custom`,
the WAT file runs the desired number of benchmark iterations internally
and the total duration of the runs is divided to get the average time
for each loop iteration.
Why WAT? When compiling these benchmarks from Rust to `wasm32-wasi`, the
output files are large, contain other WASI imports than the desired
ones, and overall it is difficult to tell if we are measuring what we
expect. By hand-writing the WAT, it is (slightly) more clear what each
benchmark is doing.
* cranelift: Cleanup `fdemote`/`fpromote` tests
* cranelift: Fix `fdemote`/`fpromote` instruction docs
The verifier fails if the input and output types are the same
for these instructions
* cranelift: Fix `fdemote`/`fpromote` in the interpreter
* fuzzgen: Add `fdemote`/`fpromote`
`wiggle` looks for an exported `Memory` named `"memory"` to use for its
guest slices. This change allows it to use a `SharedMemory` if this is
the kind of memory used for the export.
It is `unsafe` to use shared memory in Wiggle because of broken Rust
guarantees: previously, Wiggle could hand out slices to WebAssembly
linear memory that could be concurrently modified by some other thread.
With the introduction of Wiggle's new `UnsafeGuestSlice` (#5225, #5229,
#5264), Wiggle should now correctly communicate its guarantees through
its API.
We can encode more constants into 12-bit immediates if we do the following
rewrite for comparisons with odd constants:
A >= B + 1
==> A - 1 >= B
==> A > B
This commit refactors the internals of `wiggle` to have fewer raw pointers and more liberally use `&[UnsafeCell<_>]`. The purpose of this refactoring is to more strictly thread through lifetime information throughout the crate to avoid getting it wrong. Additionally storing `UnsafeCell<T>` at rest pushes the unsafety of access to the leaves of modifications where Rust safety guarantees are upheld. Finally this provides what I believe is a safer internal representation of `WasmtimeGuestMemory` since it technically holds onto `&mut [u8]` un-soundly as other `&mut T` pointers are handed out.
Additionally generated `GuestTypeTransparent` impls in the `wiggle` macro were removed because they are not safe for shared memories as-is and otherwise aren't needed for WASI today. The trait has been updated to indicate that all bit patterns must be valid in addition to having the same representation on the host as in the guest to accomodate this.
